Hacking Together htmlwidgets for Jekyll | Ben Cunningham

Hacking Together htmlwidgets for Jekyll

Written by Ben Cunningham · on June 13, 2016

This morning marks the end of my foolhardy, week-long adventure that was migrating this site to Jekyll. If you are unfamiliar with the software, Jekyll is basically just a static site generator — but a very popular one, at that. Instead of using databases, it takes content (usually written in Markdown) and renders static pages, dynamically including all of the bells and whistles (boilerplate headers, footers, etc.) you might find in templated server-side sites.

This concept might sound somewhat familiar if you’re one of the many people who already heavily integrate R Markdown and Knitr into their workflow. You markup a report (complete with embedded R chunks) and, after some options setting and YAML front matter, let the magic of Knitr weave you a standalone, human-consumable document.

Fortunately, if you’re already comfortable with this workflow, there’s almost nothing left you need to learn before jumping in. Check out any number of tutorials on installing the right Ruby gems, and take a look at Yihui Xie’s servr package for serving files directly from R. On the other hand, if you would prefer a bit more guidance, check out Brendan Rocks’s up-and-running tutorial or Yihui Xie’s Jekyll + Knitr repo for minimal examples of building a site that compiles R Markdown posts.

In fact, if you want to read about my preferred solution to the problem discussed below, read Brendan’s other post, Using htmlwidgets with Knitr and Jekyll. His fix is much more elegant, so if you find yourself pulling out hairs trying to get R-produced JavaScript frames to display on your site, I hope it works for you.

But for whatever reason, I just couldn’t seem to wrap my head around its dependency soothsaying. So here we go…

The Problem

It’s becoming fairly popular in the R community to build inline interactives, a la Leaflet, MetricsGraphics, and three.js. Case in point, all three of those libraries have been brought to R via the htmlwidgets package. But here lies the impetus of our troubles: widgets built with these tools all ultimately rely on JavaScript dependencies. A lot of legwork has been done to make their use straightforward with tools like R Markdown and Shiny, but when we throw a Jekyll workflow into the mix, stuff starts to break.

When servr::jekyll() knits an R Markdown post (usually originating in _source) to Markdown (usually output to _posts), it does just that and nothing more. Jekyll itself of course renders those Markdown posts to HTML ones, but what you see in the Markdown is what you get in the HTML content. This means that, since Markdown doesn’t handle JavaScript substance in the way that, say, R Markdown v2 or HTML do (which is to say, the Markdown renderer usually strips it and replaces it with an error), the JavaScript content never even makes it through to the Jekyll-led conversion to HTML.

In short, my beloved Leaflet maps are being excluded from blog posts before they even have a chance!

Brendan’s fix takes advantage of Jekyll’s ability to dynamically include HTML fragments in other HTML. His post (here it is again) covers this in more detail, but he essentially presents a few helper functions to grab JavaScript/CSS dependencies and frames, keep them around through the Knitr stage, and prepare them for inclusion when core Jekyll takes over.

All this being said, keep in mind that I may have made a critical mistake in my understanding of his system. After all, I just couldn’t seem to make it work with my site; no matter how many times I tried, dependencies never got copied over and the JavaScript content went missing along the way. Alas, the culprit must be some oversight of my own — based on the response I’ve seen, his method really worked for a lot of people!

For more discussion of this grand Jekyll + htmlwidgets problem, and a look at some other proposed alternatives, see this issue.

A Hacky Alternative

One solution might have been to just build my maps elsewhere, rendered as self-contained HTML, and then put a Jekyll-style include in the R Markdown file where I really wrote my post. But if there’s anything being around full-time R users has taught me, it’s that I ought to demand reproducibility as often as I can get it.

Instead, I decided that I would put the whole post, entirely reproducible, back in _R (where I keep each project’s analysis and data). This has all of my post content, written exactly as it normally is, but with the YAML header reduced to:

---
output:
  html_document:
    theme: null
    self_contained: true
---

At the same time, I add a filler R Markdown post in _source, except it only has a YAML header (but with the normal title, tags, etc.) and a single include linking to what will be the HTML output of my post. It looks something like this:

---
layout: post
title: "The Fastest Bar Crawl"
description: "A speedy walk through Iowa City."
date: "January 15, 2016"
---

{% include htmlwidgets/the-fastest-bar-crawl.html %}

Now we just need to get the rendered HTML to where the include says it will be. To do this automatically, I wrote the following functions:

knit_htmlwidgets <- function(input,
                             output_dir = "./_includes/htmlwidgets",
                             ...) {

  file_name <- rev(unlist(strsplit(input, split = "/")))[1]
  path <- rmarkdown::render(input, "html_document", output_dir = output_dir)
  remove_doctype(path)

}

remove_doctype <- function(input) {

  html_lines <- readLines(input)
  keep <- grep("^<!DOCTYPE html>$", html_lines, invert = TRUE)
  writeLines(html_lines[keep], input)

}

Unfortunately, if we just render the file and include it, the <!DOCTYPE html> tag gets copied into the post as normal body text. To avoid this, remove_doctype() strips it out (like I said, pretty hacky). The rest of knit_htmlwidgets is pretty straightforward, though: we’re just rendering the input file as HTML and placing it in the directory _includes/htmlwidgets so that `

Starting tonight, my radio co-host is reliving a timeless adventure from her father’s university days — hitting up every bar in Iowa City in one weekend. In honor of the feat, and motived by this post by my professor Sam Burer, I decided to try to find the path that minimizes walking time between the spots on her list.

My repertoire of optimization techniques is decidedly limited, but considering the stakes, I reckoned that hill climbing (and an attempt at simulated annealing) would do. Using the R package ggmap, I was able to geocode the list of bars and scrape walking directions between each establishment. Summarizing these directions, I built a cost matrix relating the estimated time to travel between points. Finally, after piecing together the local search code, I mapped out the weekend program with leaflet.

I doubt that this is a global solution, but hopefully it will save her crew a bit of walking.

## Estimated walking time: 39M 10S
##                            Bar             Address
## 1       Dave's Fox Head Tavern     402 E Market St
## 2              George's Buffet     312 E Market St
## 3             IC Ugly's Saloon       210 N Linn St
## 4                  EDEN Lounge        217 Iowa Ave
## 5               Blue Moose Inc        221 Iowa Ave
## 6                   Yacht Club        13 S Linn St
## 7                Sports Column     12 S Dubuque St
## 8              Deadwood Tavern      6 S Dubuque St
## 9           Dublin Underground      5 S Dubuque St
## 10 808 Restaurant & Night Club        121 Iowa Ave
## 11                 Joe's Place        115 Iowa Ave
## 12                      Summit     10 S Clinton St
## 13  Clinton Street Social Club 18 1/2 S Clinton St
## 14                The Airliner     22 S Clinton St
## 15             Pints Iowa City    118 S Clinton St
## 16              Donnelly's Pub    110 E College St
## 17               The Union Bar    121 E College St
## 18         The Mill Restaurant 120 E Burlington St
## 19        Brothers Bar & Grill    125 S Dubuque St
## 20             DC's Sports Bar    124 S Dubuque St
## 21        Fieldhouse Iowa City    118 S Dubuque St
## 22            Forbidden Planet    111 S Dubuque St
## 23        Quinton's Bar & Deli 215 E Washington St
## 24                  Gabe's Inc 330 E Washington St
## 25           Iowa City Brewlab 505 E Washington St
## 26                      Bardot    347 S Gilbert St
## 27                  Sams Pizza    441 S Gilbert St
## 28               Sanctuary Pub    405 S Gilbert St
## 29        Vine Tavern & Eatery   330 E Prentiss St

` can find it.

So after you we run knit_htmlwidgets on our input file, we can build the site as usual (most likely with server::jekyll()).

Up and Running

To recap, we write a post, render it with knit_htmlwidgets, create a placeholder post in _source, and serve. On Jekyll’s end, the placeholder gets converted to Markdown and our post gets included as that Markdown is converted to HTML.

If you have a personal R package, that might be a good place to keep the helper functions. For posterity, I’ll be sure to add them to mine soon too.

While certainly not elegant, I’m happy this got me back up to speed with Leaflet. All the same, feel free to shoot me an email about what I got wrong the first time through!