skip to main content

The Jamstack Build Pipeline Reimagined

published icon  |  category icon webdesign

As a follow-up to my earlier overview, Finding Stuff on Big Blogs, I decided to rewrite bits and pieces of Brain Baking by stealing great ideas from others listed in that post—in true Austin Kleon style.

The main thing that changed, besides the You Might Also Like… list below each post containing both backlinks and articles related by tag, is the contents of the index page which now contains three lists:

  1. Freshly Baked—the last five bakings a.k.a. blog posts.
  2. Best Sellers—the buns that fly off the shelves a.k.a. most popular posts.
  3. Random Treats—five randomly served posts to spice things up a little.

I contemplated adding a Personal Favorites but then I’d have to periodically curate that list and probably forget about it anyway. Too much friction involved.

The first and last are easy enough to achieve in Hugo: range first 5 and shuffle. There, done. But how can I retrieve those best sellers when Hugo, as a static site generator, has zero knowledge of this data? Jim Nielsen does so by fetching data from Netlify. This site uses GoatCounter to gather some basic web analytics without cookies or tracking data, so I came up with a script that queries the GoatCounter DB and converts that into JSON1. I’ve also opened up the page just for fun—go nuts. While Jim lists hits in absolute numbers and Hacker News highlights, I think it’s very dangerous to focus on that and went with a relative percentage instead. (Okay, I’ll admit: my “popular this month” pageviews net less than 1K. But who cares, right?)

Woah there. Back up a bit, how does that dynamic data make it into the “Jamstack”, a fancy name for a set of Javascript and API tools a static site such as this one runs on? By adding pre- and post-processing steps. For instance, Pagefind is a post-processing step, and so is sending Webmentions.

The idea is as follows:

  1. Write—You write stuff in Markdown. Great! High five! But that .md won’t cut it—it needs to be converted into .html and served.
  2. Precompile—Run custom scripts to generate extra data needed to create HTML files, such as the popular posts or a list of webmentions in JSON format or the conversion of YouTube links to images, allowing Hugo to correctly interpret these.
  3. Compile—Use Hugo, Eleventy, Jekyll, whatever to build a folder of files that a web server can serve to your visitors. Great! Now you have a docs/ dir full of HTML and CSS! Now what?
  4. Predeploy—Run custom scripts to generate extra raw files needed to deploy and host without having them parsed by your generator. Files such as Pagefind’s search index fragments.
  5. Deploy—Copy that folder to your server into /var/www and point a web server like Nginx, Caddy, whatever to that folder. Great, now people can surf to your domain and view the site!
  6. Postdeploy—Run custom scripts that should trigger once the site is up-to-date. Scripts such as sending out new Webmentions, parsing your updated RSS feed, sending a mail, whatever.

Compared to dynamic website builders such as Wordpress, Jamstacks are quite simple: it’s only HTML/CSS/JS. That can be quite frustrating if you want to inject data that lives outside of your repository. By enriching the source and destination folders, you can still exert control and make things “dynamic”, even though they’re not. The “random treats”, for example, doesn’t ever change, unless Hugo rebuilds the site—which is triggered once an hour if there’s a new commit in the repository.

I know this all sounds awfully technical, and it probably is. But don’t let the limitations of a typical Jamstack approach be a burden: just hack your way around it. Most of the above steps are optional: only write, compile, and deploy are required and can be simple one-liners.

The above steps simply get executed by calling upon bun—you are using the much faster bun instead of yarn or npm now, right? bun compile adheres to the Npm lifecycle and automatically runs scripts with pre before and post afterwards. By keeping the commands inside my package.json, each of my websites gets to do other custom stuff on each of the above steps, without having to change the deployment procedure.

For every script or custom hack I paste onto this pipeline, I first think: is this really needed? By using the default build pipelines, I’ve actually cleaned up the mess and made things less complex than before, although that perhaps isn’t entirely obvious.

Remember: Reducing Your Workflow Load Facilitates Writing! If it takes a few battered-together scripts, then it takes a few scripts, who cares!

  1. If you’re interested in the details, see↩︎

tags icon website

I'm Wouter Groeneveld, a Brain Baker, and I love the smell of freshly baked thoughts (and bread) in the morning. I sometimes convince others to bake their brain (and bread) too.

If you found this article amusing and/or helpful, you can support me via PayPal or Ko-Fi. I also like to hear your feedback via Mastodon or e-mail. Thanks!