Build Standalone Lua Applications with Zig and fnlpack

How small is technomancy/search? In its earliest form, its application code fit in a single social media post. As a small application, it doesn't need anything fancy to run. It'll run on a computer that's not powerful, it does not require that you pay any money or take a training course. It's pretty uncomplicated and straightforward to operate.

Still, like many small applications, search requires some software operations knowhow to install. It has three components:

1. Native libraries that handle data, networking, and events; these are widely available from software package repositories.
2. Application code written in the Fennel programming language. This can be downloaded from the project's website.
3. Database query data, in the form of text files that come with the application source code.

The project's official install instructions suggest using APT, the software installer from the Debian project, to install the Fennel compiler and the necessary native libraries. If you happen to be using Debian or one of its descendants, like Ubuntu, then you already have APT installed. But not all Linux systems have APT, and not all systems where you might want to run search are Linux systems.

I set out to build a standalone binary because I want a simple way to deploy a Fennel application: transfer a single file containing the program, and run it. That route is usually associated with lower-level programming languages like C, golang, and Rust. Fennel occupies a sweet spot where you can fit a search engine into a social media post, and now with the tools I've built, you can also ship it as a standalone binary.

technomancy/search depends on the sqlite3 database engine, the Lua runtime, and the lsqlite3 and lua-http libraries. These in turn depend on other native libraries like openssl, cqueues, and lpeg. Each of these projects has its own build system: some use cmake, others make, others autotools. This adds a lot of complexity to set up the libraries we need to run our small app.

To reduce this complexity, I turned to Zig, a set of tools including a programming language, compiler and build system that I could use to build all the native libraries I need and bring them together. Zig is capable of building software written in the C programming language, which includes everything we need for search's native libraries. One of the projects in the Zig space is "All Your Codebase Are Belong To Us", a repository of useful Free software that can be built using Zig. Using that as a starting point, I created Zig build recipes for sqlite3, lsqlite3, openssl, luaossl, cqueues, and lua-http. Some of those are barely tweaked from existing All Your Codebase recipes, but others required more research, and the lua-http recipe in particular taught me a lot about the Zig build system.

With all the native dependencies primed with Zig build recipes, I was able to create a final Zig recipe for search itself. It's about a hundred lines of Zig code and worth reading, even if you are not familiar with Zig yet. In my opinion Zig builds are clearer and easier to follow than Makefiles. The outcome of that build is a static binary that contains search's Lua runtime and all its native libraries. This binary also includes a bundle containing all the data and application code it needs to run, which I accomplished using a new tool I authored. That's covered in the next section.

In main.zig we embed the module preload pack like so:

  const std = @import("std");
  const c = @cImport({
      @cInclude("lua.h");
      @cInclude("lauxlib.h");
      @cInclude("lualib.h");
  });
  const lua_http = @import("lua-http");
  const technomancy_search_pack =
      @embedFile("./technomancy-search.lua");
  //  ^^ basically just another import ^^

…and then once we've got our interpreter set up, we load it:

  try lua.runString(technomancy_search_pack);
  // ^^ preload all Lua & Fennel modules ^^
  try lua.runString(
      \\require("main")
  );

To make the application code and data easy to bundle together, I created fnlpack. It takes directories containing Lua and Fennel source code and turns them into a single Lua file that embeds a virtual filesystem with all the Lua code and data necessary to run the application, so long as the native libraries are in place.

When we combine these two components, the bundle created by fnlpack and the native binary created by Zig, we produce the desired outcome: a single standalone binary that we can copy to any Linux system and run.

Lua hacker Sophie "sOwOphie" wrote a primer on bundling Lua:

[A bundler concatenates] a number of Lua modules into a single script, which can be deployed easily without having to worry about LUA_PATH. […]

[This] is achieved through the use of the package.preload table. It can be filled with functions that return the corresponding module when called. The built-in load function is the perfect fit for this. It takes Lua code in string form, plus a source name, and returns a function that executes said code. […]

[The bundler can] get around string escaping issues with Lua "literal string" literals. A literal string begins with [=[, with any number of = characters. It continues until a corresponding ]=] with the same number of = characters. Inside the literal string, escapes are not processed, everything is taken verbatim.

I didn't read her primer before beginning my implementation, but the technique I ended up using matches her description pretty much exactly, so I recommend reading her full primer to gain understanding of the technique.

The main thing that I added to fnlpack which is not described there, beyond adding support for Fennel source code in addition to Lua, is my virtual filesystem implementation. I describe it like this in the README for fnlpack (under "Expand to view VFS detail"):

You can use the virtual file system (VFS) to locate packed files at runtime.

Modules packed with fnlpack will already be preloaded in your module cache, so you do not need to use the VFS at all unless your application needs to read those files again during operation.

All files in your pack are available as strings at runtime. You can find them in the table __fnlpack.files which maps file paths → file contents. To find which path corresponds to a given Lua module, inspect the table __fnlpack.modules which maps Lua module names → file paths.

The flags --patch-dofile and --patch-io trigger inclusion into your pack of patches for dofile + fennel.dofile, or io, respectively, to make them find files in the VFS when they would otherwise fail because they cannot find a file. In cases where a file can be found at runtime, the default behavior applies. You do not need to use either flag to enable the VFS, they are only for convenience in cases where you want to make unmodified code use the VFS implicitly.

This allows fnlpack users the flexibility to build bundles that include not just source code but all the data necessary to run an application, and a convenient adapter to make Lua's native io APIs work with that data. technomancy/search was a useful practical demonstration of how this tool can simplify deployment of existing Fennel/Lua applications.

To build the bundle for search, we invoke:

  fnlpack --add-requires \
    --patch-dofile \
    --patch-io \
    --no-fennel-sandbox \
    --pack sql \
    . >/opt/build/dist/technomancy-search.lua

Before we do that, though, we run a zig build which installs our source files into the build target directory:

  const lua_http_lib = lua_http_dep.artifact("lua-http");
  const install_lua_http_scripts = b.addInstallDirectory(.{
      .source_dir = lua_http_dep.namedWriteFiles("lua-http")
          .getDirectory(),
      .install_dir = .prefix,
      .install_subdir = "share/lua/5.4",
  });

  const search_scripts =
      b.addNamedWriteFiles("technomancy-search");
  _ = search_scripts.addCopyDirectory(
      b.path("src"),
      ".",
      .{ .include_extensions = &.{ ".fnl", ".lua" } },
  );
  _ = search_scripts.addCopyDirectory(
      b.path("sql"),
      "sql",
      .{ .include_extensions = &.{".sql"} },
  );
  _ = search_scripts.addCopyDirectory(
      b.path("static"),
      "static",
      .{},
  );
  const install_search_scripts = b.addInstallDirectory(.{
      .source_dir = search_scripts.getDirectory(),
      .install_dir = .prefix,
      .install_subdir = "share/lua/5.4",
  });

The namedWriteFiles("lua-http") are files that we put into a "named files" box during the lua-http build, anticipating wanting to refer to them in downstream builds:

  const lua_scripts = b.addNamedWriteFiles("lua-http");
  _ = lua_scripts.addCopyFile(
      fifo_dep.path("fifo.lua"),
      "fifo.lua",
  );
  _ = lua_scripts.addCopyFile(
      basexx_dep.path("lib/basexx.lua"),
      "basexx.lua",
  );
  _ = lua_scripts.addCopyDirectory(
      lpeg_patterns_dep.path("lpeg_patterns"),
      "lpeg_patterns",
      .{},
  );
  // and so on...

So this sets up the final assembly:

1. Zig builds each dependency, including lua-http, which puts its various Lua modules it depends upon into its "lua-http" named files box
2. We use fnlpack to pack all those scripts, together with our own Fennel sources and SQL data, into a module preload pack
3. Zig inserts that preload pack into the binary, along with Zig code that loads it at runtime

Before we hatch a chicken we need an egg, and before we run a Zig build we need a Zig toolchain. Where do we get those tools from, and how did those tools get built, and so on down the chain?

This is known as the software bootstrapping problem, where your working set needs to pull itself up by its bootstraps. To solve this problem for myself, I used GNU Guix, which can provide all the tools we need for fnlpack's build and the ones we need for search itself. Since 2023, Guix has had a full-source bootstrap process, rooted in a tiny binary program called hex0 that allows every subsequent binary, all the way up to Zig, fnlpack and technomancy/search, to be built from source code.

This provides transparency and reproducibility. If you and I use Guix at the same revision to install the build toolchain and then run the same Zig build, we'll end up with exactly the same result, and the source code is visible and auditable all the way down to the bones.

At time of publication, my Guix revision is as follows:

$ guix describe

Generation 1337	Oct 25 2026 15:06:35	(current)
  guix b3a2b7c
    repository URL: https://git.guix.gnu.org/guix.git
    branch: master
    commit: b3a2b7cbe5113e36880a20738a4b148969e93005

The standalone binaries I'm compiling will run on any recent Linux system, but I want to build binaries for other Zig-supported platforms, including NetBSD and proprietary operating systems.

And why do we need an operating system at all to run a small application like technomancy/search? Unikraft succeeded at creating a Lua unikernel. The Mewz unikernel project is designed to run WebAssembly programs, and Zig has a WebAssembly target. tw4452852 has examples of building Zig programs for Unikraft and for the seL4 microkernel. These are exciting research opportunities hinting at the possibility of a "Boot-to-Fennel" application stack that combines easy high-level application development with a minimalistic deployment strategy.

I chose technomancy/search, a Fennel program, for this learning project, plus Zig and Guix, because I like that they share these traits in common:

• Their designs aim for simple, efficient, reusable, maintainable software, not growth for growth's sake or web-scale buzzword compliance. This is what I want to see from the tools that I rely upon. I sometimes aim to create exciting and novel things, but building on a foundation that prioritizes simplicity and maintainability frees up more energy for novelty higher in the stack.
• They are Free software. Software must give users control and respect their freedom and privacy. This is an absolute prerequisite for the user to control the computer, and not the other way around.
• Furthermore, they are hostile to AI development: Zig, Guix, and Fennel all strictly prohibit contributions made using LLMs. This is important both because AI-authored software is ethically unacceptable, and because the projects that embrace LLM-generated code are getting onto the vibe bobsled (as coined by Christine Lemmer-Webber) and, by stages, become passengers along on their software's vibe journey rather than authors and engineers.

The outcome is a step towards a #neverslop software stack that's expressive enough to fit a search engine in a social media post & deploys anywhere by copying a file. I can work with that.