beta

← Writing 7 min read

Firecracker on WSL2, without root

A microVM per coding agent on a Windows laptop, and ten things that did not behave as I expected.

.md

My coding agents run code I did not write: dependencies, test suites, scripts in repositories I cloned an hour ago. The usual sandbox shares the host kernel. That holds as long as the kernel holds.

I wanted each agent to get a kernel of its own: a small VM per run, gone when the run ends. On the machine I actually work on, a Windows laptop whose Linux is WSL2. And without root while it runs.

Firecracker, nested inside WSL2, does it. A minimal VM is ready in under 0.2 seconds; a full agent VM in about one. In the first two weeks, my agents ran nearly a thousand times, each run in a VM of its own. Getting there took ten surprises.

Two caveats up front: Firecracker does not support WSL2, and so far I have attacked the boundary only from a guest without root. Nothing here is specific to WSL2 but the nesting and the missing seat; on Debian or Ubuntu it should run the same, though I have not tested it there, and Ubuntu 24.04 and later restrict the unprivileged user namespaces it needs.

The stack

The stack around each agent's VM Nested from the outside in: Windows and Hyper-V; WSL2 with /dev/kvm, so every VM is nested; a systemd user scope in a slice of its own, whose memory, swap, task and CPU caps are read back before the start; a bubblewrap jail under a subordinate uid, with every namespace, no capability, a read-only root and kvm as its only group; Firecracker, without a network interface; inside it the guest with its own kernel and the agent. The guest's one way out is vsock, to a broker in WSL2 outside the scope, which dials the one host each port stands for. Windows and Hyper-V the WSL2 VM WSL2: Debian 13, kernel 6.18 /dev/kvm: every VM is nested systemd scope, own slice limits, read back before the start bubblewrap jail, subordinate uid no capability, read-only root Firecracker no network; one vsock device the guest: own kernel, the agent runs code nobody checked vsock broker one port per host; dials that host only
Figure 1. Each layer is a separate boundary. The guest's only way out is vsock, to a broker outside the scope.

WSL2 exposes /dev/kvm, so every Firecracker VM is nested. A systemd user scope in a slice of its own caps memory, swap, tasks and CPU. A bubblewrap jail gives Firecracker every namespace, no capability, a cleared environment and a read-only root with little more than its binary, the kernel and the image. It runs as a uid from my /etc/subuid range, as rootless Podman does, with kvm as its only group: it can open /dev/kvm, and none of my files. The guest has no network interface. Its one cable is vsock, and on the host end a broker dials the one host each port stands for.

Root is needed for two commands, once, at setup:

sudo usermod -aG kvm "$USER"    # then log in again
echo "$USER:$(getent group kvm | cut -d: -f3):1" | sudo tee -a /etc/subgid

How fast

The smallest guest I could write: a 16 MiB image holding only an /init that says ready over vsock. One vCPU, 128 MiB, twenty boots per configuration:

Time to ready, twenty boots each Twenty boots each, from systemd-run to the guest's ready line over vsock. CI kernel, my first boot args: 638 to 688 ms, median 669.5; the same, with the i8042 ones: 157 to 196 ms, median 178.5; the same, jailed: 172 to 213 ms, median 191.5; my own kernel, no keyboard driver: 147 to 208 ms, median 170. Time to ready, twenty boots each CI kernel, my first boot args 670 ms the same, with the i8042 ones 178 ms the same, jailed 192 ms my own kernel, no keyboard driver 170 ms 0 250 500 750 ms from systemd-run to ready; bar: median
Figure 2. Measured on 10 November 2026, one boot at a time, each in its capped scope.

The jail costs nothing measurable, and Firecracker v1.18.0 boots as fast as v1.17.0. The half second in the first row is a keyboard. Firecracker's CI kernels have the AT keyboard driver, and Firecracker's default kernel command line keeps it from probing for a keyboard that is not there. Any boot_args replace that line whole: mine did, and so does the example in Firecracker's getting-started guide, which then waits about half a second on every boot. Keep i8042.noaux i8042.nomux i8042.dumbkbd in your own. My kernel, built from the configuration in Firecracker's repository, has no keyboard driver at all, so it never waited.

Ten stumbling blocks

Access

1. An ACL on /dev/kvm vanishes. Firecracker's getting-started guide suggests setfacl -m u:${USER}:rw /dev/kvm. My first VM booted, the next nineteen got "Permission denied". Debian's udev rules hand kvm to whoever sits at the seat, and each new VM makes udev reapply that rule. WSL has no seat, so my entry was simply erased. Use the kvm group. (My note on the cause.)

2. sg kvm bites. It makes kvm the primary group, which hides the process's /proc entries from processes of other groups and breaks newuidmap. Log in again instead; then kvm is a supplementary group.

Limits

3. A memory cap that was not there. systemd-run --user -p MemoryMax=64M in the default app.slice once let a process take 200 MiB. No error: the scope simply had no memory.max. After a fresh WSL start it worked, and I have not been able to reproduce it. Read the limit back before starting anything, and stop if it is not what you asked for. The process in the scope can do it itself: 

systemd-run --user --scope --quiet --slice=agents.slice \
  -p MemoryMax=256M -p MemorySwapMax=0 -- \
  sh -c 'cat "/sys/fs/cgroup$(cut -d: -f3 /proc/self/cgroup)/memory.max"'

It prints 268435456 when the cap is in force, and max or an error when it is not.

4. CPU is unbounded. The emulated serial port costs a VM exit per byte; a chatty guest can keep two cores busy. Set CPUQuota= on the scope. (A pull request to batch serial I/O, firecracker#6327, is open.)

5. An OOM kill takes the whole scope. When the guest blew its cap, systemd stopped everything in the scope, my watchdog included. Only the scope's Result=oom-kill tells what happened.

The jail

6. bwrap creates mount points as 0700. Under a subordinate uid, Firecracker could not enter the directory holding its own binary. Make it 0755 explicitly.

7. No rule needed against nested namespaces. I was about to write one, then read Firecracker's seccomp filters: none of its threads may call unshare, clone, clone3 or setns. Firecracker already forbids it.

The cable

8. A message can die with the guest. My first /init wrote ready and restarted the VM; the host never got the line. Wait for an answer before treating a write as delivered.

9. Half-close does not cross vsock. A guest's shutdown(SHUT_WR) never reaches the host, and the host's ends the whole connection for the guest. Protocols that rely on half-close wait for a timeout. End the whole connection when either side ends. (A fix is proposed in firecracker#6349; the docs say nothing about it.)

Trust

10. Checksums, no signatures. Releases ship SHA-256 sums, not signatures. A maintainer confirmed that the team signs them internally but leaves the signature out (firecracker#6117, open since August). Pin the hashes, so that every update is a deliberate change.

What it became

The spike took a day. Since then it has grown into d0, short for DeckZero, the tool I run my coding agents with every day. It is not public yet; these parts of it came from the ten blocks:

  • A helper inside each scope reads back the memory, swap, task and CPU limits and refuses to start Firecracker if one is missing (blocks 3 and 4).
  • The kernel is built from source; the build stops if an option on my list is unknown or not in force.
  • One vsock port per allowed host; the broker never dials a private, loopback or link-local address, and connections end whole (block 9).
  • Every live VM gets its own subordinate uid, and the host never reads the agent's disk: work comes back as a git bundle.

Attacking it

Before writing this, I turned the guest against each boundary: code with the agent's rights, no root, in a real VM. It tried the internet, the metadata address and my WSL's own IP; all 65,535 vsock ports of the host; the broker with a foreign TLS name, no name, a plain CONNECT, a silent connection and a hundred at once; a memory bomb, a fork bomb, a full disk, busy loops, a command that ignores every signal, a process left behind, and terminal escapes in its output. Each boundary held; one only after a fix.

My tool kept a vsock port open from the start for the agent's work to come back on, and took the first connection to it. A process in the guest could connect early and hand the host a bundle of its own, past the point where everything else in the VM is killed. Now the work comes back on the connection the guest's init opened at boot, which nothing else in the guest can reach, and the port is gone.

Not attacked: a guest with root, as after a kernel exploit. One machine, nested KVM, this WSL version. And not Firecracker's own way: its supported confinement, the jailer, needs root.

Others on the same road

The parts are not new. Kastellan jails Firecracker with a user scope and bubblewrap, izba runs Cloud Hypervisor VMs without a network interface, with vsock egress, and names WSL2, and Tracekit Box gives each agent job a Firecracker VM with brokers over vsock. The combination, on WSL2 under a subordinate uid, I did not find.

Try it

The reduced version, with nothing of my tool in it: firecracker-wsl2-rootless. It fetches Firecracker and a CI kernel against pinned hashes, boots twenty jailed VMs, and checks the running jail from the host: seven namespaces, no capability, the subordinate uid, seccomp on every thread, the read-only root, loopback only, and that the jail cannot read my files. Run from a fresh clone exactly as its README says: twenty of twenty, each ready in under a quarter of a second.

Written with the help of AI; the thinking and the final say are mine.