Netboot NixOS from u-root in QEMU
The goal is to netboot a Linux distribution from u-root. This tutorial uses NixOS, but the approach applies to any distribution that publishes a netboot image and iPXE script (which you can create manually).
The obstacle is networking. The u-root demo with QEMU
boots a prebuilt initramfs and an Arch kernel, but dhclient finds no
interface: the Arch kernel ships every NIC driver as a loadable module and the
stock u-root image carries none. To get around this, this demo adds the e1000
module to u-root without repacking it or building a kernel. In production, you
should use a custom kernel with built-in network drivers.
Get the u-root image and kernel
You start with the same two downloads as the u-root demo with QEMU. You can skip this step if you’ve already done it:
curl -L -o u-root.cpio.xz https://github.com/linuxboot/u-root-builder/releases/download/v0.0.1/u-root_amd64_all.cpio.xz
curl -L -o linux.tar.zst https://archlinux.org/packages/core/x86_64/linux/download/
tar -xf linux.tar.zst
Add the e1000 module to u-root
-
Decompress the
e1000.ko.zstmodule.zstd -o e1000.ko -d usr/lib/modules/*/kernel/drivers/net/ethernet/intel/e1000/e1000.ko.zst -
Pack it into its own
cpioarchive.echo "e1000.ko" | cpio -o -H newc >modules.cpio -
Combine both archives. Concatenation is enough.
cat modules.cpio u-root.cpio.xz >u-root-e1000.cpioIf you want to use
virtio-netnetwork device instead ofe1000you have to add and later load different modules in that order:failover.ko,net_failover.ko,virtio_net.ko(first two are dependencies ofvirtio_net).
Get the NixOS netboot files
-
Download the NixOS kernel, initrd, and netboot iPXE script.
REL=https://github.com/nix-community/nixos-images/releases/download/nixos-26.05 curl -L -O "$REL/netboot-x86_64-linux.ipxe" curl -L -O "$REL/bzImage-x86_64-linux" curl -L -O "$REL/initrd-x86_64-linux" -
Replace remote
httpslinks with local paths. You can do it manually or usesedto strip the$REL/prefix.sed -i "s@$REL/@@g" netboot-x86_64-linux.ipxepxebootfetches onlyhttp,tftp, andfileURLs. It registers nohttpsscheme, so anhttps://URL fails with an unknown-scheme error. That’s why in this demo those files will be served from the host.
Serve the files over HTTP
Run an HTTP server that serves the requested files.
python3 -m http.server 8000
Boot u-root
NixOS is quite slow to boot without KVM and needs at least 2 GB of guest RAM, so
boot with KVM enabled, -cpu host, and -m 2G. Run this in another terminal as
the previous one is busy running the HTTP server:
qemu-system-x86_64 -enable-kvm -cpu host -m 2G -machine q35 -nographic \
-append "console=ttyS0" \
-kernel usr/lib/modules/*/vmlinuz -initrd u-root-e1000.cpio \
-netdev user,id=net0 -device e1000,netdev=net0
-cpu hostis required: without itdhclienthangs forever and never gets a lease. On a host without KVM, use-cpu maxinstead (and expect a slow NixOS boot).
Netboot NixOS
-
Load the network driver when booted into the u-root shell.
insmod /e1000.ko -
Configure network via DHCP:
dhclient -ipv6=falsepxeboot -fileskips DHCP, so the explicitdhclientfirst is required to configure the network interface. In a production environment, if you have configured your DHCP server to serve e.g.Bootfile NameorNext server/TFTP Serverthen you can just usepxebootdirectly without adding-file, just as traditional PXE via DHCP. -
And finally boot NixOS
pxeboot -file http://10.0.2.2:8000/netboot-x86_64-linux.ipxe -cmd "console=ttyS0,115200"Under QEMU user networking, the host is reachable from the guest at
10.0.2.2.The
-cmdargument appends to the kernel command line. This is needed so we get output on the serial console.You should see:
Welcome to LinuxBoot's Menu Enter a number to boot a kernel: 01. Linux(kernel=bzImage-x86_64-linux initrd=http://10.0.2.2:8000/initrd-x86_64-linux) 02. Reboot 03. Enter a LinuxBoot shell Enter an option ('01' is the default, 'e' to edit kernel cmdline): >You can press enter or wait a couple of seconds for it to autoboot. If you’ve done everything correctly you should see:
<<< Welcome to NixOS kexec-26.05beta-295941.gfedcba (x86_64) - ttyS0 >>> The "nixos" and "root" accounts have empty passwords. To log in over ssh you must set a password for either "nixos" or "root" with `passwd` (prefix with `sudo` for "root"), or add your public key to /home/nixos/.ssh/authorized_keys or /root/.ssh/authorized_keys. To set up a wireless connection, run `nmtui`. nixos login: nixos (automatic login) [nixos@nixos:~]$
u-root’s iPXE support
u-root’s pxeboot ships a deliberately minimal iPXE parser
(pkg/boot/netboot/ipxe).
It supports only three commands:
kernel <path> [args...]— the kernel image, with any trailing tokens joined onto the kernel command line.initrd <path>— the initrd.boot— boot the loaded kernel.
The script must begin with #!ipxe. Every other iPXE command (set, chain,
menu, dhcp, imgargs, and so on) is skipped with an
Ignoring unsupported ipxe cmd log line, and there is no variable expansion.
The NixOS script’s kernel line ends with a ${cmdline} token for
chainloading, which u-root passes through literally. The kernel ignores it, so
it is harmless.