I hosted a PXE server to boot my devices over my network (and I think I'll keep it)

One of the aspects of home lab life is that you’re constantly reinstalling devices or virtual machines. It gets annoying after a while, which is partly why my home lab server also has copies of the VMs that I use the most, so I can quickly copy in a fresh install when I break things.

However, I don’t always have the latest updated version of the installation ISOs, and I’m unable to create a copy of a bare-metal installation in the same way. And after wearing out one too many USB drives through repeated reflashing of different operating systems, it’s time for a better way for my home lab.

The solution is rather elegant, involving the setup of a Preboot Execution Environment (PXE) server that stores the ISO files, allowing my client devices to boot from them over the network. I’ve been playing around with three different variations on how this can be handled, and I’m keeping two of them, because having easy access to bootable media is invaluable.

What is a PXE server, and why would you use one?

Think of it as a centralized hub for Live USD images for installation

Finding a USB drive to create bootable media is annoying enough, but what if your client device doesn’t have accessible USB ports? Many enterprise devices are locked down so that you can’t boot from physical media, and that’s where PXE comes in. Think of it as a central repository of bootable media that any device on your network can access and boot from, even if they don’t have much of a local hard drive.

But while you could set individual devices up as diskless clients, that’s not the true power of PXE when used at home. That’s combining PXE with Proxmox, which makes setting up new VMs simple, as you never have to worry about where your ISO files are; you point Proxmox to using PXE as a boot option and choose the boot files you want.

And as all my ISO files are in one place, I never have to keep track of USB drives ever again. That’s a bunch of things I never have to think about, whether it’s installation disks for Linux, routers, firewalls, NAS, or Windows.

It’s not just good for booting

Create customized images to set up repeatable experiments

While the general idea for PXE servers is to host preconfigured ISO files, some, like the Fog Project, can image already running devices to use them as PXE boot images. If little lights just went off all over your brain, congratulations, you’ve got the home lab bug. It’s all about finding the workflow that fits your needs, and sometimes it’s easier to manually install, update, and add tools to an operating system than to tweak the ISO or rely on the installation script.

Sometimes I don’t know what I want installed until the system is running, and being able to take images after the fact, but before I start experimenting, makes all the difference. I still don’t have a reliable way to automate an OPNsense installation, because networking adapters vex me, but I can do the lengthy manual process before creating an image.

Whatever PXE server software you use, your router also needs to be set up to forward the PXE boot requests to the PXE server, and also to the TFTP server that holds the images. It depends on your router, but you can find the necessary information in the DHCP options. To enable Network Boot and TFTP Server, you must point them at the IP address of the PXE server, but that’s it.

Or use PXE over the internet

While traditionally, PXE servers have been on-premises, that doesn’t have to be the case. Netboot.xyz provides a bootable environment with a collection of ISO files, all of which are accessible over HTTP. That’s right, you don’t have to set up a PXE server, although you can set up a local Docker container to mimic the hosted site, or set up Ansible and customize the playbook provided to add your entries to the boot menus. It really is one of the easiest ways to install new VMs in the home lab, because you don’t have to keep track of distribution files, just the netbook.xyz ISO.

PXE boot is a valuable tool for the home lab

I’m keeping Fog Project installed, because then I can organize my install images for easy retrieval in Proxmox. I don’t have to go through the process of downloading new ISO files for every project, or worry if they’re up-to-date. I can boot into the image, install it quickly, and update things to the current software packages. It also means no more USB drives to wear out, and I can free up terabytes of data on my main PC, which currently stores all my installation images.