Introducing the image build flavor system
Hello, Community!
Building customized images of VyOS for different platforms has become much easier now. The last bit of the revamped build flavor system has fallen into place: the ability to build image formats other than ISO. Some other options, such as the ability to include custom packages and files, have been there for a while, but recently we fixed a few issues with them. Let's review those improvements and see how to use them.
Recently, we announced that we are working on an online image builder service. The main idea is to make it easier for everyone to build and, most importantly, maintain customized builds. However, that service will merely simplify working with the existing image build system rather than implement some proprietary alternative to it.
The build system of VyOS 1.3 and earlier only allowed building ISO images. One big reason for it is that it's the only thing that live-build (Debian image build tool) makes easy. Since we wanted to support multiple virtual and cloud platforms and there was customer demand for it, we had to develop custom procedures for building those images from the ISO — because the ISO was all we had. Initially, we used Ansible as the automation tool, which worked well enough for us. However, it was not a mechanism that could allow us or community members to easily create new image types for new platforms or environments — such as an image for some CPE box with a custom built-in config.
That is why we started working on extending the build system to allow users to specify what kind of image they want and what they want in a declarative manner. It took a while to implement fully — the problem is tricky, and we tried a few approaches, including an attempt to get live-build's raw image building functionality to work, that went nowhere. We also gave our GRUB configuration code a serious overhaul. Now that system is finally in place in the current branch and ready to use.
The system is centered on build "flavors". Build flavors are defined in TOML files — by default, the script searches for them under data/build-flavors.
Suppose you want to build a raw image for flashing on a bare metal box hard drive without virtual machine support tools. We recommend using our container image with Docker or Podman since it includes all the necessary tools.
If you want to set up a build VM from scratch, the build script will tell you what libraries and tools to install, if any are missing, but using a container is simpler.
# Clone the vyos-build repository
git clone https://github.com/vyos/vyos-build
cd vyos-build
# Start a build container
docker run --rm -it --privileged -v $(pwd):/vyos -v /dev:/dev -w /vyos vyos/vyos-build:current bash
# Inside the container:
echo 'image_format = "raw"' > data/build-flavors/raw.toml
sudo ./build-vyos-image rawImage format
This most recent addition completely changes the virtual image building game!
In the generic ISO flavors, we have image_format = "iso", naturally. But it can be any format supported by qemu-img, such as:
rawqcow2(QEMU and KVM)vdi(VirtualBox)vhdx(Hyper-V)- ...and more.
That option can be a list of formats, and you can produce multiple formats at once. For example, if you want a VirtualBox-friendly ISO and a VDI image, you can do it like this:
If you want to pass custom options to qemu-img convert, or use a file extension different from the image format name; we also have you covered. This is how you can build an image for Hyper-V with compression enabled (-c) and call it .vhd rather than .vhdx:
One note is that right now, the build process always produces an ISO, and building any format other than ISO will always produce a raw image because the build sequence for them is ISO → raw → target. We may be able to simplify that in the future, so we encourage flavor writers to specify all desired formats explicitly.
Another note is that if you are using a container, the -v /dev:/dev bit is essential. The process for building non-ISO formats involves creating a loop device to mount the image, and containers don't have a permission to do that, so you have to run them in a privileged mode and give them access to /dev/ on the host. That, of course, effectively gives the container full access to the host system, so if you are testing some untrusted code or experimental changes, it's a good idea to run the build in a full VM instead.
Custom packages
Suppose you build a Qcow2 image using image_format = "qcow2" — but does it make a good image for QEMU? To make it work better in QEMU/KVM, it's a good idea to add the guest agent package.
And that's trivial to do: you can add any package from Debian or VyOS repositories to the image by adding them to packages — that option is a list of package names that can be as long as you want.
image_format = "qcow2"
packages = ["qemu-guest-agent"]Custom configs
It's also very easy to include custom configs. I'm sure all actual configs people may want to include will be more complex than the standard default. However, for demonstration, let's see how to include a minimal config with just the default user and nothing else.
You put it in the default_config option, and it's handy that TOML supports multiline strings.
default_config = '''
system {
host-name vyos
login {
user vyos {
authentication {
encrypted-password $6$QxPS.uk6mfo$9QBSo8u1FkH16gMyAVhus6fU3LOzvLR9Z9.82m3tiHFAxTtIkhaZSWssSgzt4v4dGAL8rhVQxTg0oAG9/q11h/
plaintext-password ""
}
}
}
'''Custom files
Sometimes, you also need to include arbitrary custom files. For example, many platforms require link files for correct network interface numbering.
You can include any files inside the SquashFS image by adding them to the includes_chroot array. The best way to include multiple files is to use TOML's "array of tables" syntax with double brackets.
Each entry of that array must have two fields: path (a relative path to the file inside the image) and data (file content).
image_format = "iso"
# Simple file for demonstration
[[includes_chroot]]
path = "usr/share/doc/custom_image.txt"
data = '''
This is a very custom VyOS image!
# Another file
[[includes_chroot]]
path = "usr/share/doc/readme.txt"
data = '''
This image was built using the flavor system.
'''Architecture mix-ins
Finally, here's a feature that will become relevant when VyOS adds full support for non-x86 architectures, such as ARM64 — there's a way to add specific packages only for some architectures.
GRUB console settings
You can specify console settings for the Linux-based system in the config. However, if you are building an image for a box that only has a serial console, or for certain cloud platforms, you also need custom console settings in the GRUB config if you want to see and use the boot menu.
The flavor file syntax has options for it. For example, this is how you can specify that your target box uses ttyS0 console device with 9600 (8/N/1) speed settings.
[boot_settings]
timeout = 5
console_type = "ttyS"
console_num = 0
console_speed = 9600
Reusing ISO files
The full build procedure can take up to twenty minutes, if you are building from a clean state. That's a long wait, if you are only messing up with image format options or GRUB console settings and don't need to build the source ISO — either because you don't need anything special in the image and can use a generic ISO, or because you built a custom ISO already.
In that case, you can speed up the process. Suppose you have vyos-1.5.0-my-custom-flavor.iso. You can put it in build/ and run sudo ./build-vyos-image --reuse-iso vyos-1.5.0-my-custom-flavor some-flavor. In that case, the build script will only produce a target image from the ISO and install GRUB in it.
Note that if you reuse an ISO, the build script will not install custom packages or include custom files — it assumes that they are already there. It's a shortcut we added during development to avoid having to run a full ISO build procedure for every tiny change in the raw image build part, and we left it in place to make it easier to work on the build script, but it's an "I know what I'm doing thing" and the responsibility for using it is on you. If you suspect that anything isn't working well in build scripts, do a full build before filing a bug report.
What's next?
We hope the new flavor system makes it much easier for people to use VyOS as a platform than just a self-contained product. The online builder will make it easy to create and organize flavor files through a web UI, but the system's foundation is right there in vyos-build — if you try it, let us know if you think it can be improved or if anything doesn't work as expected!
Comments