VyOS Platform Blog

Building an open source network OS for the people, together.

VyOS Project June 2022 Update

Daniil Baturin
Posted 29 Jul, 2022

Hello, community!

It's been a while since our last update, so it is time for the progress update for June. Our schedule was definitely disrupted, but we never stopped working on improving VyOS, and we are getting back to posting development updated every month. Thank you for your patience!

In June we added a few new options for firewall, brought back the event handler — a feature we all missed, and thanks to the VyOS community, found and fixed some bugs. Read on for details!

Infrastructure issues

Many people have noticed an issue with nightly build downloads that occurred in the last week — replies from the server were coming with an incorrect certificate, so you'd have to disable certificate verification. That was a misconfiguration on our part and now it's fixed. Let us know if you notice any more issues!

There are also reports of suspected country blocks in our CloudFlare setup. We still need to investigate them: we never planned to limit access to nightly builds from anywhere (unless forced to), but we trust that those reports aren't baseless — we will look further into it.

New features

Event handler is back

You may remember that in the 1.3.0 release we removed the event handler feature ("set system event-handler"). Removing it wasn’t an easy decision: we agree it’s a useful feature. We promised to bring it back and now you are welcome to test its new incarnation.

There are several significant changes and improvements.

From Perl to Python

Implementation-wise, the old event-handler was written in Perl, and due to its primitive garbage collector, Perl is a bad language for long-running programs. It can create memory allocations that it cannot reclaim itself nor let the user free them — in other words, create memory leaks. That’s what was happening to the old event handler quite often.

The new implementation is in Python, like all our new code, so it's easier to maintain and doesn't suffer from memory leaks.

Journald’s benefits

The old event-handler could read input from arbitrary files or running programs. Allowing it to read from arbitrary files and process outputs created a bunch of edge cases when it would stop functioning correctly and there’s nothing we could do about them.

For example, suppose it's reading output of a program that crashes and for some reason cannot be restarted. It will just keep endlessly trying to restart it, and will be unable to handle the events the program is supposed to relay to it. There are also potential issues with log rotation: if a file is closed for rotation and the event handler is forced to reopen it, there is no guarantee that it will not lose any messages in that process.

Then systemd-journald came along and brought structured log storage to Linux, which the old event handler couldn't use. Journald stores logs in a binary format and there are libraries for reading it, so they are easy to read from scripts. Since the system log is the most used source of events anyway, we decided to focus on it.

The new implementation can only read from the system log through journald. It means many old configs cannot be adapted to work with it, but it’s free of those edge case. Besides, it also allows you to match service identifiers, and the event handler checks the message PID to ignore messages from itself, which at least partially solves possible infinite event loop problems.

More control

The new event handler also allows you to pass custom environment variables to scripts.

If you already have event-handling scripts from old VyOS versions, it may not be very useful, but if you only plan to write a script, it can make your life much easier.

Previously, you needed to have a dedicated script for each event because there was no way to find out what event triggered your script.

Now you can get the log message that triggered a script as an environment variable, retrieve event information from it and use it in your script logic. The complete log message string is passed in a built-in variable named message. You can also pass custom values in environment variables and perform actions based on them: for example, add an IP address to a firewall group, change interface settings, restart a specific service or VPN session, and much more.

Let’s look at a small example: we will check logs for an interface state change and then add IP addresses to a dummy interface or remove them from it, depending on whether the interface has gone up or down.

Here's the configuration:

set service event-handler event ETH0_STATE filter pattern '^eth0:.*$'
set service event-handler event ETH0_STATE filter syslog-identifier 'netplugd'
set service event-handler event ETH0_STATE script environment dummy_iface value 'dum1'
set service event-handler event ETH0_STATE script environment dummy_ipaddr value ''
set service event-handler event ETH0_STATE script path '/config/scripts/dummy_addr.py' 

And the script:

#!/usr/bin/env python3
# VyOS event-handler script example
from os import environ
from re import compile
from subprocess import run
from sys import exit

# Perform actions depending on the event
def process_event() -> None:
    # Get variables
    message_text = environ.get('message')
    dummy_iface = environ.get('dummy_iface')
    dummy_ipaddr = environ.get('dummy_ipaddr')
# Parse the message to figure out if the interface has gone down or up regex_up = compile( r'^eth0:.*UP,(?![\w,]*RUNNING[,\ ]).*->.*UP,.*RUNNING.*$') regex_down = compile( r'^eth0:.*UP,.*RUNNING.*->.*UP,(?![\w,]*RUNNING[,\ ]).*$') if regex_up.fullmatch(message_text): action = 'add' elif regex_down.fullmatch(message_text): action = 'del' else: exit(1) # Prepare a command to run command = f'sudo ip address {action} {dummy_ipaddr} dev {dummy_iface}'.split() # Execute a command run(command) if __name__ == '__main__': try: # Run script actions and exit process_event() exit(0) except Exception as err: # Exit properly in case if something in the script goes wrong print(f'Error running script: {err}') exit(1)

As you can see, with these features you get much more control over event handling and can reuse the same script for multiple events.

Feel free to try the new event-handler and share your impression and ideas with us. There is still a lot of space for improvement, and we would be happy to hear from you.

New command for clearing DHCP leases

This command forcibly removes a DHCP lease, in case you cannot wait for it to expire or make the client release it.

For example, if we need to remove the lease for address, we can just use this command clear dhcp-server lease

We used to have a command for that in older VyOS versions, but we removed it because its implementation was very fragile and could completely break the lease file in certain scenarios, so it created as many problems as it solved. We hope the current implementation is more robust, but if you find any bugs, let us know!

TTL match option in firewall rules

We already had support for matching hop count in IPv6 rules, but there wasn’t an option to match IPv4 TTL.

Now we have that option, and It gives firewall options more flexibility. When it comes to firewall options, the more of them there are, the easier it is to write a ruleset to match your needs, so we are always open for new option suggestions.

RDNSS lifetime option

Currently. VyOS supports changing the RDNSS lifetime option. We added an option to manually change the RDNSS lifetime so that you can avoid frequent expiration of RDNSS on links with high packet loss.

You can set it using this command:

set service router-advert interface eth0 name-server-lifetime <value>

Support for nested firewall groups

Previously if you needed firewall address groups that shared some of their addresses but also had addresses unique to each group, you'd have to duplicate the common addresses in each group. Managing such groups can easily become tedious.

Now, thanks to Simon, there's support for "nesting" groups: you can define one group for common addresses and reference it in other groups — no duplication needed.

For example:

set firewall group address-group FullyTrusted address
set firewall group address-group FullyTrusted address
set firewall group address-group TrustedForFoo address
set firewall group address-group TrustedForFoo include FullyTrusted 

OWAMP and TWAMP for service SLA

Now VyOS supports One-Way Active Management Protocol (OWAMP) and Two-Way Active Management Protocol (TWAMP) for round-trip metrics.

 With the new feature you can get more accurate results than from using the ordinary ICMP ping.

These are the new configuration and operational mode commands for that feature:

set service sla owamp-server
set service sla twamp-server
run force owping
run force twping

If you need additional options that aren’t implemented yet, please let us know.

Bug fixes and other enhancements

  • Fixed the numeric validator. Now it accepts signed numbers (T4467).
  • Added option to reset BGP in VRF (T4494).
  • Bugfix on address iteration. Thanks to Jack for reporting this bug (T4397).
  • PPPoE interface options and DHCP interface options honor the no-default-route option now (T4384).

What about the previous months?

Since we haven't published any updates in a while, we must mention at least some of the improvements we made during that time. Here are some highlights:

  • It's now possible to clear BGP stats and neighbor sessions for a specific VRF (T4494).
  • It's now possible to import key or certificate files into the VyOS PKI (T3642, thanks to Simon for the patch!).
  • The show system uptime command now displays CPU load averages normalized for the number of physical CPU cores, so that 100% means "all cores fully loaded" (T4432).
  • It's now possible to specify a log level for firewall rule logging (T3907).
  • There are now commands for monitoring logs for specific facilities, like monitor log kernel (T4390).
  • It's now possible to match DNAT'ed and SNAT'ed connections in firewall rules (T990).
  • There's now a general config tree diff algorithm in the backend, available through the "vyos.configtree.Diff" class.

Future plans

We haven't made new LTS releases lately, but we in fact have quite a lot of bugfixes and some feature backports already. 1.3.2 LTS release will come out soon: currently we are testing it to make sure that everything works fine.

We are also actively working on the long-promised web UI for VyOS. The UI itself is just one part, we also have to greatly extend the HTTP API to be able to create a front-end without coupling it to implementation details. In particular, we finally have a good design for standardized operational mode command scripts and a way to automatically expose them as GraphQL requests — stay tuned for updates!

:Last but not least, a new technical writer already joined the VyOS team and is now learning about VyOS and getting up to speed. With more people who can write and a dedicated person to look after the social media accounts, we should be able to communicate with the community much more frequently and effectively, so I hope our activity will not merely get back to the old level — it will increase. So, when you see posts with a new name in the signature, don't be surprised!

The post categories: