VyOS Networks Blog

As usual, we've been quite busy. First of all, sorry for the nonsensical "VyOS News Digest". The webmasters we hired to help us look after the social media accounts posted things they shouldn't have posted. It's an unfortunate incident and we regret failing to prevent such a drop in signal to noise ratio of this blog. We promise that won't happen again.

Now to the real VyOS news: let's see what we've been working on and done in June...

Partnerships with EdgeCore, EPS Global and Oracle

We are happy to announce two new partnerships. Very soon you will be able to buy EdgeCore hardware bundled with VyOS Support subscriptions via EPS Global. The hardware doesn't come with VyOS pre installed just yet—you'll receive a customer portal account and a subscription so that you can download and install the image yourself, however, it will simplify the purchase process significantly 

Second, we are working with Oracle to validate VyOS on Oracle Linux KVM, OracleVM and  Private Cloud Appliance.

More surprises ahead. Stay tuned for updates!

Clustering will be removed from the rolling release

The original clustering implementation coming from Vyatta Core has always been rather half-baked and bug-ridden. We’ve been saying that it’s deprecated for a long time, but somehow we still see people make new clustering setups and run into problems with them.

Don’t get us wrong: we aren’t against the idea of clustering! It would be really cool to be able to have clustered services with a flexible availability tracking mechanism and seamless failover.

However, our existing implementation just isn’t that. By now it’s worse than VRRP! The only type of clustered resource is IP address—and only IPv4, while in VRRP we already support both IPv4 and IPv6. We also have a mechanism of health check and transition scripts there and it’s free of original bugs that caused config permission errors when a config was modified by a script. VRRP moved forward, while clustering is still stuck where it was in 2013.

One reason it’s stuck in 2013 is that heartbeat, the project backing our clustering implementation, is nearly dead. It’s not moving forward. With last release in 2015 and no activity ever since, there's no hope we can keep using it.

Thus, not only do we need to rewrite the old scripts, we also need to replace the clustering daemon itself, most likely with Pacemaker and corosync.

Until then, the fewer people use the original clustering implementation, the easier it will be to replace it with something else. So, we need to get the message across: it’s deprecated and shouldn’t be used.

To make it clear, we are removing the original implementation from the rolling release. 

It’s not working all that well in the LTS release either, but we are committed to maintaining compatibility in LTS releases, so we are going to keep it there for now and at least try to make band aid patches for the worst issues people find.

If we can make a new implementation until the 1.3 branch freeze, then we’ll write a migration script to convert old configs to the new syntax. If not, we will write a script to migrate it to VRRP, since current VRRP implementation is functionally equivalent to the old clustering.

We also encourage everyone who’s now using 1.2.x LTS releases to migrate their old clustering setups to VRRP by hand—we promise you will not regret it.

FastNetMon is coming to the rolling release

You don’t think all we want to do is to remove old features, right? Many people kept asking about IPS/IDS support, but we haven’t been working on it because an IDS is usually a separate box anyway and it wasn’t a very high priority thing.

If you are curious why Snort was removed back in Vyatta Core times, it was because Snort maintainers would make frequent and unwarranted changes to rules file format, which forced Vyatta developers to make new maintenance releases just to keep up with Snort. That situation obviously wasn’t sustainable, so Snort had to go. That was before we made a fork, so VyOS started without an IDS. Now that’s changing.

Relevant threats changed a lot in the last years. DDoS attacks are getting more severe and more widespread than ever. FastNetMon is a popular DDoS detection tool, and a number of people have used it with VyOS already.

Now it’s coming to the rolling release images, with a CLI integration, thanks to the efforts of Dmitriy Eschenko, Viacheslav Hletenko, and of course Pavel Odintsov himself—we've been making a great collaboration with him.

It's under set service ids ddos-protection. If you are already familiar with FastNetMon, the options should be straightforward. Your help with documenting it would be appreciated!

New scripts should use vyos.config.get_config_dict()

When we first made a Python library for working with VyOS configs, it was a highly experimental prototype and no one was sure if the new approach with XML command definitions and Python scripts instead of Perl will work properly and catch on.

We are all happy that it did catch on and brought more contributors to the project within two years than it had in the previous five years of existence. However, our work on those libraries is far from done.

The original library was just a thin wrapper for the old cli-shell-api, trying to work around its worst limitations. This is problematic because cli-shell-api itself is rather slow, and calling it from Python is even slower. But performance isn't its only issue—it's also rather inconvenient to extract config values one by one with config.return_value(["interfaces", "ethernet", "eth0", "address"]) and so on. We got used to it because there was no other way, but we can do better.

What changed the situation was importing parts of the new config backend that is still work in progress. It's written in OCaml and actually understands the syntax of VyOS configs and can convert it to more universally accepted formats like JSON. The first use case of it was the vyos.configtree library that is used for migration scripts. It also powers the | json pipe found in the rolling release.

However, it's also in vyos.config library and allows retrieving a chunk of the config as a dict at once. To simplify creating dicts that you can pass straight to Jinja2 or something else that doesn't like hyphens in the keys, there's now a key mangling option that takes two regular expressions:

>>> import vyos.config
>>> c = vyos.config.Config()
>>> c.get_config_dict(["system", "options"])
{'options': {'beep-if-fully-booted': {}, 'ctrl-alt-del-action': 'ignore'}}

>>> c.get_config_dict(["system", "options"], key_mangling=('-', '_'))
{'options': {'beep_if_fully_booted': {}, 'ctrl_alt_del_action': 'ignore'}}

This is a thing that isn't possible with the old backend. There is no Perl equivalent of this and will not be.

If you are adding a new feature, please make use of it! If you are rewriting an old Perl script, please refrain from a word for word translation and also make use of it.

It makes things much simpler than the old approach. There are other improvements in progress, including a better approach to default values.

VyOS as a console server

Christian Poessinger single-handedly added a console server feature. You can find its documentation here. If you are using a retired Cisco router as a console server, now you can replace it with a VyOS box, provided you have enough serial ports there. It's not just network devices that still use serial console. Some storage arrays, industrial equipment and other boxes still use it too, so it's still a handy out of band managemt mechanism.

Sensor data

Many people requested an equivalent of JunOS' "show chassis environment". Then came Kroy and added "run show environment sensors". It does a pretty good job detecting sensors and loading modules we hope. If you are running on bare metal, let us know how that command works for you.

1.3 when?

We know "when it's ready" isn't a very good answer. You want details, and we know that. Well, we are heading towards a freeze, but there are still a couple of big things we want completed before we can freeze the branch.

LTS release is a balancing act: if you delay it too much, people are missing out on features that are long ready to use, but if you freeze to early, you are stuck with legacy code draining development resources to maintain.

One big thing we want done before the freeze is rewriting the dynamic routing protocol scripts in Python. The old scripts are messy and often split into literal hundreds of node.def files. They are very hard to extend and maintain. And now there's a good progress! Runar Borge created a new library for interfacing with FreeRangeRouting that will simplify the rewrites.

And, of course, there's a lot of normal rewrite and bug fixing work. Jerney Jakob's work on the DNS forwarding subsystem and dhclient script, Thomas Mangin's work on XML file handling and so on—the work of maintainers and community contributors alike is bringing us closer to the next release. It's not there yet, and it's unlikely that we can make a beta until this autumn, but we are heading in that direction.

Python rewrite progress

We are talking about getting rid of legacy code all the time, but how much work is still left? Here's a not very scientific take on it: comparison of lines of code in the .pl/.pm vs. .py files found in the running VyOS image from 1.1.8 to the latest 1.3 builds.

According to that measure, half the old code is gone already.

Thanks for reading the post to  the end! Remember there was a tradition of merch discount links for those who read posts to the end? It's back now.
Here is link with a 15% discount for all our merch that is valid until August the 31st. Get your t-shirt or a mug and show your support for the project.

https://teespring.com/stores/vyos?pr=JUPDATE