VyOS - Blog

VyOS Project November 2024 Update

daniil@sentrium.io (Daniil Baturin) — Fri, 29 Nov 2024 14:06:15 GMT

Hello, Community!

The November update is here. This post is short, but not all we've done lately: many internal changes in the configuration system will soon significantly improve commit speeds and open up a path to even more significant improvements. The 1.4.1 release is around the corner, together with the first VyOS Stream image — all built by the new CI system that produces tarballs with the corresponding source code for every image. But now, let's focus on the changes we made in the rolling release in October.

Adding custom operational mode commands to VyOS

e.altunbas@vyos.io (Erkin Batu Altunbas) — Thu, 28 Jan 2021 10:14:37 GMT

The operational mode (op mode for short) is half of the VyOS CLI, the other half being configuration mode (conf mode for short). Maintenance and administration take place in the op mode, using tight-knit, high-level commands and subcommands that wrap lower level programs and scripts, and form an overarching web of abstraction over the system.

Although the existing commands tend to cover most use-cases in practice, there will always be cases where the user is forced to reach into the guts of the system to carry out operations not covered by the CLI, or simply needs to integrate their own programs into the system. Keeping track of operations outside the CLI can be tiresome, and such operations can possibly disrupt the system maintained through the CLI in unexpected ways if there’s an overlap in functionality.

The clear solution to this conundrum is integrating your programs and services into the CLI. In addition to working out the problems above, you can benefit from the convenience of tab completion. Let’s see how we can go about accomplishing this.

Pipes in VyOS CLI

e.altunbas@vyos.io (Erkin Batu Altunbas) — Thu, 21 Jan 2021 09:35:09 GMT

Unix pipes are an indispensable tool on a sysadmin’s belt. Eventually, it becomes second nature to build a pipeline of several tools to transform data and one often feels frustrated when they can’t use it in a CLI environment distinct from a Unix shell.

VyOS CLI is a high-level, centralized interface where the great variety of configurable subcommands often eschews the need to pipe multiple small programs together to shape the output into the desired form. (Although, you still can do that, naturally.) However, there are cases where one feels the need of generic functions that can be applied to the output of any subcommand.

Configuration versioning and archiving in VyOS

daniil@sentrium.io (Daniil Baturin) — Sat, 10 Feb 2018 17:11:50 GMT

Last time I promised "node copying/renaming, node comments, and other little known features of the VyOS CLI", but the post actually only mentioned copying/renaming and comments, but not other features. It's time to fix that: today we'll discuss configuration versioning and archiving.

One of the great things about the config model with editing and commits being distinct stages is that it's feasible to execute some actions when the config is changed. In fact, you can execute arbitrary actions via pre/post-commit hooks, but there are built-in actions as well, namely configuration versioning and archiving to a remote location. This model, first introduced by JunOS, makes configuration is a lot more manageable than older Cisco style models.

This approach renders tools like Rancid or Oxydized redundant since the system can make a snapshot of the running config when the change is made rather than periodically. Moreover, right on the router you can see who made this or that commit and view diffs between revisions.

An additional advantage of versioning is that even if you forget to save the config (or purposely powercycle a system with an unsaved config because you forgot to use commit-confirm), you can always view recover the lost changes from the history.

Let's see how to use it.

Versioning

Configuration

By default, VyOS only keep a local archive of previous config versions. The number of revisions it keeps is controlled by the "system config-management commit-revisions $number".

In older VyOS versions the default value is 20, while the 1.2.0 nightly builds have it set to 100. Since configuration files are usually not too big, especially compared to the size of modern flash memory and hard drives, and the versioning system uses gzip compression, I usually set it to 1000 on all my routers: set system config-management commit-revisions 1000

You can disable versioning by deleting the "system config-management commit-revisions" option.

Usage

Viewing the commit history

To view the commit history, use this command: run show system commit

Here is an example:

dmbaturin@reki# run show system commit <br>0   2018-02-10 16:28:52 by dmbaturin via cli<br>    increase commit revision number<br>1   2018-02-10 16:23:39 by dmbaturin via cli<br>2   2018-02-01 05:41:33 by dmbaturin via cli<br>3   2018-02-01 05:41:27 by dmbaturin via cli<br>4   2018-01-26 02:46:31 by dmbaturin via cli<br>5   2017-12-25 04:37:54 by root via boot-config-loader<br>6   2017-12-15 18:46:13 by dmbaturin via cli<br>

The output is quite straightforward, though some things need clarification. The first field is the revision number, the second field is the timestamp, and the third field is the user who initiated the commit.

The "via" field indicates the "application" that the commit was initiated from. Commits from the interactive CLI have it set to "cli", and commits made by the config loader at boot time have it set to "boot-config-loader" (why are they archived? At boot time the config can be modified by migration scripts).

In the example above, you can see "increase commit revision number" under a revision. This is a commit comment, which you can set with this command: commit comment "some comment".

Viewing old revisions and diffs

To view a complete config file from an older revision, use this command: run show system commit file $revisionNumber
The revision 0 points to the latest commit.

To see what a certain commit changed, use: run show system commit diff $revisionNumber
It shows a typical diff:

dmbaturin@reki# run show system commit diff 0

[edit interfaces ethernet eth1]

-address 10.90.90.1/24

[edit system config-management]

>commit-revisions 2000

[edit system]

+options {

+    ctrl-alt-del-action ignore

+    reboot-on-panic true

+}

This command may seem rather limited because it only shows a diff between a single revision N and revision N-1. For more advances comparison operations, you can use the "compare" command.

With just "compare" command with no arguments, you can view the diff between the proposed (uncommited) and running configuration. Quite handy when you have a large config and don't want to spend time searching for the modified bits in the output of the "show" command:

dmbaturin@reki# compare

[edit system options]

>ctrl-alt-del-action reboot

You can also use "compare saved" command to quickly view a diff between the running and saved configuration.

Running "compare $revision" (e.g. "compare 5") allows comparing the running config with that revision, showing a cumulative diff (as opposed to a diff between $revision and $revision-1 as "run show system commit diff $revision" does).

Finally, you can use "compare $firstRevision $secondRevision" (e.g. "compare 4 20") to view a diff between any two revisions.

Archiving

Apart from the local archive, you can automatically send config revisions to a remote locations. The supported protocols are TFTP, FTP, and SFTP/SCP.

The configuration option is: system config-management commit-archive "<tftp|ftp|scp>://$user:$password@$host/dir", for example, "ftp://admin:qwerty@192.0.2.1/vyos".

Confirmed commit and rollback

One more possibility that is opened by configuration versioning is failsafe commits for risky changes and rollbacks.

Suppose you are making a risky change and you aren't sure whether you will lose your SSH connections to the router or not. The traditional approach is to schedule a reboot, make the change, and cancel the reboot if you are still connected.

VyOS offers a better approach. When you are about to make a risky change, use the "commit-confirm" command, optionally with an argument that specifies timeout in minutes (e.g. "commit-confirm 5"). The default timeout is 10 minutes.

It will warn you about what is about to happen and ask if you want to proceed. Type "y" or "yes" and press enter.

dmbaturin@reki# commit-confirm

commit confirm will be automatically reboot in 10 minutes unless confirmed

Proceed? [confirm]y

If the change works fine and you are still connected, enter the "confirm" command. If you don't issue the "confirm" commands in 10 minutes (or the time interval you specified) the system will automatically reboot and revert to the previous revision (not: not to the saved config, but to the revision just before your last commit even if it wasn't saved).

You can as well rollback to any previous revision by hand with "rollback $revision" command, but it will also reboot your system first, unlike in JunOS, which limits its utility in non-emergency cases. The reasons for this are quite complicated and lie in unfortuate design decisions of the old config backend that gave rise to the VyConf project. If you want to see non-destructive rollback sooner, consider joining the development of the new backend. ;)

Copying/renaming, node comments, and other little known features of the VyOS CLI

daniil@sentrium.io (Daniil Baturin) — Fri, 26 Jan 2018 02:54:16 GMT

I promised not to write about either IPsec or NAT this time, so we'll discuss something else: the little known features of the VyOS CLI. Many people only ever use set/delete and commit, but there's more to it, and those features can save quite a bit of time.

The edit level (never write long node paths again)

You might have noticed that after every command, the CLI outputs a mysterious "[edit]" line. This is a side effect of the system that allows editing the config at any level.

By default, you are at the top level, so you have to specify the full path, such as "set firewall name Foo rule 10 action accept". However, to avoid writing or pasting long paths, you can set the edit level to any node with the "edit" command, such as "edit firewall name Foo". Once you are at some level, you can use relative node paths, such as "set rule 10 action accept" in this case.

To move between levels, you can use the "up" command to move one level up, or the "top" command to instantly move back to the top level.

Look at this session transcript:

dmbaturin@reki# edit firewall name Foo

[edit firewall name Foo]
dmbaturin@reki# set rule 10 protocol tcp

[edit firewall name Foo]
dmbaturin@reki# edit rule 10

[edit firewall name Foo rule 10]
dmbaturin@reki# set destination port 22

[edit firewall name Foo rule 10]
dmbaturin@reki# up

[edit firewall name Foo]
dmbaturin@reki# set rule 10 description "Allow SSH"

[edit firewall name Foo]
dmbaturin@reki# top

[edit]

Node copying and renaming

If you are making a number of similar policy rules that differ in small details, or rearranging a ruleset, copy and rename commands can save a lot of time. The only issue with them is somewhat fragile and unwieldy syntax.

First, to be able to use them, you need to switch to the level just above the nodes you want to copy or rename. For example, if you want to do something to rules of a firewall named Foo, you need to do "edit firewall name Foo" first. Second, you need to use the complete relative path to the node in both left and right hand sides of the command.

It's perhaps something that is easier to show than explain:

dmbaturin@reki# edit firewall name Foo

[edit firewall name Foo]
dmbaturin@reki# copy rule 10 to rule 20

[edit firewall name Foo]
dmbaturin@reki# rename rule 20 to rule 30

[edit firewall name Foo]

Node comments

Node comments allows one to add notes to any node even if it doesn't have a built-in description option. It's also very simple to use, just "comment $nodePath $commentText" and commit.

dmbaturin@reki# comment system config-management commit-revisions "Ought to be enough for everyone"
[edit]

dmbaturin@reki# commit
[edit]

dmbaturin@reki# show system config-management
/* Ought to be enough for everyone */
commit-revisions 1000
[edit]

If you want to remove the comment, just set it to an empty string ("") and commit, and it will disappear. In this example that would be 'comment system config-management commit-revisions ""'.