VyOS development digest #10
1.1.8 release plan
There have been some uncertainty over this issue and it wasn't clear if we'll be able to make an 1.1.8 release or not with squeeze's death, but recently Kim and I got squeeze builds to work again, and this enables us to finally make one.
What's certain is that bugfixes from 1.2.0 are going to make it there. What's not yet certain is which features we should cherry-pic. OpenVPN user/password auth, for example, is definitely safe and well tested enough to bring it to 1.1.8.
1.2.0 development status
1.1.8, of course, is nothing more than a maintenance release. But, we are way closer to a full feature release now that, especially with the work done by two awesome contributors, namely Christian Poessinger and Jules Taplin. Among recent contrubutions are multiple fixes to IPsec operational and configuration mode (in particular, "show vpn ipsec sa" works properly now), correct deletion of VTI interfaces, and there's also work being done on integrating mDNS repeater.
1.2.0, Python, and code rewrites
This was already discussed in http://blog.vyos.net/vyos-2-dot-0-development-digest-number-7-python-coding-guidelines-for-config-scripts-in-1-dot-2-0-and-config-parser-for-vyconf and http://blog.vyos.net/vyos-2-dot-0-development-digest-number-5-doing-1-dot-2-x-and-2-dot-0-development-in-parallel
By now, the Python library is "beta" rather than "alpha" and it has already been used to rewrite the cron ("set system task-scheduler") scripts by Tania Dzyubenko and me.
The library is now a proper Python package and it's installed as vyos.config module. You can use it for VyOS scripting, as well as code rewrites.
It has also been moved out of the vyatta-cfg package. The package where the new rewritten code goes is https://github.com/vyos/vyos-1x
You can find the rewritten cron script here: https://github.com/vyos/vyos-1x/blob/current/src/conf-mode/vyos-update-crontab.py
As you can see, it's architecturally pretty different from the older scripts. You can find the guideline it's written according to here in the wiki: https://wiki.vyos.net/wiki/Python_coding_guidelines
The architecture boils down to this: all VyOS config reads are confined to one function that converts it into an abstract representation, the rest of the logic is split into separate "verify", "generate", and "apply" stages that, accordingly, verify config correctness, generate configuration files, and apply them to the live system.
I'll re-iterate the reasons for these changes:
- Testability: if only one place in the code really needs VyOS to work, the rest can be test on developers' workstations and build hosts, by hand as well as with automated unit and integration tests
- Easier syntax changes: same, redesigned syntax can translate to the same abstract representation or a modified version of it, and there will not be need to weed out hundreds instance of the old syntax all over the script
- Transactional commits: if the config correctness checking stage is clearly separated, once all scripts are rewritten in this manner, it will be possible to implement commit dry-run and abort commits if an error is detected before any change to the live system is made, thus greatly increasing the system's robustness
Scripts written in this manner will be reusable in VyOS 2.0 once it's ready with little change, thus ensuring more gradual rather than radical rewrite.
2.0 style command definitions in VyOS 1.2.0
As you remember, the future VyOS 2.0 and its config backend will not be using the old style command "templates" (bunches of directories with node.def files). There is no way to get rid of them in VyOS 1.x, but we still can abstract them away, thus enabling a more gradual rewrite in this area too.
There are multiple problems with those old style templates. They are notoriously hard to navigate even for experienced developers and are a repellent for newcomers. They are equally hard to syntax check and the only real way to find out if they have any chance to work is to install a package on a test VyOS instance and try them by hand.
And last but not least, they allow embedded shell scripts that further spread the logic all over and make debugging even harder than it already is.
New style templates are in XML. Before anyone says "why not JSON", tell me if JSON got a widely accepted schema language and its implementation (I'm aware of some attempts, but...). XML had been machine-verifiable for almost two decades already.
XML interface definitions for VyOS come with a RelaxNG schema ( https://github.com/vyos/vyos-1x/blob/current/schema/interface_definition.rng), which is a slightly modified schema from VyConf ( https://github.com/vyos/vyconf/blob/master/data/schemata/interface_definition.rnc) so the definitions will be reusable with very minimal changes.
The great thing about schemas is not only that people can know the complete grammar for certain, but also that it can be automatically verified. The scripts/build-command-templates script that converts the XML definitions to old style templates also verifies them against the schema, so a bad definition will cause the package build to fail. I do agree that the format is verbose, but there is no other format now that would allow this. Besides, a specialized XML editor can alleviate the issue with verbosity.
Right now that script is complete enough to produce the templates for cron, but there's still work to be done. For example, it doesn't support the "allowed:" statement used for command completion. Any testing and patches are greatly appreciated!
Comments