Updates on the Rust guest tools
In this blog, we reflect on the progress we've accomplished in recent months with our novel VM guest tools - this time, constructed entirely from scratch using Rust.
To remind you the roots of this project, here is our initial blog post on the subject:
Yann Dirson
Despite some complex challenges (more on that later), we've successfully realized a robust, initial version of the release. While it's not yet deemed "stable", it has most certainly graduated from its alpha phase to "beta"- and as you're likely aware, even our beta releases like 8.3 have proved considerably steadfast.
Let's dive into what we've achieved!
Achievements
Let's break each down individually.
A complete README
It's very important and we did it: a comprehensive README file with the goals, design, a how-to build the binary and finally a how-to run it.
Go ahead a take a look! https://gitlab.com/xen-project/xen-guest-agent#xen-guest-agent
Drop-in compatibility
When conceptualizing these new tools, we committed to ensuring they're wholly compatible with your existing XCP-ng toolstack. In simpler terms, installing our new tools won't hinder any external elements, as they retain the same conventions to store guest data. This assures us that Xen Orchestra, for example, will accurately reflect all pertinent information such as IP addresses, distro version, RAM usage and so forth.
As an added bonus, we've eliminated redundant data left over from previous guest tools - less is indeed more!
Alternative schema
For cases calling for reporting data differently, it is possible to change their format, in new ways that will deliver superior results. Even though our initial aim was complete compatibility, it doesn't exclude our aspirations for future improvements to the data schema. By default, we have constructed a retro-compatible model, named 'std', versus the new adaptable model, termed 'rfc'. More details in the usage documentation.
For a more comprehensive overview of this work-in-progress proposal to improve guest data reporting, visit https://gitlab.com/xen-project/xen-guest-agent/-/blob/main/doc/structure.md
Netlink as first class citizen
Netlink, a socket family that facilitates communication between (guest) kernel and user space processes (More Details Here), plays a pivotal role in our toolkit. It serves as an efficient tool to notify any network changes in your VM—a far superior model than checking for changes every few seconds, a method used by previous guest tools. This means that the new tools offer quicker and more efficient updates on any network changes in the guest!
The first Linux-only prototype we published already made good use of Netlink. Now a fallback system has been implemented to allow guests without Netlink support to report networking information. Those guests would not benefit from the lower-overhead data collection that Netlink allows, and need to enumerate all interfaces and look for changes every minute, just as xe-guest-utilities does today.
Not restricted to Linux
Our efforts extended to making this agent compatible with other UNIX-family systems, such as BSDs. These tools should also function seamlessly in your favourite BSD distro, like FreeBSD or its derivatives such as TrueNAS and others.
The ongoing challenge however, is to make it as efficient as it is on Linux guests, as FreeBSD 13.2 recently introduced Netlink support. However, as Netlink was until now a Linux-only interface, and Rust support for Netlink did not anticipate its spread to other operating systems. Getting Netlink to work on FreeBSD requires some work on various components. We are pushing to get it upstream, but it's a complex issue, see:
- the Rust libc create has to expose the gory details
- but this first step exposed a conflict with an older BSD-specific interface
- ... and then the
rust-netlinkcrate will need some restructuring to accept to build when not everything provided by Linux is available (and its main contender, thenelicrate needs the same amount of work)
As you can see, we are not sparing efforts to make it happen!
xe-daemon process from xe-guest-utilities package first.Rust xenstore library
We not only achieved substantial strides with our new guest tools, but we also contributed significantly to the existing upstream Rust xenstore project.
Rust libxenstore bindings needed enhanced API coverage, and our contibutions have been successfully incorporated. Meanwhile, we are ensuring its constant upkeep by financially supporting the current maintainer.
Modern builds, Security & dependency checking
Another key consideration in redesigning these tools was to facilitate reproducible builds, to which Rust's static linking and fast-moving ecosystem is a challenge. In this regard, we have initiated CI infrastructure to maintain checks and balances, notably Dependabot on the unofficial GitHub mirror.
You can monitor updates on our builds in the repository here: https://gitlab.com/xen-project/xen-guest-agent/-/jobs
While additional work is still required, we've embarked on utilizing Dependabot, as it shows promise in signaling any known vulnerabilities in dependencies and proposing PRs.
Note that while Dependabot is simple to activate on GitHub, i will require some IT work to properly activate on GitLab. And while setting up a Gitlab mirror of a Github project is breeze, likewise configuring an automated GitHub mirror of a GitLab repository needs careful setup.
We'll detail our work in that security area in a following blog post.
Code base improvements
Our decision to utilize Rust on this maiden project has rewarded us with enriched experience, allowing us to pen code that embodies more "Rust-like" characteristics. This project provided us an ideal springboard into the world of Rust, and the evolved code mirrors this growth.
