Making Rust a first-class citizen for Xen
Our team at Vates is collaborating on several Xen-related Rust projects, including the Xen Guest Agent and xcp-metrics. We're leveraging Rust's strengths in low-level systems like hypervisors and operating systems to build robust and high-performance components.
Olivier Lambert
Yann Dirson
Xen provides low level libraries for C (often packaged as xen-libs or libxen-dev) with wrappers for OCaml and Python. As of now, there is no official wrapper for Rust. Some unofficial wrappers exist though (e.g xenstore-rs, xenctrl), but they inherit some of the issues of C libraries and incur significant complexity and packaging overhead as highlighted by Yann Dirson during Xen Summit 2024.
🌩️ C bindings in Rust are hard
One approach that is often used for making use of a C library in Rust is to make the C functions usable from Rust (often named somelibrary-sys) and make a "safe" wrapper on top of it.
We can use bindgen which generates a "raw" interface for Rust from a C library header, providing us the C interface as-is, clunky to use directly (unsafe and pointers everywhere !). We build on top of that of nicer wrapper that tries to use the C interface properly and expose it in a better fashion.
It mostly works, but there are some caveats :
- you still depend on the C library (causing potential packaging complexity)
- you have many points of failures (the wrapper, the binding generator, and the C library)
With Xen Rust wrappers, due to the varied needs of the users, we also have to deal with loading the library at runtime, as some users want to use Xen interfaces only if the C libraries are available.
Static linking and Rust (C binding edition)
Xen Guest Agent uses xenstore-rs to expose the metrics to the hypervisor.
For practical reasons, we may want to not rely on xenstore dynamic linking as:
- It is subject to library's ABI changes: if a distribution wants to update to a Xen version that for some reason needs to update the xenstore library in a incompatible way, it may break the guest agent. This is also very problematic for
libxenctrlwhich shares the same ABI to Xen hypercalls, which is currently subject to future breaking changes. - In some cases, xenstore library is bundled along other Xen stuff, making the guest agent indirectly bloated.
xenstore-rs has some support for static linking, but it's not really practical maintainance-wise, in some cases, we want to use the library in a static-linked fashion, in other cases, we want to dynamically link the library at runtime (e.g check if xenstore is available before using it). Either of these mode may impact the code of the wrapper.
Dealing with bindgen issues
We use bindgen to have access to raw C functions from Rust. While it works most of the time, it may break due to a regression, which needs manual updating of the dependencies of the offending crate. Note that updating the bindgen version may cause issues due to unexpected changes that can affect the raw C interface we get from bindgen (as there is no 1.0, semantic versioning allows breaking changes even between minor versions).
All in all, while C binding works most of the time, it requires regular maintenance and testing to not break.
🚀 Pure Rust crates for Xen
Aside C FFI capabilites, Rust is also very capable of using OS-specific low level interfaces directly (even inline assembly !). That can provide a solid, alternative approach for high-quality Rust libraries.
Pure-rust xenstore crate
While using the xenstore C library may seem convenient, we still have to deal with all the aforementioned issues. But what if we actually try to not rely on it ?
The C library is not completely magic, in fact, it's actually pretty small (less than 2 kLoc) and not very complicated.
xenstore interface for Linux/BSD userland
There are actually two low-level interfaces to access xenstore from a userland program (e.g guest agent). Both of these interfaces works the same way, only one of them is usable at one time (depending on the kind of domain).
- xenbus device (at
/dev/xen/xenbus)- the interface exposed by the Linux kernel for DomUs when running under Xen
- xenstored unix socket (at
/run/xenstored/socket)- this is Dom0 xenstore backend socket
These interfaces works using regular read/write syscalls and follows a simple protocol defined in xenstore specification and xen headers.
We can easily implement this protocol relying only on Rust standard library (and no unsafe code), and also make a asynchronous implementation (by relying on specific xenstore features) for Rust async runtimes like Tokio.
This implementation doesn't rely on the original C library while having more features (e.g async interface with concurrent requests) and integrates better with the Rust ecosystem. This new xenstore implementation will serve as a base for xenstore crate 1.0 and planned to be used for the Guest Agent (making it actually pure rust).
Wenzel
xenstore interface for Windows userland
Unlike Linux, Windows doesn't have a xenstore library and doesn't expose the same low-level interface as Linux. However, there is still a interface we can use, xeniface driver exposes a IOCTL interface for userland (through a specific device).
This interface is not as practical as on Linux with devices and regular system calls, but we can still interact with it with the windows crate and regular Windows API.
TSnake41
I made a experimental crate that implements the traits of the xenstore crate.
🐼 Xen crate
There is an ongoing upstream initiative to develop a pure-Rust crate for interfacing with Xen hypercalls and exposing various hypervisor primitives across both hosted platforms (e.g., Linux, BSD) and freestanding environments (e.g., kernels, unikernels, embedded operating systems).
This effort aims to provide pure Rust-based alternatives to existing C libraries like xenctrl (for hypercalls) and xeneventchan (for event channels), serving as a foundation for future Rust projects within the Xen ecosystem.
Recent progress includes a Request for Comments (RFC) on integrating this work into the Xen source tree and discussions on the crate's design. Notably, the xen-sys crate is been developed to offer Rust-native Xen hypercall and basic interfaces support, with another xen component for exposing more idiomatic interfaces. These could be used for instance to empower oxerun (from rust-vmm) for Xen unikernel generation.
Additionally, a series of patches titled "Introduce xenbindgen to autogen hypercall structs" has been proposed. This series introduces a hypercall ABI IDL parser and generator to the Xen tree, replaces certain existing hypercalls, creates a Rust crate with autogenerated content, and establishes a CI job to ensure consistency.
🛠️ Next-gen toolstack for Xen
Building on top of these Rust projects, we aims to build a new low-level toolstack for Xen. It aims to replace libxl and be used in XCP-ng (among others) to provide low-level Xen guest and host management primitives.
This work is still in its early stages and is part of the upstream toolstack refactoring initiative. At Vates, we are proud to lead these efforts and have established a dedicated working group to accelerate progress on this important project.
