Edge Case #3 - Xen and Raspberry Pi
Few months ago, an article on Linux.com presented the story behind Xen running on the Raspberry Pi. We'll see why it's important, and how it could shape the future of Xen. It's more than just a technical story, the future of computing is being written as we speak.
Welcome to "Edge Case" number 3, our series talking about Edge Computing! Let's talk about Arm, shall we?
A bit of context
Back in 2011, I was myself a firm believer that Arm CPUs would succeed very soon© inside the data center, competing on the market against x86 servers.
But Arm shortcomings were too high back then (at various levels, not only technical, even if 32-bits was one of the main problems). In short, one of biggest players, named Calxeda, simply failed. As the spearhead of the idea to compete massively with x86 in the server world, it de facto froze Arm success in data centers for the next… 10 years at least! (with few exceptions)
Fast forward to now: the compute world evolved, it's not only inside the data center. Edge Computing is a fast growing market, where Arm CPUs can shine: energy efficiency when compute power isn't that important in most situations, especially versus the traditional server market.
Want a recap on Edge Computing? Go read our introduction to Edge here!
Xen on Arm: not a new thing
Xen running on Arm isn't a new thing. In fact, it was "ported" back in 2013, as you can see here:
For curious people, you can read this interesting "white paper" on the architectural choices made.
However, unlike x86, the Arm world isn't straightforward: unlike Intel or AMD, Arm (now Nvidia) isn't selling chips, but licenses. You purchase those licenses to build your own CPU with ARM instructions, usually a SoC (System on chip). This lead to a big "problem" vs x86 behemoths: fragmentation.
I made a personal blog post about this problem, but it's not the core of this article. Let's focus on one solution to this fragmentation issue. And one solution is a platform that's almost everywhere, because it's cheap and it works pretty well with an amazing community.
Raspberry Pi: de facto standard?
Last time I asked out loud "is there any standard Arm platform we could target one day if we were to run Xen on Arm for a general purpose virtualization platform?". This was the kind of things I'm often wondering about, and luckily a friend of mine was around and answered: "what about Raspberry Pi? Almost everyone has one!". And it's true.
Despite its own shortcomings, this little Arm board is cheap and sold by the millions (35! 🤯). That is one way to become a standard, the de facto way. Not the "best" one technically speaking, but still a pretty effective one. And the latest version of the board has what's needed to do virtualization!
Not that easy
So now you have a board that's "compatible enough, so it should work out-of-the-box, shouldn't? Well… not quite. Because this little board is a bit unique in several ways:
- Low memory addresses (only first 1GiB RAM)
- Specific kernel for memory translation
- Specific DMA mechanism
No wonder Xen devs had issues making it work "out-of-the-box" 😆 . You can read in depth about these problems in the original Linux.com article by Stefano Stabellini, one of the Xen pioneers in the ARM world, and Roman V Shaposhnik (one member of EVE project, see below).
Why it's important
Having Xen running on this board is more than just a fun technical story. A lot of things can come out of it, especially in the Edge Computing world. Because you can use those cheap boards to make prototypes and proof of concepts using Xen (or even use them in production after all!). And because Xen is great in terms of isolation, you are basically bringing security to the edge, even on small devices. It will push the community to get more people testing it, playing with it and bringing innovation into the Xen world. Having a great technology like Xen is one thing, bringing it to the mass is another one, and I believe this is the key to succeed.
Edge Computing: Adam and EVE
At some point, Edge Computing and IoT use cases will require a coherent framework. One of them is EVE, a project meant to be "the Android of Edge". And it's a Linux Foundation project inside LF Edge.
The EVE model is interesting: you have a controller (Adam) and then you have your devices on the EVE stack. Your controller can run into a traditional virtual machine, let's say on XCP-ng 😉. This controller will talk to the devices, that can be Raspberry Pi 4 running with Xen (but not XCP-ng, which is another story). We are obviously keeping an eye on this project, because we could find ways to get integration between our current stack (XCP-ng/XO) and EVE stack, to have the best of both worlds (compute and edge).
What about XCP-ng?
The easiest path to be included in Edge Computing while being Arm friendly is to work with LF and EVE project. XCP-ng + XO is a great platform that might be able to control and compute what's needed on the Edge, even indirectly, for example by hosting an EVE controller and exposing Edge features in Xen Orchestra. Remember XCP-ng can run on Edge hardware (eg from Protectli, HPE, All Terrain IT and so on…), acting as a control stack for all smaller Arm devices.
