Online Xen Summit 2020

Event Jul 13, 2020

It's really hard to sum-up this kind of summit, especially because Xen is a large project with an incredible diversity of contributors in different areas. However, we'll do a quick wrap up here from "our" point of view, which is more focused on server virtualization!

First online summit

Due to the worldwide pandemic situation, this summit was held online. Initially, it should have been in Bucharest (Romania). We'd love to travel there, but health first!

Usually, a Xen Summit is a kind of a mix between a "user summit", "dev summit" and a "hackathon". That's why there's some talks (classic format) but also design sessions, which are very interesting because teams work together to find a solution and common approach, with more interaction between people.

Finally, there's also the "Break & Hallway Chats", trying to emulate what's going on physically during these events: meeting people outside "rooms" and trying to discuss various topics (and to know new people!).

George Dunlap doing the kick-off for this Xen Summit 2020
As soon as we have session recordings, we'll put them here!

Schedule

For those who want to discover the schedule, it's available here. So many topics in 4 days! It was really intense, and we tried here at Vates (as a sponsor too!) to follow a maximum number of sessions. Topics covered were huge, I'll try to sum them up into different categories.

Xen Security

It's likely one of the biggest topic these days. Because of its design, there's a lot of really interesting things you can do with Xen and security: advanced isolation, memory introspection, etc. No wonder why it's used in ultra-secure products around the world (even in very sensitive places). Main topics discussed were:

  • dom0-less systems on x86 as well as ARM (domB)
  • VM forking and fuzzing via Xen
  • reducing Xen's attack surface via Direct Map removal (secret-free Xen)
  • Xen for hardened access terminals
  • Xen system boot attestation (with DRTM and TPM2)
  • VM memory introspection with Xen
  • Using CET in Xen

It's interesting to see there's a lot of traction there, and I believe it is mainly because of Intel security issues revealed over the last 3 years.

Embeded world

Xen is heavily used in the embedded world: from the automotive industry to security appliances. ARM is the main platform in this world:

  • Self-driving cars and Xen
  • Xen with U-Boot
  • PCI passthrough on ARM
  • Cache coloring (for less cache contention in real-time systems)

Despite the embedded world not being our current focus, it was really interesting to see how you could use Xen in different contexts. Some features might be interesting in the x86 world in the future too (eg: cache coloring).

Xen features and performances

In parallel with all the previous talks, there's also a lot of discussion regarding the future of your favorite hypervisor. This is obviously where we've been keeping an eye mostly. It helps to understand what's next, and how we could leverage that for future XCP-ng releases.

  • Hypervisor file system (like sysfs but for Xen)
  • Future of reference counting in Xen
  • Live upgrading QEMU
  • pvshim perf improvements (and tooling to do so)

Beyond those sessions, a lot of discussion happened in parallel about performance related topics. Hard to sum up, but it's clearly going into interesting directions!

Xen Project improvements

The Xen project is a massive collaboration between different large entities (Citrix, AWS, Suse, ARM, Intel, AMD etc.) and also smaller ones (security companies, or even... us!). Being able to have good coordination between everyone isn't trivial. That's why there's always discussions on how to reduce friction. In other words, trying to eliminate time wasted to manage/coordinate vs actual contributions.

We are also very interested in this, because it will help us to improve the Xen project directly, in an easier manner. Main topics were:

  • Better CI for Xen project
  • Optimizing roles in the project
  • Gitlab based development flow

The last one was very very interesting for us. Right now, the "mailing list" workflow is used within the Xen project. There are a lot of powerful things you can do with it (offline patch review, "fine-grained" workflows, integration within your own tools to be really efficient etc...)

However, there are some drawbacks too, especially versus GitLab: it's harder for new people to come and contribute (let's face it: mailing lists are austere), harder to follow the project "from outside", same for review.

That's why discussions around adopting Gitlab are in the pipes. But this is a big change in current workflows, especially in the code review phase. I personally understand why it's hard to change that, but testing this at small scale first might be a great opportunity. An opportunity for Xen project to be more transparent for "external" people, and potentially lead to more contributions directly inside it.

For people who want to learn more: read this great documentation about the GitLab flow!

Xen on RISC-V

There is currently an effort to port the Xen hypervisor to RISC-V, which would bring a new architecture to the table alongside x86 and ARM.  We are leading this effort at Vates with our Xen engineer Bobby Eshleman, in collaboration with Alistair Francis of Western Digital and the Xen community, working to implement the fundamental features of the hypervisor on this new architecture.

Some of the points discussed include:

  1. Without an IOMMU in RISC-V yet, it will be necessary to identity map passthrough devices.
  2. The lack of interrupt injection in the RISC-V PLIC specificiation is an existing issue and is being addressed by the RISC-V architecture team.  Trapping on guest PLIC access will suffice.
  3. The Xen/RISC-V project may avoid issues seen on other architectures by clarifying early the interaction between the virtualization sub-architecture and the cache (A specific issue on Xen/ARM32 was discussed, in which guest ownership of its cache view could result in incoherency between the guest cache and RAM).
  4. Upstream is interested in carrying a minimal RISC-V build in CI to help reduce the overhead of rebasing for the RISC-V port.
  5. Start upstreaming general cleanup now instead of later.

We will be posting further functional and technical updates as they occur on our blog!

Feedback of XCP-ng and XO on the field

Inside all those deeply technical talks, there's also sometimes feedback of "end users" on the field. This is of the utmost importance to us because this is where we learn about the good and the bad of using Xen in the real world.

This year, it was a talk called "Xen Hosting: Running Servers in Production" by Larry Sawyer of Sawyer Networks.

An example of how XCP-ng and XO are used on the field

Conclusion and thoughts

This was great summit: despite the inherent difficulty of gathering people from all over the world at a virtual event, the distance, different timezone, technical glitches. Despite all of that, it was a really rich event.

Personally, I find this kind of virtual event even more exhausting than physical events. Also, it's a bit harder to bond with people. But in the end, it's relevant to exchange and discuss about various topics.

Will it replace "in real life" summits? Clearly, no. At least, not for me. But it might be complementary, in a way that instead of one "physical" summit, we might have one extra smaller summit 6 months after? It won't hurt to increase communication and have more design sessions: trust me, those are intense and very helpful to move forward!

Tags

Olivier Lambert

Along with Bobby Eshleman

Vates CEO & co-founder, Xen Orchestra and XCP-ng project creator. Enthusiast entrepreneur and Open Source advocate. A very happy Finnish Lapphund owner.