Current Xen RISC-V support status

As some readers may know, we've been working on porting Xen to RISC-V.  If you are one of those who didn’t read the previous article, you can have a look here:

Xen and the RISC-V Hypervisor Extension

As some readers may know, we’ve been working on porting Xen to RISC-V. This blog looks at why we care about RISC-V and how RISC-V satisfies what is needed from an ISA in order to support virtualization.

XCP-ng BlogBobby Eshleman

In this new blog post, we will describe the current status we had before we start sent patches to mainline with RISC-V support and what has been done since.

What do we have now

Mainline Xen could build only head.o and no other TARGET was currently supported.

head.o could be built with:

$ make XEN_TARGET_ARCH=riscv64 SUBSYSTEMS=xen -C xen tiny64_defconfig
$ make XEN_TARGET_ARCH=riscv64 SUBSYSTEMS=xen -C xen TARGET=riscv64/head.o

The current support status is that the patches from Bobby Eshleman and Alistair Francis have been provided and sent to the Xen-devel mailing list, but they weren’t merged yet and required reworking/refactoring.

One of the reasons is that the patch series is too big and not convenient for review. Another reason is that some parts of the patch series are unrelated to RISC-V and are gotten from other architectures (x86, ARM) or projects (Linux).

What was done & what's next

It was decided to start committing and reworking some patches step by step.

Minimal build

Make Xen RISC-V buildable which should require as minimal as possible. So at this step, we have only a buildable head.S file with an infinite loop inside it. Also, a minimal linker script was provided, updated Makefile with targets to produce minimal Xen binary, and arch.mk was updated with overridden ALL_OBJS and ALL_LIBS to compile only head.S.

It has already been done and the patch has already been merged.

Cross-build test

Add RISC-V cross-build test on Xen’s GitLab CI&CD. The patch aimed to continuously check that other changes related to and not to RISC-V don’t break at least the RISC-V build. During the step, build.yaml was updated and added a RISC-V job.

The patch has already been merged into the mainline!

Beyond build

Jump to C world and print the message “Hello from C environment”. It required setting up a stack and introducing xen_start function and early_printk functionality. Mostly it was merged to the mainline and only patches were left to be merged. Also, the step headers such as asm/types.h were introduced and SBI stuff related to the console.

Printk feature and the what's next

Other patches related to the topic “Early printk functionality” can be found in discussion with the subject “xen/risc: introduce early_printk basic stuff” in the Xen-devel mailing list.

1. Introduce smoke test and job to verify early printk functionality. It required an update of the RISC-V container with qemu-system-riscv64 and added of sh script to grep “Hello from  C env” from QEMU’s output. The patch is still under review and can be found in the Xen-devel mailing list.
2. Introduce basic trap handling. To show that basic trap handling works were decided to implement macros from asm/bug.h header, context save/restore and introduce some new headers. The patch series has already been sent for review and can be found in the Xen-devel mailing list.
3. Update the smoke test to verify the functionality from asm/bug.h. It is a part of the patch series.
4. While working on the previous patch series, the separate subtasks occurred related to asm/bug.h as asm/bug.h functionality was based on Arm as it is the most portable, so there is a sense to make it generic across the architectures (x86, Arm, RISC-V). The patches have been sent to the Xen-devel mailing list and are awaiting review.
5. Basic MMU support: set up page tables, jump to the first instruction after MMU is turned on, and add additional tests to do page fault. The step hasn’t been started yet and is considered the next step.
6. Various other bits of functionality, one at a time, e.g., smp support, console, enable xen/common, etc.

What problems we have encountered so far

Obviously, porting an hypervisor (even a micro-kernel like Xen) to a new arch isn't an easy task.  While working on RISC-V support the following issues were met:

1. It is needed to commit a lot of empty headers from asm folder which will be empty till the end of RISC-V support. Andrew Cooper started discussing this topic and suggested some solutions.
2. Some things related to x86 or Arm and won’t be used by RISC-V are present in the patches provided by  Bobby Eshleman and Alistair Francis and should be refactored or reworked.
3. Some things should be reworked and made more generic and it consumes a lot of time and investigation on how it works and how to make it more generic. For example, macros bug.h has to be generic between architectures ( at least RISC-V and Arm).

But all fo this is a great demonstration that porting Xen to RISC-V could also help to make the Xen code base better and more generic, even if it's more work on the short term, it's better for the project on the long run!

Conclusion

Work on porting is still in progress (I'm working full time on it), and it's far from being done. However, there's a lot of momentum going on with our work on RISC-V+Xen, as there is significant support and interest from the Xen community itself and even beyond!

If someone would like to help, can give useful advice about RISC-V or is interested in the topic feel free to join the Xen devel Community Matrix room.

See you again and stay in touch 😉