New Xen updates on RISC-V

Devblog May 23, 2023

In this update, we're going to explore the recent additions to Xen's staging. These updates encompass bug fixes, initialization/checks, and enhancements to RISC-V smoke testing. You can view our initial report at the following location:

Current Xen RISC-V support status
Take a look at our current progress on our RISC-V port to the Xen Project.

Since then, all blockers have been merged. Let's dive into the highlights now.

Spoiler alert: we also managed to boot our first Dom0 on top of a RISC-V architecture while using clean and mergeable code!

Smoke tests

Big ups to Andrew C. for making RISC-V smoke testing a whole lot easier. His patch has now made it so that updating the RISC-V smoke test after rolling out each new feature isn't needed anymore. Before this change came along, we had to tweak the string in the RISC-V smoke test every single time, just to ensure we hadn't messed up anything in the RISC-V build.

What is smoke testing? It's a preliminary testing to reveal simple failures severe enough to, for example, reject a prospective software release. It's very useful and used in the Xen Project to avoid breaking code being merged.

Initialization and checks

We managed to add the .bss section initialization. .bss is a section of an executable that contains declared variables but not assigned. Following this, reading and saving of hart (CPU) id and DTB passed by a bootloader was also merged. As a part of the patch, the start_xen() function prototype was updated. It started to receive the boot CPU id and an address of DTB.

Also, FPU was disabled to detect illegal usage of floating points in kernel space.

Then, Andrew C. (again!) provided linker assertions to check the safety of the zeroing loops for .bss section. In short, the linker assertion is a mechanism to check if all is fine with a binary/exectable during linkage stage of compilation.

Another recent update saw the merging of a new patch designed to maintain all addresses of cpu0_boot_stack (among others) as Program Counter (PC)-relative. This change is beneficial in situations where a binary needs to be relocated. Under such circumstances, we wouldn't need to alter the address of a variable, such as cpu0_boot_stack; we would still be able to access the variable without causing any faults. The issue originated from the pseudo instruction la, which can be transformed to either auipc/addi or auipc/l{w|d}, depending on the .option directive: nopic and pic, or compiler flags. Before the introduction of the patch, la would transform to auipc/l{w|d}. In the case of cpu0_boot_stack[], this would result in the usage of GLOBAL_OFFSET_TABLE, where all addresses would be without taking into account the possibility that the linker address might differ from the load address (meaning that addresses inside got sections will be relative to linker time).

As a result of the previous patch and to be sure that .got section won't be used explicit check that .got{.plt} section is empty was added.

BUG() macro

Since the last update, the most significant feature that has been integrated is a generic implementation of BUG() and its associated macros (originating from bug.h). Within Xen's code base, a substantial portion of bug.h's content is replicated across all architectures. Consequently, it was determined that a new configuration, CONFIG_GENERIC_BUG_FRAME, should be established. The base version of bug.h from x86 was adopted as the foundation. You can access these four commits in these locations: 1, 2, 3 and 4.

While the generic implementation hasn't been utilized for RISC-V yet, it is slated for use as soon as the integration of basic exception handling takes place. Up until now, only the x86 has transitioned to this generic implementation, whereas Arm is due for the switch after some further testing is done.

This further demonstrates that the development of a new CPU architecture aids in streamlining the entire code base, thereby making it cleaner!

Finding bugs and helping Arm arch

During the development of the generic bug functionality, an alignment issue was discovered with Arm's *( section. Entries in this section are expected to conform to a 4-byte alignment, and the compiler will interact with them using 4-byte load instructions. With Arm32, the processor stringently enforces this alignment; any deviations can lead to a data abort. Unfortunately, the current linker script doesn't specify this alignment requirement, meaning we are dependent on the compiler/linker's ability to appropriately pad the preceding sections. This issue was detected during attempts to utilize the forthcoming generic bug infrastructure in conjunction with the compiler provided by Yocto.

MMU and exceptions

Aspects relating to the Memory Management Unit (MMU) and basic exception handling are currently undergoing code review. This is the next element that is set to be prepared and integrated into Xen's staging branch. You can track the ongoing discussion on the Xen mailing list by searching for the following topics:

  • Enable MMU for RISC-V
  • RISC-V basic exception handling implementation

As you can observe, significant progress is being made across numerous subjects!

Booting a Dom0

The other big progress is the ability to now boot a Dom0 with clean code that could be merged upstream soon, without any conflict. This project is still ongoing, but it's a clear indication that we're making rapid strides! You can find the most recent updates (in the latest branch) in the following location:

Files · latest · xen-project / people / olkur / xen · GitLab
Mirror of the Xen Repository (PRs not accepted see:

If you want to test it yourself, you can do the following command to build Xen:

CONTAINER=riscv64 ./automation/scripts/containerize make XEN_TARGET_ARCH=riscv64 -C xen build V=1

And then to run it, you need to use Qemu this way:

qemu-system-riscv64 -M virt -smp 1 -nographic -m 2g -kernel xen/xen -device 'guest-loader,kernel=<path/linux>/arch/riscv/boot/Image.gz,addr=0x80400000,bootargs=rw root=/dev/ram console=hvc0 keep_bootcon bootmem_debug debug"

That wraps up today's update. Thank you for your time and interest in reading this piece. I hope you found it enlightening and useful. Be sure to stay connected for more thrilling updates and insights in the future.


Oleksii Kurochko

Along with Olivier Lambert

Hypervisor and Kernel Software Engineer at Vates. Focused on porting Xen Hypervisor to the RISC-V platform.