OpenBSD & XCP-ng 8.3, virtio (Net&GuestTools) no longer works.
-
I've updated a few systems from 8.2 to 8.3.
What has happened since is none of my OpenBSD VMs were accessible.It looks like the Xen drivers are incompatible with the newer Xen virtio drivers as the network interfaces would show as active devices but couldn't be assigned addresses (the device names show up as 'rl#' instead of 'xnf#.' When I changed the network interfaces in advanced from RL8139 to Intel e1000 network started working again which narrowed me down to the virtio drivers.
Taking it a bit further the hostctl tool communicates with the hypervisor. I was able to get OS name, and the updated em0 devices communicating their IPs back to the management agent. I needed to adjust the script to look for the em# interfaces instead of xnf# interfaces.
With the Xen tools not being detected it doesn't get the shutdown/reboot requests which requires 'forcible' shutdown now.
I'll add more info to this thread as I narrow down the final details.
-
When doing a fresh install (7.6, install76.iso, amd64), the installation detects the network interface as re0 (as opposed to xnf0). Set the device to 'autoconf' and get kernel 'er0: watchdog timeout' and fails out assigning an IP.
dmesg sees the device as:
re0 at pci0 dev 4 function 0 "Realtek 8139" rev0x20: RTL8139C+ (0x7480), apic 1 int 32 address [MAC ADDR]Power off VM and change to e1000 (from Advanced page of VM) and it detects fine and pulls a lease with dhcp.
Also using BIOS and UEFI setup didn't boot post setup (i think I'm missing the efi partition or something).
-
@nullstream-0 Correct, OpenBSD 7.6 does not correctly recognize and use all the Xen PV devices and therefore uses the hardware emulated devices for both disk and network.
-
@Andrew respectfully, 7.5 has the same issue as does 7.4. As well Xcp-ng 8.2 worked perfectly fine for the Xen PV devices. Totally reproducible.
- Install Xcp-ng 8.2
- Setup/import OpenBSD VM.
- Boot and see xnf devices just fine.
- Upgrade to Xcp-ng 8.3
- Boot the same VM and see the xnf devices show up as re devices (realtek driver) and watch interface give re0: watchdog timeouts during an install but if VM already built you'll be able to see the re# devices but can't assign addresses to them.
The difference appears to be the xen version difference between 8.2 and 8.3.
-
Hi @nullstream-0, have you tried booting a -current snapshot to see if something was changed post 7.6-release?
What does dmesg look like on the 7.6 VM? In particular, lines related to pvbus, xen, xbf, and xnf (
dmesg | grep -E "OpenBSD|pvbus|xen|xbf|xnf")..joel
-
I was able to root cause this, and with a small kernel patch, the PV drivers attach now. I've submitted the patch to OpenBSD for review (https://marc.info/?l=openbsd-tech&m=173154320510806&w=2).
One a side note, if anyone is reading this and knows how to "hide" the 9pfs device from a VM, such that the VM doesn't try to probe the device and load a driver for it, that would also likely solve this issue for OpenBSD VMs without needing a kernel patch.
.joel
-
Ah great news, thanks for keeping us posted. Indeed, there's a large Xen version bump between 8.2 & 8.3
Great job for the investigation and the patch!
