@MajorP93 said: What you are describing must be a storage live migration then (XenMotion). That should work across pools, yes. I stand corrected because I did not factor in this variant in my initial answer. Yes, that's it - sorry - I should have been more explicit, even if 'lumbers along' wasn't a hint //EDIT: also there is no such thing as non-pool hosts. Even a single host setup has it's own pool. Pedantic, but true I technically meant 'not hosts in the source pool' I guess.

Your write-up is the kind of thing that saves the next person a weekend. One thing worth knowing before you commit to the whole-controller route. The passthrough page has a "Passing through Keyboards and Mice" section further down, and it says XCP-ng ships /etc/xensource/usb-policy.conf with DENY rules for mice and keyboards by default. You edit those to ALLOW, then refresh with /opt/xensource/libexec/usb_scan.py -d followed by xe pusb-scan host-uuid=<host_uuid>. It's at https://docs.xcp-ng.org/compute/ under USB Passthrough. I have no idea whether that covers your USB-to-serial adapter, which is a different device class, and passing the whole controller may still be the cleaner setup for two discrete workstations anyway. Might be worth a mention to @Team-Hypervisor-Kernel on the display question, because that one still puzzles me.

@poddingue Hello, sorry for the delayed response. We had some issues with the Supermicro IPMI and were waiting for a resolution. Here are the two log files: dmesg_xcp-ng-8320250606-aacraid-legacy.log dmesg_xcp-ng-8320250606-aacraid-uefi.log The alternative kernel did not work with your xcp-ng-8.3.0-20250606-aacraid.iso. I am also providing the log after installing the standard xcp-ng-8.3.0-20250606.iso: The files can be found at https://drive.google.com/drive/folders/1H-0FEruGK26oclD2WInDSOBJhVW2k8Xz?usp=sharing, as I wasn't able to upload the log files directly to the forum. I am also ready to participate in any further testing of new images, up to and including XCP-ng 9.0.

Hello @mgr42 I can confirm that indeed today AMD iGPU passthrough is not working on XCP-ng. I am currently (slowly) working on it. There are other references about this topic in the Forum and on GitHub FYI: https://xcp-ng.org/forum/topic/11918/amd-barcelo-passthrough-issues-any-success-stories https://xcp-ng.org/forum/topic/8909/issue-to-load-gpu-passthrough-invalid-pci-rom-header-signature-expecting-0xaa55-got-0x4556 https://github.com/xcp-ng/xcp/issues/806 https://github.com/xapi-project/xen-api/commit/930ffb34e42d674b8234c3d8399fdba97efa6b38 On the setup I am working on, the Expansion ROM is provided indeed by firmware via the VFCT ACPI table. I dumped it to a file and provided it to qemu-dm xen-pci-passthrough device via the romfile= argument (via a modified XAPI build). I can now indeed see a ROM bar in the guest but unfortunately its content is not the content of the file therefore nothing works (amdgpu driver is unable to get its ROM and fails). So now I'm debugging qemu to try to understand the issue with the emulated/passed-through ROM BAR. Regards, Yann Kernel & Hypervisor team

@yllar I'm not sure, maybe it'll slip to July. But what I can say is that the ISO is currently being tested in our QA process. Brace yourself, it's coming soon

Good to hear, thanks a lot for your feedback.

That leftover NIC is probably a stale PIF that XAPI is still holding, even though the card is physically gone, and the Refresh button re-scans rather than removing it. The docs have proper "remove a physical NIC" steps that end in forgetting the old PIF with xe pif-forget: https://docs.xcp-ng.org/networking/#remove-a-physical-nic. For the GPU side, the PCI passthrough flow (hiding the device from dom0, then assigning it) is here: https://docs.xcp-ng.org/compute#detaching-a-pci-device. If the GPU still won't show up as assignable once the stale PIF is gone, it might be worth a mention to @Team-Hypervisor-Kernel.

@poddingue Thank you kindly! Honestly, whatever organizational structure you think is best is fine by me.

@maximsachs Thanks for getting back to me on this. I think (and hope) that in the mean time we will have released an official new ISO that fixes the issue you are having. So, you will be better off testing the new official ISO rather than this unofficial one I'll try to ping here when the new official ISO is out. Thanks! Regards, Yann

Thank you all (@olivierlambert , @teddyastie ) for your valuable responses

@tjkreidl I found that some Intel PRO GPU card have SRV-IO without any additional license, but this functionality is supported in Linux kernel 6.12 and up. I hope in new XCP-NG will have 6.12+ and support GPU sharing.

@taghjichte From my testing you would need to passthought a device for audio. If need for professional audio work then yes a pci passthough would be the perferred option. With that said. @dinhngtu As for sound I have never looked much into it but the only vm I heard make sound is from Fedora even while booted from install iso. Kubuntu or windows with xen drivers installed no audio. (find for me at the moment) While audio seems to work in Fedora from iso it seems to be limited to left channel. Local system is connected to 5.1 sound.

Nice!! Thanks for the feedback @mattrc , it's really cool to see that you can fully enjoy your machine

From what I understand, the OS-to-iDRAC pass-through is a direct USB link between the host and the BMC, designed for the hypervisor host itself to talk to the iDRAC, not something that would naturally bridge through Xen's networking layer into a VM. That might be why the PIF shows up in XCP-ng but the VM still can't reach 169.254.0.1; the bare metal case works because the OS sits directly on the hardware with no virtualisation layer in between. If you need the VM to have direct BMC access, USB passthrough might be worth exploring; XCP-ng does support passing USB devices through to VMs (https://docs.xcp-ng.org/compute#usb-passthrough), though I'm not certain the iDRAC virtual USB NIC would show up as a passthrough-able device. Might be worth a mention to @Team-Hypervisor-Kernel to find out; others here will know more than me.

Hello @codemercenary In theory Xen is NUMA aware (See https://wiki.xenproject.org/wiki/Xen_on_NUMA_Machines) so you would leave memory interleaving option of the BIOS off so that Xen does what it thinks is necessary. But these kinds of settings are very very workload sensitive and in fact it's usually a good idea to just test what works best for you. Regards, Yann

@yannsionneau Thank you very much. Please let me know if there’s anything I can help with.

@gleveque Hello Can you retry with an up-to-date XCP-ng 8.3 host? PCI expansion ROM are now enabled when doing pass-through. The guest VM can now access the ROM bar. Regards, Yann

I've seen this happen after a BIOS update or if a CMOS battery starts failing. Sometimes the motherboard resets to 'Legacy' instead of 'UEFI' (or vice versa), and GRUB just loses its mind because it can't find the partition it expects. It’s worth a quick check in your firmware settings to make sure the boot mode and the drive order haven't shifted on you.

@TeddyAstie I hadn't seen anything special was needed in a VM other that a kernel new enough for support which 6.17 has, is there more to the story with adding things to Ubuntu?