XCP-ng 8.1 PCIe onboard usb controller passthrough does not work
Sorry for starting a new thread but i dont know if the old one was in the correct area for people to see.
I have XCP-ng 8.1 beta installed and im trying to passthrough my onboard usb controller on my gigabyte X79-UD5 and vm cannot be started.
This is the error message i receive: "Failed","Starting VM 'test'
Internal error: xenopsd internal error: Cannot_add(0000:00:1d.0, Xenctrlext.Unix_error(30, "1: Operation not permitted"))
Time: 00:00:08","xcp-ng-vkdehkmr","Feb 3, 2020 1:09 PM"
I know this worked fine in Xenserver 6.5 SP1. i have tested 8.0 but have problems even running default VMs on it and i believe this passthrough blocking issue is a thing that came in Xenserver 7 or something has i temporarily switched to proxmox around that time for this reason IIRC.
I believe there is some rule blocking this since it is usually not stable? maybe like USB passthrough usb-policy blocking some usb class devices.
Thanks in advance
Do you have all hardware virt options enabled?
All that exist in my UEFI/BIOS yes ofc. VT-d and so on. im running an xeon E5-1660v1 btw
Does this page help in any way? https://github.com/xcp-ng/xcp/wiki/USB-Pass-Through
No, i tried that and its doesnt work for keyboard and mouse even with the usb policy change, my keyboard and mouse shows up in the gui and VMs start fine with no errors but they still control dom0 and not the guests.
it does not matter because the topic is actually about pcie passthrough that refuse to passthrough my onboard moterboard usb. i think there is some configuration that citrix has implemented.
I'm experiencing the exact same issue on 8.1 stable. I'm able to passthrough and use any PCIe devices except USB controllers. There must be something preventing USB controllers from being able to be assigned (even though the PCI address shows up in
xl pci-assignable-listafter it's added to the pciback kernel parameter), but I'm not sure where to go looking for it.
I realize this is old but I figured out what's going on here. Has to do with RMRR: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf
Check out your
/var/log/xen/hypervisor.logand you should see something like this:
[2020-08-22 10:09:03] (XEN) [ 297.542134] [VT-D] It's disallowed to assign 0000:08:00.0 with shared RMRR at 7ba77000 for Dom32753. [2020-08-22 10:09:03] (XEN) [ 297.542136] d[IO]: assign (0000:08:00.0) failed (-1)
The redhat paper goes into good detail about what's going on here, but the tl;dr is that your BIOS has access to memory segments of USB devices for PS/2 emulation and therefore DMA isn't allowed in those regions.
Additional reading: https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt
Lots of people have been trying to figure this one out, especially in the Qubes community. Here's some discussions from all over the place that I found:
In my case appending
other-config:pcivalue didn't change anything, though some tickets indicated that had to be passed another way. I wasn't sure where I was supposed to set
pci_strictreset, as all the examples were qubes-specific tooling.
Some of these discussions, as well as this part of xen's wiki give me the impression that maybe there's sub devices in my BDF address that need to be passed too, but if there are I couldn't find them with
lspci. I haven't found any way to see what devices share an RMRR space other than one of the guys in one of those google groups discussing manually reading the hex of a dump from some tool.
Ahhh, oh well. This isn't mission critical for me, so I'm likely going to give up here. Hopefully this information provides helpful context for anyone running into this in the future, and if someone does manage to get PCI passthrough of a USB controller working, please do share your hardware and configuration.
That's a really interesting feedback, thanks a lot @hkparker for sharing this with us (it might be even interesting to write a recap on that into our official doc, what do you think?)
Yeah, I'd be happy to write up a condensed version of this for the docs. Is that something I can PR somewhere?
Using the link on the bottom of the doc will create a PR