Yeah, I'd be happy to write up a condensed version of this for the docs. Is that something I can PR somewhere?
Posts
-
RE: XCP-ng 8.1 PCIe onboard usb controller passthrough does not work
-
RE: XCP-ng 8.1 PCIe onboard usb controller passthrough does not work
I realize this is old but I figured out what's going on here. Has to do with RMRR: https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf
Check out your
/var/log/xen/hypervisor.log
and you should see something like this:[2020-08-22 10:09:03] (XEN) [ 297.542134] [VT-D] It's disallowed to assign 0000:08:00.0 with shared RMRR at 7ba77000 for Dom32753. [2020-08-22 10:09:03] (XEN) [ 297.542136] d[IO]: assign (0000:08:00.0) failed (-1)
The redhat paper goes into good detail about what's going on here, but the tl;dr is that your BIOS has access to memory segments of USB devices for PS/2 emulation and therefore DMA isn't allowed in those regions.
Additional reading: https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt
Lots of people have been trying to figure this one out, especially in the Qubes community. Here's some discussions from all over the place that I found:
https://discussions.citrix.com/topic/378774-xenserver-7-usb-passthrough-on-xen-46/
http://xen.1045712.n5.nabble.com/shared-RMRR-issue-td5736717.html
https://groups.google.com/g/qubes-users/c/gS1XTal8XYs?pli=1
https://bugs.xenserver.org/browse/XSO-720
https://discussions.citrix.com/topic/381091-how-to-pass-rdm_policyrelaxed-through-xe-vm-param-set-in-xenserver-70/
https://groups.google.com/g/qubes-users/c/5x6iU28GadQ
https://lists.xen.org/archives/html/xen-users/2017-05/msg00053.html
http://xen.1045712.n5.nabble.com/quot-It-s-disallowed-to-assign-lt-device-gt-with-shared-RMRR-at-lt-address-gt-for-Dom-lt-N-gt-quot-a6-td5733137.html
https://github.com/QubesOS/qubes-issues/issues/1544In my case appending
,rdm_policy=relaxed
to myother-config:pci
value didn't change anything, though some tickets indicated that had to be passed another way. I wasn't sure where I was supposed to setpci_strictreset
, as all the examples were qubes-specific tooling.Some of these discussions, as well as this part of xen's wiki give me the impression that maybe there's sub devices in my BDF address that need to be passed too, but if there are I couldn't find them with
lspci
. I haven't found any way to see what devices share an RMRR space other than one of the guys in one of those google groups discussing manually reading the hex of a dump from some tool.Ahhh, oh well. This isn't mission critical for me, so I'm likely going to give up here. Hopefully this information provides helpful context for anyone running into this in the future, and if someone does manage to get PCI passthrough of a USB controller working, please do share your hardware and configuration.
-
RE: XCP-ng 8.1 PCIe onboard usb controller passthrough does not work
I'm experiencing the exact same issue on 8.1 stable. I'm able to passthrough and use any PCIe devices except USB controllers. There must be something preventing USB controllers from being able to be assigned (even though the PCI address shows up in
xl pci-assignable-list
after it's added to the pciback kernel parameter), but I'm not sure where to go looking for it.