AMD Radeon GPU Passthrough
-
Hello Everyone.
I saw this video on XCP-ng - AMD Radeon GPU Passthrough I thought it was excellent.
https://www.youtube.com/watch?v=zN67vnNkuYw
However when trying to do the same in my home lab with similar equipment I get an error message.
Internal error: xenopsd internal error: Cannot_add(0000:0b:00.0, Xenctrlext.Unix_error(30, “1:Operation not permitted”))
I am running xcp-ng 8.2, and I am using an old HP Prolaint G6 ML350 running 2 Xeon x5670 and 144gb of ram. I have added a AMD Sapphire Radeon RX 550 card and I am attempting GPU pass through as in the video link.
VT-d is enabled and USB pass through works, and I have an HDMI blank/dongle connected to the output display of the sapphire card rx550.
I noticed most my cards are on the same IRQ 11, is this an issue?
As a test, The Embedded mother board video card ( ATI ES1000 ) works to pass through when set not to be use by the server as display.
My onboard card is my primary display and AMD Radeon card is my secondary.
All attempts have the same error message.
Internal error: xenopsd internal error: Cannot_add(0000:0b:00.0, Xenctrlext.Unix_error(30, “1:Operation not permitted”))
I think XCP-NG sees this Radeon card as two different cards.
A video card and a sound card.
When i run lspci from the console line I can see 2 listings:
0b:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev ff)
0b:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X]
When I run a xl pci-assignable-list I get
0000:0b:00.0
0000:0b:00.1
as assignable. When I try to assign both of them they assign ok with no errors:
xe vm-param-set other-config:pci=0/0000:0b:00.0,0/0000:0b:00.1 uuid=a0552ff6-87f1-2044-ba59-ed920db6377e
Its only after I try boot up the VM I get the new error message:
Internal error: xenopsd internal error: Cannot_add(0000:0b:00.1, Xenctrlext.Unix_error(30, “1:Operation not permitted”))
I still think this is a issue that’s related to the BIOS , like an IRQ issue.
Any ideas?
-
I have been trying to pass through a graphics card as well.
Previously, on XCP-ng 8.1, I had no issues doing this, except for the reset issue that is caused by the graphics card. On XCP-ng 8.2, passing through the graphics card has become really difficult. The VM that is assigned the pci lanes, in my situation a Windows vm, becomes really unstable. The vm sees the graphics card, but if I attempt to install the Radeon drivers, the vm crashes and I'm no longer able to start it.
@Johnkrips These issues might be due to the pci issues related to the 8.2
https://xcp-ng.org/forum/topic/4033/persistent-usb-passthrough?_=1610136013449
https://github.com/xcp-ng/xcp/issues/470closed PCI passthrough issues #470
-
Hi, I can pass through a Radeon Pro WX 4100 Graphics in 8.2 to a Windows VM with no issue. It is basically the same as the RX550 just aimed at a different application.
I do not need to hide it. It appears as a GPU available for passthough in the VM's Advanced options in XO. Its also shows up as a GPU available for passthrough in XCP-ng Centre.
If you have hidden it (it shouldn't be necessary for supported GPUs - I did initially and had issues), then try unhiding it and see if you have any better luck!
I'm not so sure if its is the same for 'consumer / gamer' GPUs.
-
@jmccoy555
It's definite not the same for consumer grade cards unfortunately
At least not any recent card. I have an old ATI HD5xxx card that XCP-ng recognizes and am able to assign to a VM via the VM's advance settings in XO.Do you need to install the drivers for that card in the VM, or does it just work?
The problem for me arises when I try to install the drivers, but before I try to install them the VM becomes very unstable, which tells me there's something going wrong when I pass the pci lanes
-
I got the same error message before I hid the card.
Internal error: xenopsd internal error: Cannot_add(0000:0b:00.0, Xenctrlext.Unix_error(30, “1:Operation not permitted”))
However how do you unhide?
-
@l1c I did passthough my RX580 a while ago (I was trying to virtualise my desktop but had USB issues and parked the idea), either in 8.0 or 8.1 I can't remember and it did work.
Yes I did install the driver, but only the drive (via Device Manager if in Windows), and not to whole AMD control panel. I think the file is around 80MB rather than 300+MB.
-
@johnkrips I would edit the grub file in /boot and remove the line that gets added or you can just run the command that you used to hide it but without the device address. So if you want to hide no devices it will just end in =
-
/opt/xensource/libexec/xen-cmdline --delete-dom0 xen-pciback.hide -
@l1c Thank you.
I did it but still same issue.
Internal error: xenopsd internal error: Cannot_add(0000:0b:00.0, Xenctrlext.Unix_error(30, “1:Operation not permitted”))
Is there anyway to get a more detailed log of issue?
-
Have a read of this thread (found by googling the generic part of your error message).
-
I've never gotten errors like that, it just doesn't work when it is pass through.
Another thing I found a little odd was that the graphics card shows up as a usb device in Windows -
@l1c I got it to work. But not in that HP G6 server.
I have an identical ML350 G6 and its working fine with same version of XCP-NG.
Only difference is there are less pci cards in the machine. This goes back to my original idea, because other cards share the same IRQ the pass-through becomes for difficult.
Any configuration suggestions? Is there a way to edit the IOMMU groups?
