AMD 'Barcelo' passthrough issues - any success stories?

DustyArmstrong

I am trying to pass through my AMD GPU to a Debian host. This sort of succeeds, but I am getting error messages about a "BIOS ROM". I have had a look online and found some resources to suggest how to provide one, but I am quite unsure on that and cannot locate a specific one for my GPU (AMD Ryzen 7 5825U with Radeon Graphics, I think Vega 8).

I am showing it has at least tried to load/associate the kernel module:

lspci -nnk -s 00:08.0
00:08.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Barcelo [1002:15e7] (rev c1)
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:1636]
	Kernel modules: amdgpu

But dmesg output states it cannot locate a ROM.

[    4.655776] amdgpu 0000:00:08.0: amdgpu: Unable to locate a BIOS ROM
[    4.655797] amdgpu 0000:00:08.0: amdgpu: Fatal error during GPU init
[    4.655812] amdgpu 0000:00:08.0: amdgpu: amdgpu: finishing device.
[    4.656681] amdgpu 0000:00:08.0: probe with driver amdgpu failed with error -22

Is there a trick to this, has anyone had success with this kind of AMD GPU? On my old hosts, enabling pass through was enough for it to just kind of work (Intel HD 530). The host machine outputs to a display normally when the card is in-use by the host. I am of the understanding the ROM is just part of the motherboard/GPU, there is some suggestion it can be dumped from the host-side, but I'm unsure on this.

Can anyone shed some light on how this should work, or whether it's just something in my setup that isn't working? I have IOMMU and SR-IOV enabled in the BIOS, but I don't know if it's working.

EDIT: It looks like I may just have a fake BIOS? The settings to enable all the relevant components (IOMMU, DMAr support etc) don't actually seem to do anything, they might just be for show - dmesg | grep -i iommu returns nothing, dmesg | grep -i -e dmar -e vfio -e pciback only shows pciback info, and cat /proc/cmdline contains nothing about IOMMU. Oddly, XO is still reporting that IOMMU is enabled:

The output of xe host-param-get uuid={uuid} param-name=chipset-info also returns iommu: true, but the output of xl info | grep "iommu" returns nothing.

TeddyAstie

@DustyArmstrong

EDIT: It looks like I may just have a fake BIOS? The settings to enable all the relevant components (IOMMU, DMAr support etc) don't actually seem to do anything, they might just be for show - dmesg | grep -i iommu returns nothing, dmesg | grep -i -e dmar -e vfio -e pciback only shows pciback info, and cat /proc/cmdline contains nothing about IOMMU. Oddly, XO is still reporting that IOMMU is enabled:

dmesg in the Dom0 will not report the information you're looking for.
To know if PCI Passthrough is supported (e.g IOMMU enabled), you should check xl info | grep virt_caps and look for hvm_directio. You can also look for IOMMU-related stuff in xl dmesg.
As you managed to passthrough the device (even if not working in the guest), I don't see a issue there.

[ 4.655776] amdgpu 0000:00:08.0: amdgpu: Unable to locate a BIOS ROM
[ 4.655797] amdgpu 0000:00:08.0: amdgpu: Fatal error during GPU init
[ 4.655812] amdgpu 0000:00:08.0: amdgpu: amdgpu: finishing device.
[ 4.656681] amdgpu 0000:00:08.0: probe with driver amdgpu failed with error -22

Is there a trick to this, has anyone had success with this kind of AMD GPU? On my old hosts, enabling pass through was enough for it to just kind of work (Intel HD 530). The host machine outputs to a display normally when the card is in-use by the host. I am of the understanding the ROM is just part of the motherboard/GPU, there is some suggestion it can be dumped from the host-side, but I'm unsure on this.

Looks like the GPU ROMBAR is missing in the guest, while it's ok for many devices, many others will fail to work without it (like this GPU).
To me, there's something missing on the PCI Passthrough logic, I just brought the topic internally to see what we can do.

DustyArmstrong

@TeddyAstie Thanks for that - I was going off of other old XCP hosts I have had running that did show IOMMU info there, so I figured it wasn't working on these particular machines. I also had the the Intel GPU working on those hosts, which further supported my theory. I do have a USB controller passed through to the same VM and that works perfectly, basically native functionality, but from the research I was doing it was suggested USB controllers are way less bothered about imperfect passthrough.

I was able to find a ROM for my particular GPU, but I don't know what - if anything - I can do with it in XCP. I've found lots of info for QEMU and Proxmox, but I'm not sure they directly translate.

If there's nothing I can do for the moment then I can live without the GPU, running decode on the CPU isn't really taxing it that much. Appreciate you looking into it.