@tuxen said (https://xcp-ng.org/forum/topic/3652/no-free-virtual-function-found-vgpu-s7150/4?_=1731502751059)
After some digging, could be the case of a GPU firmware being incompatible with UEFI. Do you have any spare server for testing XCP-ng boot in legacy/BIOS with this GPU?
Perhaps it is the issue ?
Nov 13 11:30:21 xen03 kernel: [10188.720655] AMD IOMMUv2 driver by Joerg Roedel jroedel@suse.de Nov 13 11:30:21 xen03 kernel: [10188.720656] AMD IOMMUv2 functionality not available on this system
This is expected, Dom0 Kernel (Linux) is not supposed to access the IOMMU when it is already used by Xen. To check if AMD-Vi is working, you need to check xl dmesg instead.
I took a quick look at kern_gim_compiled.txt, and it look likes it timed-out somewhere
Oct 23 20:49:32 xen03 kernel: [ 80.657394] gim error:(wait_cmd_complete:2387) wait_cmd_complete -- time out after 0.003004460 sec
Oct 23 20:49:32 xen03 kernel: [ 80.657408] gim error:(wait_cmd_complete:2390) Cmd = 0x17, Status = 0x0, cmd_Complete=0
3ms looks like a short timeout for me, but aside that, it looks like a driver(gim) or hardware issue
PCIe AER needs proper PCIe, which in practice needs Q35 chipset in the guest (or some other guest type/PCI passthrough way).
Q35 support is currently work in progress
I think it is the same MSI-X/PBA issues that may be partially fixed with https://gitlab.com/xen-project/xen/-/commit/b2cd07a0447bfa25e96ae13e190225b61a3670cb
However, with this device, MSI-X vector table and PBA are in a same page (vector table in 46800 and PBA in 46068) though, which is threated a bit differently
If PBA lives on the same page, discard writes and log a message.
Technically, writes outside of PBA could be allowed, but at this moment
the precise location of PBA isn't saved, and also no known device abuses
the spec in this way (at least yet).
But Coral appears to abuse this according to DKMS driver by having more than just MSI-X and PBA on a single page
https://github.com/google/gasket-driver/blob/main/src/apex_driver.c#L103-L140
Hello @hvm,
Can you give the output of xl dmesg in XCP-ng and of dmesg in the guest that has the issues ?
I have the impression that something is going wrong with reserved regions related to the SATA controller.