@planedrop

I have onboard Radeon graphics, and NVIDIA dGPU

@branpurn

Same result with NVIDIA's Studio Driver, "The device is working properly" (though it's not, seemingly) then upon reboot, Code 43.

Still somewhat confused as to why I have console display in XOA when "VGA" in 'Advanced' is set to 'Off'...

Toggling it back and forth seems to have worked, thanks.

@branpurn

Food for thought, here's all parameters for the guest,

params=all.txt

@olivierlambert

At risk of getting off topic the original issue, it sounds like I am correct in understanding guest tools are not relevant to the PCI passthrough.

Fresh install Win 11, it may be worth capturing the initial error pre-display driver, as the way it is languaged might be different than a typical missing driver (unconfirmed)

-- Shortly hereafter, BSOD, VIDEO_TDR_FAILURE (I think Windows Update detected the GPU and tried to install driver)

EDIT: Appears BSOD loop after WU on fresh Win 11 install grabbed the driver; I think this is similar to complete reinstall of driver on previous iteration of this Win 11 install, post-passthrough of the audio sub-device (Driver would likely not have been auto-grabbed previous install of Win 11 due to the sub-device not being present?)

EDIT: Trying 'cloud reset' from Win 11 recovery partition options for the heck of it... EDIT: As one might expect, not as "clean" an install as a clean format and reinstall, even when selecting "remove everything" and "clean drive"-- assumedly driver, and thus BSOD, remain with that method.

Anyway, getting into the weeds with Windows here. I don't think the solution lies on the OS side.

@topher

Were it to matter, I do not have drivers available in 'Advanced'/do not have access to Citrix guest tools (and the Github for XCP-ng guest tools hasn't been updated since 2019?)-- I am operating under the assumption that with PCIe passthrough, I am better worrying strictly about the baremetal video drivers at this time.

@olivierlambert

I have (and have not lost) console display output, I have never yet had it able to provide display output via HDMI/detect a wired display. EDIT: (I am actually somewhat confused why I still have console display in XOA when "VGA" in 'Advanced' settings is set to 'Off')

EDIT: Having uninstalled the driver completely and re-installed, I now appear to consistently be getting BSOD, VIDEO_TDR_FAILURE: nvlddmkm.sys (NVIDIA generic); going to do a fresh install of Win 11 on the guest in meantime.

@olivierlambert

Thanks-- take two,

[08:36 xcp-ng ~]# xe vm-param-set other-config:pci=0/0000:01:00.0,0/0000:01:00.1 uuid=0360a31e-d8ef-0ff1-e7d1-24bcd276d7f5
[08:36 xcp-ng ~]# xe vm-param-get param-name=other-config param-key=pci uuid=0360a31e-d8ef-0ff1-e7d1-24bcd276d7f5
0/0000:01:00.0,0/0000:01:00.1
[08:36 xcp-ng ~]#

Initially, Code 43. Disabling and re-enabling, Code 43 is gone ("The device is working properly" with no display output), but upon reboot, Code 43 returns.

EDIT: (FYSA, kernel params were also updated accordingly and XCP-ng rebooted, [08:48 xcp-ng ~]# /opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(0000:01:00.0)(0000:01:00.1)")

"Auto power on" configured in 'Advanced' settings on the XOA guest from XOA itself does not seem to restart the XOA guest when XCP-ng host power cycles.

Is there a different place to configure this with GUI? Would like clarification on that before I configure param, xe vm-param-set uuid=<VM_UUID> other-config:auto_poweron=true and reboot.

Thank you!!

BLUF: Windows 11 guest is unable to use NVIDIA GPU passthrough. Error in Device Manager is the classic, "Code 43."

Environment:

• Hypervisor: XCP-ng 8.3
• Processor: AMD Ryzen 9 7950X 16-Core, 32-Thread
• Motherboard: ASUS Prime X670-P Socket AM5 (LGA 1718) ATX Motherboard
• GPU: ZOTAC Gaming NVIDIA GeForce RTX 4080 16GB
• RAM: CORSAIR Vengeance DDR5 64GB (2x32GB)
• Hypervisor SSD: SanDisk SSD PLUS 1TB Internal SSD - SATA III 6 Gb/s

Circling back as time allows, I did a passthrough of my GPU to a fresh Windows 11 guest, following the documentation at: https://docs.xcp-ng.org/compute/

[17:48 xcp-ng ~]# lspci | grep NVIDIA
01:00.0 VGA compatible controller: NVIDIA Corporation Device 2704 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 22bb (rev a1)
[17:48 xcp-ng ~]# xl pci-assignable-list
0000:01:00.0
[17:49 xcp-ng ~]# xe vm-param-get param-name=other-config param-key=pci uuid=0360a31e-d8ef-0ff1-e7d1-24bcd276d7f5
0/0000:01:00.0
[17:49 xcp-ng ~]#

After installing the drivers in Windows, prior to rebooting the guest, the GPU (while not actively able to render display to attached HDMI panel) appears to be functioning properly:

However, after rebooting the guest (with hope to render to attached HDMI panel), Code 43 is present:

Device Manager:

Event Viewer:

---

I am not using vGPU, as documentation suggests that NVIDIA GPUs are incompatible (and furthermore, I am not clear on potential performance drawbacks to vGPU).

I have disabled "VGA" in XOA per the Windows 11 guest's advanced settings.

In the past, VMware ESXi GPU passthrough did not work reliably without enterprise/Quadro cards. Proxmox GPU passthrough tended to "just work." Not sure what the expectation is with XCP-ng, but I am assuming it should work without a Quadro card.

Any thoughts on potential oversights/suggestions are greatly appreciated!