XCP-ng GPU Passthrough - No GPU found

olivierlambert

Yeah, we had specific ~~threa~~ instructions to remove the Nvidia/Citrix vGPU thing for XCP-ng because it's not Open Source.

On Intel side, recent Xeon supports it (but you know Intel graphic perfs lol…) and also AMD FirePro S7150 (16vGPU).

DustinB

So a rock and a hard place. . . don't get the performance for the money or spend a ton of money. . .

olivierlambert

vGPU is still "reserved" for Virtual Desktop Infrastructure, which is a VERY lucrative and closed source world.

tjkreidl

@olivierlambert Indeed. NVIDIA in particular is bent on your having to license any sort of vGPU technology and it would not surprise me to see even passthrough leveraged at some point in the future. There are already restrictions on being able to make use of Quadro features, for example, of you don't have licenses.

Also, note that you need one video board reserved to run the native OS (XCP-ng) dom0 instance itself, so the server has to effectovely have at least two graphics boards such that one can be used as a GPU.

olivierlambert

Yeah, with GRID you need to pay also for the "driver" if I'm correct.

AMD approach is far better IMHO (using "standard" SRIOV) but that's because they are probably lagging in the market. Nvidia and Intel have somehow very similar business methods

olivierlambert

@tjkreidl hmm it's not possible to "remove" the GPU for the dom0? It won't boot without it?

tjkreidl

@olivierlambert No, it needs it (some sort of video board is necessary or it won't run at all). It has to at least be able to render the xsconsole interface.

olivierlambert

Okay good to know I had the hope that a serial console would be enough. I'll take a look if I can have any luck.

misterpc23

So when will PCI passthrough be available through the GUI for lazy people such as myself? I don't mind dedicating a whole card per VM (Prefer it actually) but am highly hesitant to use the console. Thank you for all of the hard work, XCP-ng is awesome!

borzel

@misterpc23 sometimes even lazy people have to go to the cli

misterpc23

@borzel It wasn't scary at all! Only downside is my gpu hasn't showed up so I'm thinking it is not supported (R370). Where could I find a list of GPU's that will work with PCI passthrough with XCP-NG? Thank you for any help

imtrobin

KVM is quite easy to spoof the VM so the nvidia drivers work. Is it possible to do easily for xcp?

https://lists.xenproject.org/archives/html/xen-devel/2016-07/msg01713.html

olivierlambert

You can pass the entire GPU if you like, it's no big deal. vGPU is another story.

imtrobin

I'm talking about GPU Passthrough, not vGPU. The GPU is passthrough fine but the nvidia driver results in code 43, meaning the driver detected it is running in a VM.

KVM bypass this by spoofing the VM, and it works, so can XCP do similar?

olivierlambert

IDK what it means to spoofing the VM. If you can tell what's needed exactly, maybe we could do something

imtrobin

In KVM, I need to do these

turn on cpu hidden state

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#"Error_43:_Driver_failed_to_load"_on_Nvidia_GPUs_passed_to_Windows_VMs

Assign the gpu to vfio directly (I guess we may not need to do this here)
Blacklist the nvidia drivers like this

https://techblog.jeppson.org/2018/03/windows-vm-gtx-1070-gpu-passthrough-proxmox-5/

https://forum.proxmox.com/threads/gpu-passthrough-tutorial-reference.34303/

imtrobin

Hi, so some people have been able to do it

https://gridforums.nvidia.com/default/topic/9108/#

https://gridforums.nvidia.com/default/topic/8934/#

Pejcik

Hello,

My idea was to passthrough GPU to specific VM. According to many informations from the Internet, I'm doing this like that:

/opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(03:00.0)(03:00.1)"
but still even after reboot I see it on lspci
xe vm-param-set other-config:pci=0/0000:03:00.0, 0/0000:03:00.1 uuid=c41704d0-a1f2-6bc1-fe2e-adc26697e636
for old Nvidia 8600 and GT430 it doesn't work - Windows VM see that device, drivers are installed but all the time I have error 43 and this device is stopped
for Radeon 7870 it work ok

Audio and USB is working with that metod also.

Is this connected with Vbios of this GPU's ?

linhnb

Finally, we have GPU passthrough in the GUI,
But it is not working...
My setup:
CPU 2700x
MOBO: x470 Taichi
RAM: 32GB
GPU: Rx580 sapphire Nitro 4GB

Please, I did use CLI and GUI, no work.
Cannot see it in Windows.
![alt text](image url)

anthonyh

Ironically, I am trying to do this as well. I put a second graphics card in a host (NVIDIA NVS 510) and made sure the appropriate virtalization settings were enabled in the BIOS. Running up to date XCP-ng 8.0 (yum update as of yesterday afternoon). I can assign the GPU to the Windows Server 2016 guest in XCP-ng Center, and the NVIDIA drivers seem to install fine, but Device Manager reports the following:

What's also interesting is even though it warns that you must be able to access the VM remotely as there will be no local console, there is still a local console. I don't know if this means anything.

Any ideas?