nVidia Tesla P4 for vgpu and Plex encoding

olivierlambert

@msupport can you detail a bit what you did? thanks!

msupport

Install XCP-NG Version 8.2.1 (8.3 did not work)
Install all update yum update
reboot
Download NVIDIA vGPU drivers for XenServer 8.2 from NVIDIA site. Version NVIDIA-vGPU-CitrixHypervisor-8.2-550.54.10.x86_64 (Version 17.0)
Unzip and install rpm from Host-Drivers
reboot again
Download free CitrixHypervisor-8.2.0-install-cd.iso from Citrix site
Open CitrixHypervisor-8.2.0-install-cd.iso with 7-zip, then unzip vgpu binary file from Packages->vgpu....rpm->vgpu....cpio->.->usr->lib64->xen->bin
9.Upload vgpu to XCP-ng host to /usr/lib64/xen/bin and made it executable chmod +x /usr/lib64/xen/bin/vgpu
Deployed VM with vGPU and it started without any problems
Copy License File from Nvidia License Portal (*.tok) to C:\program files\Nvidia Corperation\vGPU Licensing\ClientConfigToken
Install Windows Nvidia Driver on Windows 10 VM (need connection to api.dis.licensing.nvidia.com Port TCP 443, use Nvidia Control Panel to set the hostname and port for Licensing the Nvidia Card)

Works fine for me

If you want to install the NVidia driver on XCP-NG 8.3. Manipulate /etc/xensource-inventory (line PRODUCT_VERSION from 8.3.0 to 8.2.0) during the installation. Then xe-install-supplental-pack NVIDIA-vGPU-CitrixHypervisor-8.2-550.54.16.x86_64.iso.
After installation, change PRODUCT_VERSION back to 8.3.0
The driver now also works in version XCP-NG 8.3
Do not forget to copy the vgpu file to /usr/lib64/xen/bin/vgpu. (change the chmod to 755)

Nvidia vGPU M10 | A16 on XCP-NG 8.3 Beta2 only work without XCP-NG updates. After the update, the error message "An emulator required to run this VM failed to start" appears. It must be due to one of the 76 updates that can be installed. I am trying to find out which update is causing this problem.

22.07.2024 [NEW]

Installation XCP-NG RC1 Nvidia 17.1 GPU

install XCP-NG 8.3 RC1
download XenServer Driver Nvidia 17.1 (NVIDIA-GRID-XenServer-8-550.54.16-550.54.15-551.78)
unzip driver and copy host driver (NVIDIA-vGPU-xenserver-8-550.54.16.x86_64.iso) I used winscp to copy the driver to the tmp directory.
download XenServer iso file (https://www.xenserver.com/downloads | XenServer8_2024-06-03.iso)
copy the file (vgpu-7.4.13-1.xs8.x86_64.rpm) in the packages directory ! Do not use CitrixHypervisor-8.2.0-install-cd file vgpu-7.4.8-1.x86_64
unpack file vgpu-7.4.13-1.xs8.x86_64
copy the file \usr\lib64\xen\bin\vgpu (size 129KB) to \usr\lib64\xen\bin\ on your XCP-NG host (chmod 755)
(putty) /tmp/ xe-install-supplemental-pack NVIDIA-vGPU-xenserver-8-550.54.16.x86_64.iso
reboot
install guest driver on the VM client (551.78_grid_win10_win11_server2022_dch_64bit_international.exe)
token file from Nvidia (C:\Program Files\Nvidia Corporation\vGPU Licensing\ClientConfigToken*.tok)
Nvidia drivers 17.2 and 17.3 do not work yet (Guest driver crashes) Test with Windows 11 23H2

My environment:
16x Hosts HPE DL380
6x Hosts HPE DL380 with vGPU Nvidia M10 and A16
5x HPE 3PAR Storage and 1x HPE MSA 2050 Storage
2x 96 port fibre channel switch

I have migrated from Vmware to XCP-NG with XOA.

olivierlambert

Okay so the only binary you need to get from Citrix is vgpu, right?

austinw

Out of curiosity what are you using this for? what are these VM's doing?

msupport

@olivierlambert
yes, download and extract. Don't forget the permission.

Download free CitrixHypervisor-8.2.0-install-cd.iso from Citrix site

Open CitrixHypervisor-8.2.0-install-cd.iso with 7-zip, then unzip vgpu binary file from Packages->vgpu....rpm->vgpu....cpio->.->usr->lib64->xen->bin

Upload vgpu to XCP-ng host to /usr/lib64/xen/bin and made it executable chmod +x /usr/lib64/xen/bin/vgpu

msupport

@austinw
I use these Windows 10 clients with UDS Enterprise VDI System. From October 2025, it will be forbidden to run Office on the terminal server. That's why we switched to virtual desktops.

https://learn.microsoft.com/de-de/deployoffice/endofsupport/windows-server-support

austinw

@splastunov Do the AMD GPU's not require a license?

mohammadm

@austinw said in nVidia Tesla P4 for vgpu and Plex encoding:

@splastunov Do the AMD GPU's not require a license?

Nope. These work easy out of the box. Installed the GPU on one of our servers yesterday.

splastunov

@austinw no licenses, but a lot of troubles.....

mohammadm

@splastunov said in nVidia Tesla P4 for vgpu and Plex encoding:

@austinw no licenses, but a lot of troubles.....

Curious, what troubles?

splastunov

@mohammadm
I'm talking now about vGPU not passthrough

old drivers
no way to monitor GPU load
Sometimes the GPU on Dom0 stops responding and the only thing that can be done to solve this problem is to reboot the entire server with all the virtual machines on it.
and etc.... do not remember all troubles I had with it

mohammadm

@splastunov said in nVidia Tesla P4 for vgpu and Plex encoding:

@mohammadm
I'm talking now about vGPU not passthrough

old drivers

no way to monitor GPU load

Sometimes the GPU on Dom0 stops responding and the only thing that can be done to solve this problem is to reboot the entire server with all the virtual machines on it.
and etc.... do not remember all troubles I had with it

I installed the Firepro S7150x2 yesterday without any issues. It's been about 24 hours, so far no issues. I do agree I am missing the nvidia-smi command to get a better overview.

Why is the support regarding vGPU so bad and mostly outdated

olivierlambert

I will have the opportunity to discuss more with AMD (on a regular basis, for some reasons), I'll try to see if I can connect to their GPU division

austinw

@splastunov Dom0 would need to have a vGPU in this scenario?

JamesG

@olivierlambert As mentioned in another thread...Intel Flex GPU's seem primed for this. nVidia is closed and license greedy. AMD seems a little lost and wandering. Intel has said, "No licensing...Just use it." but they require some development.

It should be relatively easy to incorporate the Intel Flex GPU's, but I'm not sure if the newer kernels are required. That might be where the wheels fall off for now.

mohammadm

@JamesG This would indeed be awesome! I would prefer going the Intel route. Any contacts there @olivierlambert ?

JamesG

From my perspective, there's literally money on the ground for any virtualization platform to pick up VDI with Intel. The GPU's are affordable and performant for VDI work. They currently work with Openshift and Proxmox is at work on it.

stormi

@splastunov why do you extract the vgpu RPM rather than just installing the RPM directly?

high-voltages

I am currently trying to get a NVIDIA Telsa M10 working with the latest updated Xcp-ng 8.2.1 Build Date 2024-07-17.

The Drivers "NVIDIA-vGPU-xenserver-8-550.54.10.x86_64.rpm" are installed and the system was rebooted. Now it is possible to select a vGPU via XOA for the VM.

Further the extracted vgpu file from the CitrixHypervisor-8.2.0-install-cd.iso was copied to /usr/lib64/xen/bin/vgpu and is executable.

The Start of the VM exited with:

"FAILED_TO_START_EMULATOR(OpaqueRef:0cc388f0-b606-469d-b68c-b4713c7f4abb, vgpu, Daemon exited unexpectedly)
"

Is there someone who has solved this Problem?

Platform:
HPE DL380 Gen10

msupport

@high-voltages

download XenServer iso file (https://www.xenserver.com/downloads | XenServer8_2024-06-03.iso)
copy the file (vgpu-7.4.13-1.xs8.x86_64.rpm) in the packages directory ! Do not use CitrixHypervisor-8.2.0-install-cd file vgpu-7.4.8-1.x86_64
unpack file vgpu-7.4.13-1.xs8.x86_64
copy the file \usr\lib64\xen\bin\vgpu (size 129KB) to \usr\lib64\xen\bin\ on your XCP-NG host (chmod 755)