Intel Flex GPU with SR-IOV for GPU accelarated VDIs

nick.lloyd

@olivierlambert Looks like it would arrive April 14th:

https://www.dell.com/fr-fr/shop/intel-flex-140-pcie-75-w-12go-passif-unique-wide-profil-bas-gpu-installation-par-le-client/apd/490-biio/cartes-graphiques

olivierlambert

I received one card, I quickly tested it, you can actually see 2x GPU (as expected), PCI passthrough each to a different VM. So at least, basic stuff works.

A next step (when we can staff someone) is to check how to manually enable SRIOV on the device and pass it to the VMs, and in the end, get that done in XAPI to make it integrated with XO.

sanjay

@olivierlambert Great to hear...

Looking for forward to have Arc GPU support. If possible can you please be specific on the GPU hardware you have and which is being tested. As you know, someone might have the one you have or a similar GPU that they can use. From what you posted, I assume that you might have the ARC Pro B60 Dual 48G by Maxsun - correct me if I am wrong.

Lately, I am trying to source a Arc Pro B60. I do not have any timeline when I would be able to obtain one.

At least for the time being and considering the alleged MSRP for battlematrix - Arc Pro cards, Intel seems to be the best value.

olivierlambert

No, I have an Intel Flex 170

JamesG

@olivierlambert If I remember right, you should be able to see 62 VF's on that card. There might be a tool needed to define how many VF's are present like on a NIC.

olivierlambert

Yes but in order to see the VF I need a working driver in the Dom0 first Probably a lot easier with a more recent kernel in XCP-ng 9.0

JamesG

@olivierlambert Ideally you need to be somewhere into Kernel 6. 6.12 is sticking out in my head, but I'm not positive when support got fully integrated.

olivierlambert

Upstream inclusion is ultra recent, but you can always add drivers yourself (still requiring a relatively recent kernel IIRC). Anyway, we have the hardware and it will be in the roadmap at least to explore

JamesG

@olivierlambert While VDI is maybe not as vital as it once was...I'm experimenting with multimedia work in XCP-ng. Having a VM with GPU off-loading of CODEC encoding would be nice. It's a pretty big CPU hit to make that go.

sluflyer06

@sanjay Kinda bringing this back, does anyone know if or when support for newer Intel cards like B60+ will be added in XCP-NG? I can only assume there's something that doesn't support these cards in the kernel possibly?

Me and another user bought B50's on release day and tried them and neither of us could get them to function, you could kinda see them in the VM passed through but they were non-functional, even on the latest ubtuntu developer kernels. It felt like the cards simply weren't supported on xcpng.

olivierlambert

PCI passthrough shouldn't be a problem in theory. Do you have more context/logs to bring?

JamesG

@olivierlambert

I agree that pass-through should just work.

I couldn't seem to get the GPU acceleration to work on a linux guest when I tried it on a Debian guest and I know if had the supported kernel running on the guest. I suspect this is due to the two different graphics devices on the guest VM (XCP-ng emulated and the physical passed-through GPU) and needing some sort of adjustments in X (Xlibre in my case) in order to tell it to use the GPU even though there's no display attached to it. I dropped that effort awhile back as I had bigger tasks at hand...But that's my suspicion. I need to tell X to offload processing to the GPU somehow, yet render the output to the emulated graphics adapter on the guest.

I've never really used GPU's for graphics acceleration on Linux. I don't game, and almost all of my Linux experience is headless server stuff. So I'm in uncharted territory for me.

sluflyer06

@olivierlambert I'm willing to go procure another to try to troubleshoot, 2 of us tried 3 different cards on different servers and spent a lot of time and never got anywhere with the cards. This is some messages I still have have from back when I was trying. More or less, XCPNG could see the cards, they could be passed to a VM either in XO or CLI, we tried a couple different linux VM's on 6.17 kernel and would get errors like below, I also tried a windows VM and it couldn't identify the card at all, tried latest intel drivers, etc.

[    3.109115] xe 0000:00:08.0: [drm] Found battlemage (device ID e212) discrete display version 14.01 stepping B0
[    3.110277] xe 0000:00:08.0: Direct firmware load for xe/bmg_guc_70.bin failed with error -2
[    3.110281] xe 0000:00:08.0: [drm] GuC firmware xe/bmg_guc_70.bin: fetch failed with error -2
[    3.110283] xe 0000:00:08.0: [drm] GuC firmware(s) can be downloaded from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
[    3.110284] xe 0000:00:08.0: [drm] *ERROR* GT0: GuC init failed with -ENOENT
[    3.110299] xe 0000:00:08.0: [drm] *ERROR* GT0: Failed to initialize uC (-ENOENT)
[    3.110526] xe 0000:00:08.0: probe with driver xe failed with error -2

JamesG

@sluflyer06 Double check all your virtualization settings in the BIOS of your system.

There are some firmware updates that need to happen on the Intel cards that are frankly just easier to do in Windows, and even further in a physical machine (not passed-through). I had no issues updating my card via a passed-through Windows VM, but others haven't been as successful.