Intel Flex GPU with SR-IOV for GPU accelarated VDIs

john.c

@olivierlambert Another option is possibly applying for Intel Developer Zone Premier. At this level they provide hardware testing support, I'm not sure if this means they can ship their hardware to companies to test against.

But if that's the case then you could then get the Intel Flex GPU and their other products ahead of time. As well as a closer connection to Intel for any issues, which come up involving XCP-ng and Xen Orchestra.

https://www.intel.com/content/www/us/en/developer/programs/overview.html#gs.ko8q8e

olivierlambert

Entering a big vendor program is probably 100 or 1000 times harder to grabbing a GPU ourselves. Now, the challenge is finding one in stock in EU.

john.c

@olivierlambert said in Intel Flex GPU with SR-IOV for GPU accelarated VDIs:

Entering a big vendor program is probably 100 or 1000 times harder to grabbing a GPU ourselves. Now, the challenge is finding one in stock in EU.

Maybe but the value out of it being able to access information, as well as easy access to people at Intel would pay for itself. As well as access to up coming products, to use for testing. Also would make accessing the card easier as, currently you need to buy a solution from a builder that uses the card.

@jrouerdc Have you read the https://xcp-ng.org/blog/2025/03/14/the-future-of-xcp-ng-lts/ blog post yet? Evaluating, planning and potentially using XCP-ng version 8.3 would be a very good idea. This because the LTS release of 8.3 will have the 2TB VDI disk limit removed, also an extensive support lifecycle compared to 8.2.

nick.lloyd

@olivierlambert Looks like it would arrive April 14th:

https://www.dell.com/fr-fr/shop/intel-flex-140-pcie-75-w-12go-passif-unique-wide-profil-bas-gpu-installation-par-le-client/apd/490-biio/cartes-graphiques

olivierlambert

I received one card, I quickly tested it, you can actually see 2x GPU (as expected), PCI passthrough each to a different VM. So at least, basic stuff works.

A next step (when we can staff someone) is to check how to manually enable SRIOV on the device and pass it to the VMs, and in the end, get that done in XAPI to make it integrated with XO.

sanjay

@olivierlambert Great to hear...

Looking for forward to have Arc GPU support. If possible can you please be specific on the GPU hardware you have and which is being tested. As you know, someone might have the one you have or a similar GPU that they can use. From what you posted, I assume that you might have the ARC Pro B60 Dual 48G by Maxsun - correct me if I am wrong.

Lately, I am trying to source a Arc Pro B60. I do not have any timeline when I would be able to obtain one.

At least for the time being and considering the alleged MSRP for battlematrix - Arc Pro cards, Intel seems to be the best value.

olivierlambert

No, I have an Intel Flex 170

JamesG

@olivierlambert If I remember right, you should be able to see 62 VF's on that card. There might be a tool needed to define how many VF's are present like on a NIC.

olivierlambert

Yes but in order to see the VF I need a working driver in the Dom0 first Probably a lot easier with a more recent kernel in XCP-ng 9.0

JamesG

@olivierlambert Ideally you need to be somewhere into Kernel 6. 6.12 is sticking out in my head, but I'm not positive when support got fully integrated.

olivierlambert

Upstream inclusion is ultra recent, but you can always add drivers yourself (still requiring a relatively recent kernel IIRC). Anyway, we have the hardware and it will be in the roadmap at least to explore

JamesG

@olivierlambert While VDI is maybe not as vital as it once was...I'm experimenting with multimedia work in XCP-ng. Having a VM with GPU off-loading of CODEC encoding would be nice. It's a pretty big CPU hit to make that go.

sluflyer06

@sanjay Kinda bringing this back, does anyone know if or when support for newer Intel cards like B60+ will be added in XCP-NG? I can only assume there's something that doesn't support these cards in the kernel possibly?

Me and another user bought B50's on release day and tried them and neither of us could get them to function, you could kinda see them in the VM passed through but they were non-functional, even on the latest ubtuntu developer kernels. It felt like the cards simply weren't supported on xcpng.

olivierlambert

PCI passthrough shouldn't be a problem in theory. Do you have more context/logs to bring?

JamesG

@olivierlambert

I agree that pass-through should just work.

I couldn't seem to get the GPU acceleration to work on a linux guest when I tried it on a Debian guest and I know if had the supported kernel running on the guest. I suspect this is due to the two different graphics devices on the guest VM (XCP-ng emulated and the physical passed-through GPU) and needing some sort of adjustments in X (Xlibre in my case) in order to tell it to use the GPU even though there's no display attached to it. I dropped that effort awhile back as I had bigger tasks at hand...But that's my suspicion. I need to tell X to offload processing to the GPU somehow, yet render the output to the emulated graphics adapter on the guest.

I've never really used GPU's for graphics acceleration on Linux. I don't game, and almost all of my Linux experience is headless server stuff. So I'm in uncharted territory for me.

sluflyer06

@olivierlambert I'm willing to go procure another to try to troubleshoot, 2 of us tried 3 different cards on different servers and spent a lot of time and never got anywhere with the cards. This is some messages I still have have from back when I was trying. More or less, XCPNG could see the cards, they could be passed to a VM either in XO or CLI, we tried a couple different linux VM's on 6.17 kernel and would get errors like below, I also tried a windows VM and it couldn't identify the card at all, tried latest intel drivers, etc.

[    3.109115] xe 0000:00:08.0: [drm] Found battlemage (device ID e212) discrete display version 14.01 stepping B0
[    3.110277] xe 0000:00:08.0: Direct firmware load for xe/bmg_guc_70.bin failed with error -2
[    3.110281] xe 0000:00:08.0: [drm] GuC firmware xe/bmg_guc_70.bin: fetch failed with error -2
[    3.110283] xe 0000:00:08.0: [drm] GuC firmware(s) can be downloaded from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
[    3.110284] xe 0000:00:08.0: [drm] *ERROR* GT0: GuC init failed with -ENOENT
[    3.110299] xe 0000:00:08.0: [drm] *ERROR* GT0: Failed to initialize uC (-ENOENT)
[    3.110526] xe 0000:00:08.0: probe with driver xe failed with error -2

JamesG

@sluflyer06 Double check all your virtualization settings in the BIOS of your system.

There are some firmware updates that need to happen on the Intel cards that are frankly just easier to do in Windows, and even further in a physical machine (not passed-through). I had no issues updating my card via a passed-through Windows VM, but others haven't been as successful.

sluflyer06

@JamesG I'll have to pickup another one to test, I did check all my BIOS settings, rebar was enabled, and everythign else was set right, me and another guy both tried for weeks and nowhere, both of us have Threadripper 7xxx series on Supermicro boards for xcpng hosts. I don't think I ever updated the firmware on the card, I picked up the card right when it launched.

TeddyAstie

@sluflyer06 said:

@olivierlambert I'm willing to go procure another to try to troubleshoot, 2 of us tried 3 different cards on different servers and spent a lot of time and never got anywhere with the cards. This is some messages I still have have from back when I was trying. More or less, XCPNG could see the cards, they could be passed to a VM either in XO or CLI, we tried a couple different linux VM's on 6.17 kernel and would get errors like below, I also tried a windows VM and it couldn't identify the card at all, tried latest intel drivers, etc.
[    3.109115] xe 0000:00:08.0: [drm] Found battlemage (device ID e212) discrete display version 14.01 stepping B0
[    3.110277] xe 0000:00:08.0: Direct firmware load for xe/bmg_guc_70.bin failed with error -2
[    3.110281] xe 0000:00:08.0: [drm] GuC firmware xe/bmg_guc_70.bin: fetch failed with error -2
[    3.110283] xe 0000:00:08.0: [drm] GuC firmware(s) can be downloaded from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
[    3.110284] xe 0000:00:08.0: [drm] *ERROR* GT0: GuC init failed with -ENOENT
[    3.110299] xe 0000:00:08.0: [drm] *ERROR* GT0: Failed to initialize uC (-ENOENT)
[    3.110526] xe 0000:00:08.0: probe with driver xe failed with error -2

The driver is telling you that you are missing the Intel Xe firmwares (that are probably in linux-firmware package).

sluflyer06

@TeddyAstie I hadn't seen anything special was needed in a VM other that a kernel new enough for support which 6.17 has, is there more to the story with adding things to Ubuntu?