JamesG

@dkidd255 I'm a LOOONG time XCP/XenServer/XCP-ng user in my homelab...And it pains me to say this...

But this just works in Proxmox.

If you want to help us dig in and troubleshoot/explore this topic...Fine.

If you want to actually use your hardware and make progress on your ideas, go to Proxmox.

I seem to recall some recent firmware shenanigans from Intel. Intel maybe thought a little too much of their hardware, or they've decided to just scale things back a bit, but newer versions of firmware have reduced VF's available. I don't think the B70 is impacted by this, but it has massive memory in comparison to the smaller cards.

@TeddyAstie

Thought...If you want access to a server with a B50, I think I could setup VPN access and put a host online with IPMI access for you to do whatever you want with.

You will have an AMD Epyc 7302 with 64GB of ram some flash and HDD storage, and an Intel B50. PM me and we can work out the details of what you need to experiment with, and I can setup some remote access for you to get into the system.

I will gladly work with you guys any which way I can to move this forward.

@TeddyAstie I think there's alot of interest in the Intel platform because of the lack of ridiculous licensing for virtualization features. The problem is that you need a pretty current 6.x kernel to use them and XCP-ng/XenServer is just stuck on a really aging kernel right now.

As exposed in this thread, the PCIe pass-through isn't really working properly under XCP-ng.

Even if pass-through worked, that's kinda lame because the real purpose would be to chop up the GPU into SR-IOV instances so multiple workloads could have a GPU.

The Proxmox community is already rockin' and rollin' with this. I watched a video a couple of days ago where a B70 was chopped up into Qty.4, 8GB GPU's via SR-IOV and passed to clients.

No ridiculous mega-$$$$ licensing. It just worked.

@andriy.sultanov It's possible that this is just a fundamental issue with trying to use (maybe just Battlemage) Intel GPU's in XCP-ng.

The card itself appears to be a PCIe bridge in front of the actual GPU hardware on the card. If the driver on the guest requires "the whole thing" sent to it, and you can't pass-through the whole bridge and everything behind it, then there's no real way to use these GPU's in a guest on XCP-ng.

The lack of recent kernel or sufficient backports in the XCP-ng kernel, means the SRIOV features of Intel GPU's are lost as well.

All of this stuff works in Proxmox with seemingly little issues.

James

@TeddyAstie From what I've tried...It won't work at all in Linux.

It kind of works in Windows, and while I do see some load on the GPU when running something, there's as much if not more load on the CPU. It's not what I expected at least.

I've never really gamed on PC, and definitely never Linux. My only uses for GPU's have been in media composition which was always on dedicated systems and Nvidia GPUs. More recently I've been doing some AI tasks, again with passed-through NVidia GPUs.

Side tangent...
I ran into another workload last night where the app wanted OpenGL...But OpenGL doesn't work with the "normal" VGA adapter in XCP-ng, and I don't have a GPU on that particular pool to try a pass-through. That lead me to VirGL, but it doesn't look like that's gone anywhere.

Yep...

Stripped it all out....Rebooted...

Set the pcihideback for dom0...Rebooted.

Verified all devices were assignable.

Assigned to guest VM, booted...No go.

Initially the log said it couldn't assign 82:02.0
So I removed it.
Then it couldn't assign 82:01.0
So I removed it.
Then it couldn't assign 81:00.0
So I removed it.

Now the VM works...But we're effectively back to just passing through the GPU itself, and I guess the audio device.

No functional change.

Argh...

Okay...First up:

lspci:

81:00.0 PCI bridge: Intel Corporation Device e2ff (rev 01)
82:01.0 PCI bridge: Intel Corporation Device e2f0
82:02.0 PCI bridge: Intel Corporation Device e2f1
83:00.0 VGA compatible controller: Intel Corporation Device e212
84:00.0 Audio device: Intel Corporation Device e2f7

Execute the hide:

/opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(0000:81:00.0)(0000:82:01.0)(0000:82:02.0)(0000:83:00.0)(0000:84:00.0)"

Reboot and verify the hide:

xl pci-assignable-list
0000:84:00.0
0000:82:01.0
0000:83:00.0
0000:81:00.0
0000:82:02.0

Looks good....make the assignment:

xe vm-param-set other-config:pci=0/0000:81:00.0,0/0000:82:01.0,0/0000:82:02.0,0/0000:83:00.0,0/0000:84:00.0 uuid=9d76f849-16bf-0414-9072-6904176bbb01

Verify:

xe vm-param-list uuid=9d76f849-16bf-0414-9072-6904176bbb01

other-config (MRW): pci: 0/0000:81:00.0,0/0000:82:01.0,0/0000:82:02.0,0/0000:83:00.0,0/0000:84:00.0; import_task: OpaqueRef:9e15d1aa-5e79-79d9-af81-4dc299725217; xo:9d76f849: {"creation":{"date":"2025-12-16T19:19:14.724Z","template":"1f8728f9-6354-4a68-adc3-222c34f2bb94","user":"ae9cf9c9-bbcc-4dc9-b1bb-d9fb1b657c60"}}; base_template_name: Windows 11; mac_seed: 5891089b-e9ec-3050-30e5-564db3e6e170; install-methods: cdrom

Starting up the VM results in this:

  "params": [
    "xenopsd internal error: Cannot_add(0000:82:02.0, Device_common.QMP_Error(1, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"Failed to initialize 11/15, type = 0x1, rc: -1\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-000006-1\\\"}\"))"
  ],

I'm gonna strip it all back, reboot and start over.

Nevermind...Typo...Rebooting now to see...

@andriy.sultanov First problem...

/opt/xensource/libexec/xen-cmdline --get-dom0 xen-pciback.hide
xen-pciback.hide=(0000:81:00.0)(0000:82.01.0)(0000:82:02.0)(0000:83:00.0)(0000:84:00.0)

That looks good...

xl pci-assignable-list
libxl: error: libxl_pci.c:444:libxl_device_pci_assignable_list: Looks like pciback driver not loaded

Not so much...

I did go ahead and assign those PCI devices to the guest, but of course it won't start, complaining that the pciback driver isn't loaded.

@andriy.sultanov Absolutely. I wasn't sure if the old way was incompatible with the new way or not.

So the old 8.2 method will still work without conflict in 8.3??

If so, I'll have an answer for you shortly.