PCIe Pass-through lanes and lane performance
-
@teddyastie I’ve added feedback in regards to the above on Vates Feedback portal, see link below.
-
@pandusen Your Windows test results are the perfect real-world example of this exact issue.
When you noted that the Intel driver panel was explicitly complaining about "PCI compatibility and reBar" and that performance was heavily borked, you hit the exact technical bottleneck. Without ReBAR support at the hypervisor layer, the guest OS is forced to interact with that massive 16 GB VRAM space through a rigid 256 MB window, choking it out with massive memory paging overhead.
Even @teddyastie's Linux testing reflects this—noting that host-device transfer bandwidth is "a bit slower than theoretical."
@olivierlambert From a strategic standpoint, as home labs and enterprise environments aggressively pivot toward localized AI, LLM inferencing, and heavy GPU compute, raw PCIe passthrough is shifting from an edge case to a core infrastructure requirement. Implementing a virtualized form of ReBAR support within Xen/XCP-ng would natively lift this 256 MB restriction. It ensures modern GPU silicon can run at 100% efficiency, keeping XCP-ng highly competitive against monolithic hypervisors that handle large BAR mapping.
Especially with VMware VCF 9.0 and up-to-date Hyper-V on Windows Server 2025 advancing their platforms, Vates needs to have virtualised ReBAR support to maintain its edge against this competition.
-
@john.c I would add that in addition to AI and VDI, media functions would be greatly assisted with GPU accelerated encode/decode. I need to do more with less. Having a GPU I could carve up and assign to VM workloads would be amazing.
-
@john.c I would add that in addition to AI and VDI, media functions would be greatly assisted with GPU accelerated encode/decode. I need to do more with less. Having a GPU I could carve up and assign to VM workloads would be amazing.
@JamesG Absolutely spot on. Media encoding/decoding and VDI density are huge components of this. Being able to slice up a GPU efficiently means VMs need to interact with that VRAM cleanly without hitting performance walls.
Whether it's splitting the card via vGPU profiles or passing it through fully for raw compute, lifting these platform memory limitations is what unlocks that "do more with less" capability for media stacks. It all ties back to how robustly the hypervisor fabric can handle modern GPU memory mapping.
If my feedback on Vates Feedback will help your use case please vote for it.
-
@Team-XAPI-Network @bleader Looping in the toolstack side as well. Even if the underlying Xen hypervisor layers align to handle large BAR configurations, XAPI will need the native capabilities to orchestrate and pass these extended memory mapping boundaries down to the VM configuration templates. Without XAPI awareness, we won't be able to easily toggle or manage virtual ReBAR allocations via the CLI or Xen Orchestra.
-
I get all this.. I was merely trying to help Teddy run into issues with the Intel arc card.
In regards to reBar, It is partly solved if you run a Linux 7 kernel. It is mostly a challenge from the guest os, rather than the hypervisor. As far as i know, xcp-ng reBar is passed through for complete PCI passthrough (not yet for Dom-0 and vfio), but I didn't concern myself too much with that.. The bigger issue was with the device grouping, the missing endpoints like the audio device and the drivers crashing when calling any of the tool stacks from intel. As well as the PCI link only negotiating a 1x1. I never got to a point of playing with transcoding for media nor AI Inferencing engines, as the tools and drivers never got stable.
-
I get all this.. I was merely trying to help Teddy run into issues with the Intel arc card.
In regards to reBar, It is partly solved if you run a Linux 7 kernel. It is mostly a challenge from the guest os, rather than the hypervisor. As far as i know, xcp-ng reBar is passed through for complete PCI passthrough (not yet for Dom-0 and vfio), but I didn't concern myself too much with that.. The bigger issue was with the device grouping, the missing endpoints like the audio device and the drivers crashing when calling any of the tool stacks from intel. As well as the PCI link only negotiating a 1x1.
Those tool-stacks from Intel crashing are evidence of the lack of virtualised ReBAR support, as they’re getting throttled to the point of choking on the 256MB per block limit.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login