@gleveque Hello

Can you retry with an up-to-date XCP-ng 8.3 host?
PCI expansion ROM are now enabled when doing pass-through.
The guest VM can now access the ROM bar.

Regards,

Yann

Hello @codemercenary
In theory Xen is NUMA aware (See https://wiki.xenproject.org/wiki/Xen_on_NUMA_Machines) so you would leave memory interleaving option of the BIOS off so that Xen does what it thinks is necessary.

But these kinds of settings are very very workload sensitive and in fact it's usually a good idea to just test what works best for you.

Regards,

Yann

@dkg Please also note that you should not use @jul1an ISO installer on production environment. Only on test env where you can afford to break things and lose data.

FYI this is not being forgotten, we still have a card about this assigned to someone, it will just take a bit more time than I thought before we work on this.
I'll send updates here when we have new elements.
Regards,
Yann

@dkg Oops sorry I thought you were someone else having the same issues.

Hello @codemercenary
In theory Xen is NUMA aware (See https://wiki.xenproject.org/wiki/Xen_on_NUMA_Machines) so you would leave memory interleaving option of the BIOS off so that Xen does what it thinks is necessary.

But these kinds of settings are very very workload sensitive and in fact it's usually a good idea to just test what works best for you.

Regards,

Yann

@dkg Please also note that you should not use @jul1an ISO installer on production environment. Only on test env where you can afford to break things and lose data.

@timemaster5 To give you more feedback, I am currently testing with an AMD Ryzen Embedded V2748 with Radeon Graphics host with a Renoir [Radeon Vega Series / Radeon Vega Mobile Series] iGPU.
I'm also talking to XAPI team to coordinate the effort and try to see what we can/want to do about this.
My first effort is to confirm what works/does not work and give all the information they need to the XAPI team so that they can make the decision on what to do.
I'll update you with my findings.

Can you retry with an up-to-date xcp-ng 8.3 please?

FYI on recent XCP-ng 8.3 versions the pci-passthrough will enable the ROM expansion bar. The guest VM will have access to it, so no need to pass it via qemu anymore.
See my comment on GitHub: https://github.com/xcp-ng/xcp/issues/786#issuecomment-4281846490

Regards,

Yann

timemaster5 created this issue in xcp-ng/xcp

closed PCI ROM BAR not exposed to guest when using xen-pci-passthrough #786

@gleveque Hello

Can you retry with an up-to-date XCP-ng 8.3 host?
PCI expansion ROM are now enabled when doing pass-through.
The guest VM can now access the ROM bar.

Regards,

Yann

FYI this is not being forgotten, we still have a card about this assigned to someone, it will just take a bit more time than I thought before we work on this.
I'll send updates here when we have new elements.
Regards,
Yann

Hi,

Someone from the Hypervisor & Kernel team will have a look shortly, we'll get back to you with our findings.

Thanks a lot for the very detailed report!

Yann

@abudef Some quotes from the documentation to clarify the situation: https://docs.xcp-ng.org/compute/#-nested-virtualization

@samuelolavo Thanks for your answer

It's very weird because by seeing the command outputs that you pasted, it looks like everything is behaving as it should be.
Even the PCI ID (segment:bus:device:function) seems to stay correct (0000:03:00.0)

I'll ask others internally.