@maximsachs Thanks for getting back to me on this.
I think (and hope) that in the mean time we will have released an official new ISO that fixes the issue you are having.
So, you will be better off testing the new official ISO rather than this unofficial one 
I'll try to ping here when the new official ISO is out.
Thanks!
Regards,
Yann
@gleveque Hello
Can you retry with an up-to-date XCP-ng 8.3 host?
PCI expansion ROM are now enabled when doing pass-through.
The guest VM can now access the ROM bar.
Regards,
Yann
@yllar @mik @Pilow @marcvds Hello, we think we now understand better this issue.
We think it's a Xen issue with regard to PCI devices on segment 0001 (which is your case) and not a dom0 driver issue.
Could you give a try to this ISO installer please?
Please note that this is a custom test ISO and it is therefore not officially supported, do not use it for production, it is just to confirm that it fixes your issue, we will release a supported ISO in the future (target date for now is June).
unsupported test ISO for you to test: https://nextcloud.vates.tech/index.php/s/iHf8sqq39EqE683
Regards,
Yann
Hello @codemercenary
In theory Xen is NUMA aware (See https://wiki.xenproject.org/wiki/Xen_on_NUMA_Machines) so you would leave memory interleaving option of the BIOS off so that Xen does what it thinks is necessary.
But these kinds of settings are very very workload sensitive and in fact it's usually a good idea to just test what works best for you.
Regards,
Yann
FYI this is not being forgotten, we still have a card about this assigned to someone, it will just take a bit more time than I thought before we work on this.
I'll send updates here when we have new elements.
Regards,
Yann
@yllar I'm not sure honestly (I will have to ask the rest of the team), but anyway, even if it were fixed by latest xen package, we still didn't publish a new installer ISO with the fix.
Target is still around June for the new ISO.
Regards,
Yann
@maximsachs Thanks for getting back to me on this.
I think (and hope) that in the mean time we will have released an official new ISO that fixes the issue you are having.
So, you will be better off testing the new official ISO rather than this unofficial one
I'll try to ping here when the new official ISO is out.
Thanks!
Regards,
Yann
@yllar @mik @Pilow @marcvds Hello, we think we now understand better this issue.
We think it's a Xen issue with regard to PCI devices on segment 0001 (which is your case) and not a dom0 driver issue.
Could you give a try to this ISO installer please?
Please note that this is a custom test ISO and it is therefore not officially supported, do not use it for production, it is just to confirm that it fixes your issue, we will release a supported ISO in the future (target date for now is June).
unsupported test ISO for you to test: https://nextcloud.vates.tech/index.php/s/iHf8sqq39EqE683
Regards,
Yann
Hello @maximsachs,
on top of trying what @thierryescande told you, could you also give a try to this ISO installer please?
We have done some internal investigations and we think that the issue could be because of an issue in Xen handling of PCI devices that are on 0001 segment which seems to be your case.
Please note that this is a custom test ISO and it is therefore not officially supported, do not use it for production, it is just to confirm that it fixes your issue, we will release a supported ISO in the future (target date for now is June).
unsupported test ISO for you to test: https://nextcloud.vates.tech/index.php/s/iHf8sqq39EqE683
Regards,
Yann
@dkg Oops sorry I thought you were someone else having the same issues.
Hello @codemercenary
In theory Xen is NUMA aware (See https://wiki.xenproject.org/wiki/Xen_on_NUMA_Machines) so you would leave memory interleaving option of the BIOS off so that Xen does what it thinks is necessary.
But these kinds of settings are very very workload sensitive and in fact it's usually a good idea to just test what works best for you.
Regards,
Yann
@timemaster5 To give you more feedback, I am currently testing with an AMD Ryzen Embedded V2748 with Radeon Graphics host with a Renoir [Radeon Vega Series / Radeon Vega Mobile Series] iGPU.
I'm also talking to XAPI team to coordinate the effort and try to see what we can/want to do about this.
My first effort is to confirm what works/does not work and give all the information they need to the XAPI team so that they can make the decision on what to do.
I'll update you with my findings.
Can you retry with an up-to-date xcp-ng 8.3 please?
FYI on recent XCP-ng 8.3 versions the pci-passthrough will enable the ROM expansion bar. The guest VM will have access to it, so no need to pass it via qemu anymore.
See my comment on GitHub: https://github.com/xcp-ng/xcp/issues/786#issuecomment-4281846490
Regards,
Yann
@gleveque Hello
Can you retry with an up-to-date XCP-ng 8.3 host?
PCI expansion ROM are now enabled when doing pass-through.
The guest VM can now access the ROM bar.
Regards,
Yann
