Argh...Correction... 82 shows up as 82:01 and 82:02... lspci -vv -s 0000:82:01.0 82:01.0 PCI bridge: Intel Corporation Device e2f0 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=82, secondary=83, subordinate=83, sec-latency=0 Memory behind bridge: f7c00000-f7dfffff Prefetchable memory behind bridge: 000001f800000000-0000020002ffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0 ExtTag+ RBE+ DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+ LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [a0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [94] Subsystem: Intel Corporation Device 0000 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [220 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+ ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Kernel driver in use: pcieport And 82:02... lspci -vv -s 0000:82:02.0 82:02.0 PCI bridge: Intel Corporation Device e2f1 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=82, secondary=84, subordinate=84, sec-latency=0 Memory behind bridge: f7e00000-f7efffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0 ExtTag+ RBE+ DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [a0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [94] Subsystem: Intel Corporation Device 0000 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [220 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+ ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Kernel driver in use: pcieport

Just to give an update. I still am having the issue with current commit 090ce. Regards, Marc

Sorry it's back. Just to line things up for whoever picks this up: the original full-backup Body Timeout was tracked in https://github.com/xapi-project/xen-api/pull/6786 and shipped with the March 2026 8.3 updates, so what you're seeing on metadata backups might be a cousin rather than the same bug. From Olivier's earlier note the error just means XO got nothing from the pool master for 5 minutes mid-transfer, so the useful question is what the master was doing during that silence. The fact that a toolstack restart clears it for a while and then it returns is probably itself a clue. Could you grab the complete logs from one failed metadata run (the ask pierrebrunet made earlier), plus a rough note of whether it lines up with the master being busy or briefly unreachable? I'm out of my depth on the internals, but that's the piece the team will want. last-genius opened this pull request in xapi-project/xen-api closed stream_vdi: Only process allocated clusters for VHD and QCOW on XVA export #6786

Hello, Thanks for all !

@pedro_udifar Hi, We think we have found two issues into one: can you test this branch please? It has the changes from previous branch you built: fix_throttled_stream This should fix the stream issue and fix the warning.

@olivierlambert said: I'm not sure it's a great idea, because even 64 hosts is huge in terms of VMs and fallout if you have a problem on your pool DB. Even if you drastically improve the current mechanism, the impact of a problem pool wide is far bigger with 64 hosts than 24 for example. It's more than purely tech, it's also a tech design/choice. Exactly, the failure domain just grows exponentially when going beyond a certain collective pool size. I was doing some more research and I found a few possible choices that Tesco may have moved too, but don't want to detract from this forum here.

@laszlobortel Hehe yeah, those are pretty old kernels. I'd say there's a good chance kernel upgrades will go a long way to alleviating the CPU hangs. I can't say it's exclusively a problem with lvmohba storage, that's just what we use because of our previous VMware infrastructure was block storage over fiber channel. We knew a physical infra overhaul wasn't in the cards for us for this migration so we stayed with our existing storage system. This bug bit us half way through the migration until we figured out upgrading the kernel generally fixed it.

@All-Ki Thanks for your work! ProLiant DL360p Gen8 UID Light | 0x01 | ok Sys. Health LED | no reading | ns 01-Inlet Ambient | 33 degrees C | ok 02-CPU 1 | 55 degrees C | ok 03-CPU 2 | 50 degrees C | ok 04-P1 DIMM 1-6 | 45 degrees C | ok 05-P1 DIMM 7-12 | 44 degrees C | ok 06-P2 DIMM 1-6 | 38 degrees C | ok 07-P2 DIMM 7-12 | 43 degrees C | ok 08-P1 Mem Zone | 42 degrees C | ok 09-P1 Mem Zone | 45 degrees C | ok 10-P2 Mem Zone | 39 degrees C | ok 11-P2 Mem Zone | 40 degrees C | ok 12-HD Max | 35 degrees C | ok 13-Chipset 1 | 52 degrees C | ok 14-Chipset1 Zone | 45 degrees C | ok 15-P/S 1 Inlet | 34 degrees C | ok 16-P/S 1 Zone | 40 degrees C | ok 17-P/S 2 Inlet | 42 degrees C | ok 18-P/S 2 Zone | 43 degrees C | ok 19-PCI #1 | disabled | ns 20-PCI #2 | disabled | ns 21-VR P1 | 60 degrees C | ok 22-VR P2 | 56 degrees C | ok 23-VR P1 Mem | 39 degrees C | ok 24-VR P1 Mem | 36 degrees C | ok 25-VR P2 Mem | 32 degrees C | ok 26-VR P2 Mem | 36 degrees C | ok 27-VR P1Mem Zone | 38 degrees C | ok 28-VR P1Mem Zone | 36 degrees C | ok 29-VR P2Mem Zone | 31 degrees C | ok 30-VR P2Mem Zone | 33 degrees C | ok 31-HD Controller | 71 degrees C | ok 32-HD Cntlr Zone | 52 degrees C | ok 33-PCI 1 Zone | 43 degrees C | ok 34-PCI 1 Zone | 46 degrees C | ok 35-LOM Card | 70 degrees C | ok 36-PCI 2 Zone | 50 degrees C | ok 37-System Board | 52 degrees C | ok 38-System Board | 45 degrees C | ok 39-Sys Exhaust | 43 degrees C | ok 40-Sys Exhaust | 46 degrees C | ok 41-Sys Exhaust | 46 degrees C | ok 42-SuperCAP Max | 33 degrees C | ok Fan Block 1 | 94.86 percent | ok Fan Block 2 | 94.86 percent | ok Fan Block 3 | 94.86 percent | ok Fan Block 4 | 94.86 percent | ok Fan Block 5 | 94.86 percent | ok Fan Block 6 | 94.86 percent | ok Fan Block 7 | 94.86 percent | ok Fan Block 8 | 94.86 percent | ok Power Supply 1 | 205 Watts | ok Power Supply 2 | 210 Watts | ok Power Meter | 430 Watts | ok Power Supplies | 0x01 | ok Fans | 0x02 | ok Memory | 0x40 | ok C1 P1I Bay 1 | 0x01 | ok C1 P1I Bay 2 | 0x01 | ok C1 P1I Bay 3 | 0x01 | ok C1 P1I Bay 4 | 0x01 | ok C1 P2I Bay 5 | 0x01 | ok C1 P2I Bay 6 | 0x01 | ok C1 P2I Bay 7 | 0x01 | ok C1 P2I Bay 8 | 0x01 | ok ProLiant DL360 Gen10 UID | 0x01 | ok SysHealth_Stat | 0x01 | ok 01-Inlet Ambient | 19 degrees C | ok 02-CPU 1 | 52 degrees C | ok 03-CPU 2 | 63 degrees C | ok 04-P1 DIMM 1-6 | disabled | ns 05-PMM 1-6 | disabled | ns 06-P1 DIMM 7-12 | 44 degrees C | ok 07-PMM 7-12 | disabled | ns 08-P2 DIMM 1-6 | disabled | ns 09-PMM 1-6 | disabled | ns 10-P2 DIMM 7-12 | 48 degrees C | ok 11-PMM 7-12 | disabled | ns 12-HD Max | 35 degrees C | ok 13-Exp Bay Drive | disabled | ns 14-Stor Batt 1 | 18 degrees C | ok 15-Front Ambient | 24 degrees C | ok 16-VR P1 | 48 degrees C | ok 17-VR P2 | 54 degrees C | ok 18-VR P1 Mem 1 | 31 degrees C | ok 19-VR P1 Mem 2 | 29 degrees C | ok 20-VR P2 Mem 1 | 35 degrees C | ok 21-VR P2 Mem 2 | 36 degrees C | ok 22-Chipset | 42 degrees C | ok 23-BMC | 79 degrees C | ok 24-BMC Zone | 49 degrees C | ok 26-HD Cntlr Zone | 40 degrees C | ok 29-I/O Zone | 37 degrees C | ok 30-PCI 1 | disabled | ns 31-PCI 1 Zone | 45 degrees C | ok 32-PCI 2 | disabled | ns 33-PCI 2 Zone | 44 degrees C | ok 34-PCI 3 | disabled | ns 35-PCI 3 Zone | disabled | ns 37-Rear HD Max | disabled | ns 38-Battery Zone | 43 degrees C | ok 39-P/S 1 Inlet | 42 degrees C | ok 40-P/S 2 Inlet | 48 degrees C | ok 41-P/S 1 | 46 degrees C | ok 42-P/S 2 | 55 degrees C | ok 43-E-Fuse | 45 degrees C | ok 44-P/S 2 Zone | 53 degrees C | ok 49-CPU 1 PkgTmp | 84 degrees C | ok 50-CPU 2 PkgTmp | 94 degrees C | ok 61-AHCI HD Max | disabled | ns 69-PCI 1 M2 | disabled | ns 70-PCI 1 M2 Zn | disabled | ns 71-PCI 2 M2 | disabled | ns 72-PCI 2 M2 Zn | disabled | ns 73-PCI 3 M2 | disabled | ns 74-PCI 3 M2 Zn | disabled | ns Fan 1 | 0x01 | ok Fan 1 DutyCycle | 26.26 percent | ok Fan 1 Presence | 0x02 | ok Fan 2 | 0x01 | ok Fan 2 DutyCycle | 23.52 percent | ok Fan 2 Presence | 0x02 | ok Fan 3 | 0x01 | ok Fan 3 DutyCycle | 23.52 percent | ok Fan 3 Presence | 0x02 | ok Fan 4 | 0x01 | ok Fan 4 DutyCycle | 23.52 percent | ok Fan 4 Presence | 0x02 | ok Fan 5 | 0x01 | ok Fan 5 DutyCycle | 23.52 percent | ok Fan 5 Presence | 0x02 | ok Fan 6 | 0x01 | ok Fan 6 DutyCycle | 24.30 percent | ok Fan 6 Presence | 0x02 | ok Fan 7 | 0x01 | ok Fan 7 DutyCycle | 24.30 percent | ok Fan 7 Presence | 0x02 | ok Power Supply 1 | 0x01 | ok PS 1 Input | 200 Watts | ok Power Supply 2 | 0x01 | ok PS 2 Input | 190 Watts | ok Power Meter | 400 Watts | ok Fans | 0x01 | ok Power Supplies | 0x01 | ok Memory Status | 0x40 | ok Megacell Status | 0x04 | ok Intrusion | Not Readable | ns CPU Utilization | 252 unspecified | ok PS 1 Output | 190 Watts | ok PS_Volt_Out_01 | 12 Volts | ok PS_Volt_In_01 | 205 Volts | ok PS_Curr_Out_01 | 15.80 Amps | ok PS_Curr_In_01 | 1 Amps | ok PS 2 Output | 180 Watts | ok PS_Volt_Out_02 | 12 Volts | ok PS_Volt_In_02 | 204 Volts | ok PS_Curr_Out_02 | 15.10 Amps | ok PS_Curr_In_02 | 0.90 Amps | ok 27.1-LOM-Communi | 68 degrees C | ok 28.1-LOM Card-I/ | 84 degrees C | ok 25.1-HD Controll | 39 degrees C | ok 25.2-HD Controll | 45 degrees C | ok 25.3-HD Controll | 41 degrees C | ok LOM_Link_P1 | 0x02 | ok LOM_Link_P2 | Not Readable | ns LOM_Link_P3 | Not Readable | ns LOM_Link_P4 | Not Readable | ns ALOM_Link_P1 | 0x02 | ok ALOM_Link_P2 | 0x02 | ok Dr_Stat_1I1_B001 | 0x01 | ok Dr_Stat_2I1_B005 | 0x01 | ok CPU_Stat_C1 | 0x80 | ok CPU_Stat_C2 | 0x80 | ok

@bogikornel Same here.....was getting the errors a while back and then they stopped. Over the last week it now happens a lot. I found that it rarely happens on incremental backups.

@Danp said: Smart Reboot option found on the host's Advanced tab does what you are asking Very nice!

@julienXOvates Noted, thank you!

@the_jest Not showen in this picutre but this is where the message would be displayed. Next to the name of the host... [image: 1782931263879-screenshot-2026-07-01-144023.png]

For your storage question, it's fully explained in the doc: https://docs.xcp-ng.org/storage/#-how-to-modify-an-existing-sr-connection And yes, it's planned to get the complete error visible in XO, sadly, it's not "obvious" since the error message isn't returned by XAPI when you try to start but by another method we need to call after it fails ("assert can be started here" from the top of my head). Let me ping @julienXOvates

@pierrebrunet ticket created https://help.vates.tech/#ticket/zoom/59723

Meant to update this again last night - as a road map for those who might try similar at some point: This community's documentation led me to some relevant information on mouse/keyboard pass-through. Namely, if you want this to work you have to pass the entire PCI controller through to the guest(s), Won't work with motherboard devices. But I'm out of PCIe slots. What to do... Simple. Chuck the two serial adapters and install two dual USB 2.0 adapters in their places. Connect the keyboard/mouse to one of the ports and a 4/8/16 port USB to serial adapter to the other. Then pass the entire controller through. In this particular server (R810) I have 5 available PCIe slots. Two will be used for the video cards, one for a 4x NVMe to PCIe adapter (which is eventually where the VMs will reside) and the remaining two for the USB interfaces. This effectively gives me two discrete physical systems, a third available VM instance (say, for network management, monitoring or whatever) and the XOA instance. For application/radio control systems hypervisors (nested) I'll probably have 2ea Win11 and whatever flavor of Linux that supports the GT730 GPU on the system - each assigned the pass-through devices - but only two running at any given time. So no resource contention issues. As far as the displays: I have the main interface of the server connected via SIP to a 2160AS KVM. I'll take 2 more SIPs and use these as the keyboard/mouse inputs for the USB cards. Video...I use an HDMI active splitter on other systems in the network so I'll employ that here. One of the ports on the add-in GPUs goes to the splitter, one of the splitter outputs goes to the SIP via HDMI to VGA adapter and the other feeds the main ceiling mount monitor. The Primary display will be replicated to the KVM screen. Resolution isn't an issue as I have a video scaler inline with the KVM for just this scenario. The concept could be expanded. I'm looking at picking up a T630 and building 4 effective "systems" out of it, following the same principle. The 630 has 8 available PCIe slots. Will update when the next batch of hardware arrives - or I get the Linux side of things sorted. That's next.

@yannsionneau said: segment bus device function address of the device Hi all, thanks for your reponses! the below is done with the assumption that dom0 is just the host cli. If that is incorrect please let me know and I can update this info: dom0: lspci -vvv -nn -s 03:00.0: 03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:7551] (rev c0) (prog-if 00 [VGA controller]) Subsystem: ASRock Incorporation Device [1849:5413] Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+ Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 120 Region 0: Memory at 7f000000000 (64-bit, prefetchable) [size=32G] Region 2: Memory at 7f800000000 (64-bit, prefetchable) [size=256M] Region 4: I/O ports at 3000 [size=256] Region 5: Memory at f2200000 (32-bit, non-prefetchable) [size=512K] Expansion ROM at f2280000 [disabled] [size=128K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [64] Express (v2) Legacy Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x16, ASPM L1, Exit Latency L0s unlimited, L1 <1us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: Unknown, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO+ CmpltAbrt- UnxCmplt+ RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [200 v1] #15 Capabilities: [240 v1] Power Budgeting <?> Capabilities: [270 v1] #19 Capabilities: [2a0 v1] Access Control Services ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- Capabilities: [2d0 v1] Process Address Space ID (PASID) PASIDCap: Exec+ Priv+, Max PASID Width: 10 PASIDCtl: Enable- Exec- Priv- Capabilities: [320 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [410 v1] #26 Capabilities: [450 v1] #27 Capabilities: [500 v1] #2a Kernel driver in use: pciback VM lspci -vvv -nn -s 03:00.0: 00:08.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:7551] (rev c0) (prog-if 00 [VGA controller]) Subsystem: ASRock Incorporation Device [1849:5413] Physical Slot: 8 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+ Latency: 0 Interrupt: pin A routed to IRQ 17 Region 0: Memory at 800000000 (64-bit, prefetchable) [size=32G] Region 2: Memory at c0000000 (64-bit, prefetchable) [size=256M] Region 4: I/O ports at c300 [size=256] Region 5: Memory at e0800000 (32-bit, non-prefetchable) [size=512K] Expansion ROM at 000c0000 [disabled] [size=128K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [64] Express (v2) Legacy Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 32GT/s, Width x16, ASPM L1, Exit Latency L1 <1us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 32GT/s, Width x16 TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- AtomicOpsCap: 32bit+ 64bit+ 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, AtomicOpsCtl: ReqEn+ LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Kernel driver in use: amdgpu Kernel modules: amdgpu dom0 xl dmesg output: xldmesg.txt dom0 dmesg output: dom0dmesg.txt vm dmesg output: vmdmesg.txt I believe this is all the output requested (I only provided the lspci output for a single GPU as they are identical GPUs , but happy to provide all the outputs if desired (will note I am not entirely sure I selected the exact GPU in both contexts due to the mismatched IDs in vm vs dom0, not sure that is an issue but may be worth calling out) edit: all output was taken with the VM running (but at idle). I did apply the grub changes to use the -d.gz file and iommu=debug as well so ideally that output should be included)

PRs upstream are in review

Hello @mgr42 I can confirm that indeed today AMD iGPU passthrough is not working on XCP-ng. I am currently (slowly) working on it. There are other references about this topic in the Forum and on GitHub FYI: https://xcp-ng.org/forum/topic/11918/amd-barcelo-passthrough-issues-any-success-stories https://xcp-ng.org/forum/topic/8909/issue-to-load-gpu-passthrough-invalid-pci-rom-header-signature-expecting-0xaa55-got-0x4556 https://github.com/xcp-ng/xcp/issues/806 https://github.com/xapi-project/xen-api/commit/930ffb34e42d674b8234c3d8399fdba97efa6b38 On the setup I am working on, the Expansion ROM is provided indeed by firmware via the VFCT ACPI table. I dumped it to a file and provided it to qemu-dm xen-pci-passthrough device via the romfile= argument (via a modified XAPI build). I can now indeed see a ROM bar in the guest but unfortunately its content is not the content of the file therefore nothing works (amdgpu driver is unable to get its ROM and fails). So now I'm debugging qemu to try to understand the issue with the emulated/passed-through ROM BAR. Regards, Yann Kernel & Hypervisor team timemaster5 created this issue in xcp-ng/xcp open Unable to set romfile for xen-pci-passthrough devices #806 0 freddy77 committed to xapi-project/xen-api CA-418993: Enable PCI ROM BAR before attempting to give permissions to the VM Currently the VM is not able to access the ROM as the host ROM BAR is disabled and not set. Signed-off-by: Frediano Ziglio <frediano.ziglio@citrix.com>