@john.c said: also did you check "dmesg" output on the affected guests? real error messages are more interesting than speculation in this case Yes I did among a ton of other diagnoses, a billion times.. I am talking about findings i did over a long period(months). I am not going to sit down and write a book about something I gave up on almost a month ago. I know what I need to get it working, but I could not get to that state in XCP-ng. I could however get there using Proxmox. But that was on another system though, the x99 6900k. I am running as bare metal now... The former(my current xcp-ng box) was on a ROMED8-2T Epyc Milan. I tested that bare metal on a USB, and it also worked. Yes Above 4G has to be enabled otherwise reBar wont report a bar-0 or bar-1 at 16gb in linux, I got it to report that at one point (Sorry, I was not aware of the importence at that point, I just thought I had missed something, and was at a perfectly normal point in the process), but the guest driver still had issues(So it was no success anyway). All this led me to other findings, both here and in other discussions about people not succeeding with intel pass-through in xcp-ng. Again, I only mentioned those points, because I could see that Teddy, basically hadn't tested it beyond what I had already succeed at. "speculation" I am sry, but that was really I could get to.. I have searched everywhere, I have had Claud and GPT chew on it and looked through Intel's documention as well as my own hardware. The only thing I couldnt see, was what was happening under the xen hood. I will look through my flyby notes and if I find something of interest, or perhaps in regards to getting that reBar to report 16gb on the guest, I will post it here! My current Ubuntu bare metal, works as expected, I get a 16x8 on PCI v3 wich is expected from the 8 lanes electrical of the Arc pro b50. reBar is working and reporting correctly, and the performance numbers match what others have measured in a healthy system. Yes I have run plenty of heavy inference and compute.. mostly inference.. I have not used it for transcoding media yet, as that was a job it was suppose to have on the xcp-ng server.. It is even running a rtx 2080 card in the same box as well, which is running isolated along side the intel card and they are juggling data back and forth every day. No issues. But as I also said, I might suspect PCI grouping of certain motherboards, not playing well with xcp-ng, which might explain your "Its working on my computer" phenomenon. I am not currently at a point where I feel like taking my AI server apart and put the Intel card back in the xcp-ng server.. Maybe when something has actually changed, I would be willing to give it a shot again and I promise at that point, I will be sure to have some notes with me.

Confirming this on a newly created VM with empty disk of 2MiB. XO source: 635f0 [image: image.jpeg] [image: image.jpeg]

@poddingue Hi, we have a ticket at Vates regarding this. (Ticket#7765809)

@bogdantomasciuc Glad to hear. I was hoping it was just something missed or miss typed. If you can replicate the issue let me know. Its possible i might have missed that little nuance or something.

Third update: It looks more and more that I had goofed something up in my testing. So please disregard everything in the post below. The driver does work to make our LPe37000/LPe38000 series FC-cards work. Which is nice. Thanks! ~~Good morning So I might've found an issue with this driver release. That I'm trying to troubleshoot and collect data on. We have two LUNs presented to our dev-pool (xcp-dev), running on R650xs. When I have attached one LUN to xcp-dev I can't see the other LUN. This works on our older R350s Removing the attached LUN, makes a LUN visible again. But now we're getting some strange behaviour. And looking into if it is misconfigured from the Dell Powerstore, or if it is on the host(s) itself.~~ Details: multipath is enabled Host on xcp-dev 98:00.0 Fibre Channel [0c04]: Emulex Corporation LPe37000/LPe38000 Series 32Gb/64Gb Fibre Channel Adapter [10df:f500] (rev 11) Subsystem: Emulex Corporation Device [10df:f522] Kernel driver in use: lpfc Kernel modules: lpfc14 98:00.1 Fibre Channel [0c04]: Emulex Corporation LPe37000/LPe38000 Series 32Gb/64Gb Fibre Channel Adapter [10df:f500] (rev 11) Subsystem: Emulex Corporation Device [10df:f522] Kernel driver in use: lpfc Kernel modules: lpfc14 Update: ~~After managing to revert my previous tests and forget the existing (wrong) LUN from xcp-dev. I can now confirm a problem. There are three LUNs. Let's call them 101, 102, 103 When I attach one LUN (103) to xcp-dev, I am no longer able to see the other LUNs. That means I can't add a second one. I am expecting to be able to see 102 available. However, on old-dev, that has two LUNs attached already (101 & 102). It can see 103 as expected.~~ host on old-dev 02:00.0 Fibre Channel [0c04]: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100] (rev 03) Subsystem: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100] Kernel driver in use: lpfc Kernel modules: lpfc 02:00.1 Fibre Channel [0c04]: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100] (rev 03) Subsystem: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100] Kernel driver in use: lpfc Kernel modules: lpfc Second Update: This might be unrelated. Seems that we're having weird behaviour with the LUNs in general. Please disregard the statement above for now

@jqueiroz said: Bom dia a todos, Estou com um problema em uma VM, ela está ligando/desligando sozinha, repetidamente. Não encontro nenhuma mensagem de erro explicando o motivo. Alguém pode me dar alguma dica sobre como solucionar isso? Uso o XCP-NG 8.0 em um cluster de dois servidores HP Proliant DL-360, com o HA-Lizard. Tudo vinha funcionando bem até alguns dias atrás, quando houve uma queda de energia no datacenter da localidade. Tenho o mesmo arranjo de VMs/servidores em mais seis localidades, nenhuma apresenta esse problema. Analisar esse comportamento instável do cluster é quase como prever resultados complexos, parecendo a análise de riscos que a malta faz ao dar palpites no https://mostbetbrasil.lat. Grato. Como o problema começou depois da queda de energia e só acontece nessa localidade, eu começaria verificando os logs do XCP-ng e do HA-Lizard para entender quem está disparando os reinícios. Pode ser alguma política de HA tentando recuperar a VM repetidamente, algum problema de armazenamento ou até inconsistência no estado do host após a interrupção. Também vale conferir o estado do pool e os logs do Xen Orchestra, se estiver usando. Se possível, desative temporariamente a automação de recuperação dessa VM, com cuidado para não comprometer a disponibilidade, e veja se ela continua reiniciando sozinha. Isso ajuda a isolar a causa.

@hrvojer My posts above will help you get out of a technical debt trap that you are currently pitfall-trap trapped in! If you do this, it will drastically help to reduce your operational costs. Maintaining an old legacy stack like this—where every component from the hypervisor to the database engine is completely End of Life (EOL)—is scandalously expensive. It sucks up valuable engineering hours in firefighting, forces you into hyper-expensive custom support agreements, and leaves you exposed to massive downtime risks—which is a nightmare scenario, especially if your other organizational costs are going up at the same time. I highly recommend framing this to your leadership using these exact terms: The Pitfall Trap: Explain that the infrastructure team is currently stuck at the bottom of a pitfall trap of compounding technical debt due to a historical lack of a system-wide overhaul. The Rescue Plan: Make it clear that trying to find deprecated, 2018-era VM tools is just a short-term sticking plaster fix that buries the team deeper. What you need is a definitive rescue plan—a clean overhaul that decouples your data layer, migrates you to a modern OS/database (or an open-source alternative), and permanently lifts you out of the trap. As well as other dependencies - systems that communicate with that one, but won’t be able to without upgrade or migration, even with configuration tweaks. The Linux Performance & Driver Advantage: If you choose to migrate your database server to Linux (running PostgreSQL or MariaDB), you gain massive performance, reliability, and security boosts. Unlike Windows, you will completely bypass the fragile legacy Citrix xenvbd.sys driver ecosystem. On Linux, the paravirtualized storage and network drivers are compiled directly into the Linux kernel. The only thing you need to install separately is the modern, official XCP-ng guest agent—which Vates has beautifully rewritten as an ultra-secure, memory-safe Rust-language agent. This gives you a lean, stable database foundation that can handle heavy memory allocations and high I/O workloads without ever breaking a sweat.

Thanks for the help; this error has been resolved following the installation. However, we have a new issue regarding CBT. We are experiencing recurring CBT failures with the following configuration: Veeam Backup & Replication: 13.1.1.18 Hypervisor: XCP-ng 8.3, three hosts Identical packages on all hosts: sm-3.2.12-25.1.xcpng8.3 blktap-3.55.5-11.1.xcpng8.3 xapi-core-26.1.19-1.1.xcpng8.3 These packages were installed on October 3, 2026, before the failures occurred. Affected VM: NMNM-V-S-L007 Source disk UUID: 2f546ede-efcb-4ccf-8c23-2ea5a5e0d01f Storage: LVM over iSCSI, SR name 70TB_HDD SR UUID: 26d9572f-01c1-9cfd-79de-ca96eab601cf The backup job finishes with a Warning. The failing operation is VDI.list_changed_blocks, which returns: SR_BACKEND_FAILURE_460 Failed to calculate changed blocks for given VDIs. [opterr=Source and target VDI are unrelated] The logs show repeated failures on October 12 and 13, 2026. During cleanup of an older snapshot, CBT metadata is merged directly into the live disk despite the existence of a newer snapshot. This suggests a possible inconsistency in the CBT parent/child relationships, although the root cause has not been confirmed. The reference snapshot used on October 13 was deleted only after the CBT failures, so its subsequent absence does not explain those failures. Attached logs: cbt-old-vdi-history.txt cbt-source-disk-history.txt cbt-chain-cleanup.txt Please investigate CBT chain handling during snapshot creation and deletion, verify the installed Xen plug-in version, and advise whether a relevant hotfix or a supported procedure to re-establish CBT tracking is available. Thank you.cbt-old-vdi-history.txt cbt-chain-cleanup.txt [cbt-source-disk-history.txt.zip](Invalid file type .zip. Allowed types are: .png, .jpg, .bmp, .txt, .diff, .patch, .jpeg)

thanks @poddingue I've opened: https://github.com/vatesfr/packer-plugin-xenserver/issues/212

lspci -nn -s fra dom0 21:15 xcp-ng-test ~]# lspci -n -vvv -s 0f:00.0 0f:00.0 0300: 1002:13c0 (rev c1) (prog-if 00 [VGA controller]) Subsystem: 1002:13c0 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 42 Region 0: Memory at fc10000000 (64-bit, prefetchable) [size=256M] Region 2: Memory at fc200000 (64-bit, prefetchable) [size=2M] Region 4: I/O ports at e000 [size=256] Region 5: Memory at fc700000 (32-bit, non-prefetchable) [size=512K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [64] Express (v2) Legacy Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 16GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+ EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest- Capabilities: [a0] MSI: Enable- Count=1/4 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [c0] MSI-X: Enable+ Count=4 Masked- Vector table: BAR=5 offset=00042000 PBA: BAR=5 offset=00043000 Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [270 v1] #19 Capabilities: [2a0 v1] Access Control Services ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- Capabilities: [2b0 v1] Address Translation Service (ATS) ATSCap: Invalidate Queue Depth: 00 ATSCtl: Enable-, Smallest Translation Unit: 00 Capabilities: [2c0 v1] Page Request Interface (PRI) PRICtl: Enable- Reset- PRISta: RF- UPRGI- Stopped+ Page Request Capacity: 00000100, Page Request Allocation: 00000000 Capabilities: [2d0 v1] Process Address Space ID (PASID) PASIDCap: Exec+ Priv+, Max PASID Width: 10 PASIDCtl: Enable- Exec- Priv- Capabilities: [410 v1] #26 Capabilities: [450 v1] #27 Kernel driver in use: pciback

@john.c I've put this on the feedback portal as a feature request, so it doesn't get buried in a thread that started out about pool masters: https://feedback.vates.tech/posts/121/publish-build-date-and-package-manifest-sbom-with-the-xoa-xva-download . As far as I know votes there help the team prioritise, so if you and @Forza want it, that's the place to say so. There's an open PR adding SBOM generation (https://github.com/vatesfr/xen-orchestra/pull/10368), but going by the files it touches it covers XO's own JavaScript dependencies, not the Debian packages in the appliance. I've only looked at the file list, not the code, so happy to be corrected. How often the downloadable XVA gets rebuilt, I still don't know: @Danp said back in 2024 that it isn't rebuilt for every XOA release (https://xcp-ng.org/forum/post/81220), which would fit the 2025.12 tag, but I'd rather not guess beyond that.

Hello, We have received your request and thank you for the feedback on this matter. A task card has been created and assigned to one of our teams. It has not yet been assigned to a specific developer, but we will be sure to get back to you once it has been addressed and resolved.

@timemaster5 @dustyarmstrong I published a thread with packages to test, if you have a non production host on which you can test it would be awesome: https://xcp-ng.org/forum/topic/12523/igpu-pci-passthrough-new-experimental-packages-need-testers Thanks! Regards, Yann

Glad the ticket sorted it, @jacob.becker, and thanks for coming back to say so. I don't know whether a patch for the stable channel is planned before 6.9 moves there. If someone else on stable lands here with the same Body Timeout Error, opening a ticket like you did seems to be the quickest route for now.

If a mount uses the hard mount option, it will lock any process touching it, including unmount during shutdown, if the mount source is unavailable. It is a kernel thing and I believe there is no solution for it at present.

@bug-meister Yeah, I would say the "mechanism" is deprecated too. Getting this mechanism to work, either by using ansible-pull or by some clever scripting during the first boot, requires Ansible to be preinstalled on your VM templates. That just makes your VMs messy and bloated with additional stuff you only use once. Also, managing Ansible versions becomes more complex. If Ansible is installed from distro repositories, you will have all different versions of Ansible across different distros. If you use pip to install Ansible, it can mess up your system if Ansible dependencies are installed on a system level. You can use Python virtual environments to prevent that but requires additional managing. It's much better and much more practical to have Ansible installed and used from a single place - Control Node. That way you have one place, one OS, one Ansible version to think about. You already have a machine you are accessing your VMs from? Use that as a control node... unless it is Windows?

@acebmxer I would have said the logs depend on how you deployed XO. Basically it would be an error printed when you ran yarn start. But it depends if XO is started with systemd or something else. But as you found yourself the solution... this is not needed anymore for now. I'll see if we can report an helpful message in XO. Something like "Cannot reach docker.sock. As you're running a RedHat like distro, ensure SELinux rules let sshd reach it. See this link for more... Etc."