@MajorP93 I'l do that, didn't know about it.

@poddingue said: Narrowing it to something that changed between the late-November 2025 build and the July 2026 ones gives everyone a better place to start than "it just times out". The backup function was definitely still good in the 2026-05-28 build. So I think you should look at the last 1 month.

@pedro_udifar Hi, We think we have found two issues into one: can you test this branch please? It has the changes from previous branch you built: fix_throttled_stream This should fix the stream issue and fix the warning.

@PessimistTech said: Well thanks everyone for the responses! After some more trial and error I got it working. I dug through the bios on my board and enabled ARI Support, PCIe AER, and ACS. After reboot the VM was able to run workloads without issue. EDIT: at least partially working. Some workloads seem fine, other workloads seem to behave the same as before... Do you have examples of workloads that are struggling still? and no, didn't do anything special on the xcp host that I'm aware of. Was already using it to passthrough a radeon pro 7500 to a windows vm for revit clarity automation server. It's been a year so I'll triple check if anything special was done back then.

Argh...Correction... 82 shows up as 82:01 and 82:02... lspci -vv -s 0000:82:01.0 82:01.0 PCI bridge: Intel Corporation Device e2f0 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=82, secondary=83, subordinate=83, sec-latency=0 Memory behind bridge: f7c00000-f7dfffff Prefetchable memory behind bridge: 000001f800000000-0000020002ffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0 ExtTag+ RBE+ DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+ LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [a0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [94] Subsystem: Intel Corporation Device 0000 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [220 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+ ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Kernel driver in use: pcieport And 82:02... lspci -vv -s 0000:82:02.0 82:02.0 PCI bridge: Intel Corporation Device e2f1 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=82, secondary=84, subordinate=84, sec-latency=0 Memory behind bridge: f7e00000-f7efffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0 ExtTag+ RBE+ DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [a0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [94] Subsystem: Intel Corporation Device 0000 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [220 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+ ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Kernel driver in use: pcieport

@poddingue Just ran one more backup to confirm working, and it fell back to full backup again.... What logs or other information should I look into to find the root cause or do you still think it fall under the known issue? [image: 1783507806440-screenshot_20260708_064922.png]

@Danp No, it's not quite that. We don't use HA anywhere. But reading the source code I did notice exactly what's mentioned in that bug: in some HA cases, the xen-api code will throw a NOT_ENOUGH_FREE_MEMORY that is just absolutely misleading. At the macro scale, I think that you could say that the issue I'm running into is that the RPU migrations aren't planned out across the entire process of the RPU. If you have VMs that take up a significant fraction of some hosts' memory, failure to plan the whole process at once can lead the system to get itself "wedged" where it's unable to evacuate one of the hosts. More narrowly, you could say that the issue is that during an RPU the host evacuation code gives up more easily than it could. Say that you're trying to evacuate host A with a 64 GB VM, and you have two other hosts, B and C, with 60 GB free. It will give up, even if there are VMs that you could move from B to C (or vice versa) that would open up one of them to having 64 GB free. If the evacuation during the RPU was more resilient, then the macro-scale issue would be less of a problem. (Although with a more resilient evacuation command but no large-scale planning, you might end up with an RPU that succeeds but is wildly inefficient.)

Merci beaucoup pour vos retours ! @nathanael-h : C'est l'étape suivante !!! J'ai découvert cluster API le mois dernier et j'ai déja testé sur d'autres plateforme ! ça déchire !!! Je ne savais pas qu'il y avait un support de XCP-ng ! Je finis ma série d'article pour une installation classique et je bascule sur cluster API ! @ataxyanetwork Merci pour ton repo. Je vais regarder ça

Added local user support. Current sample config file. # Example xo-apply configuration. # Copy this into your own PRIVATE repo, edit, then: # export XO_URL=https://xo.example.lan XO_TOKEN=... # xo-apply diff config.yaml # xo-apply apply config.yaml # # Secrets never go in this file: use ${env:VAR_NAME} placeholders, # resolved from environment variables when the tool runs. # # A section that is ABSENT is unmanaged (xo-apply won't touch or report that # resource type). A present-but-empty section means "manage this type, none # should exist" (only deleted when you pass --prune). remotes: # NFS share on a NAS - name: nas-backups type: nfs host: 192.168.1.50 path: /export/xo-backups # port: 2049 # optional # mountOptions: vers=4 # optional mount(8) options # SMB / Windows share — host is "HOST\share" (single backslash in YAML # double-quoted strings must be written as \\) - name: windows-share type: smb host: "192.168.1.60\\backups" domain: WORKGROUP username: backup password: ${env:SMB_BACKUP_PASSWORD} # path: xo # optional subfolder inside the share # S3-compatible object storage (AWS, MinIO, Backblaze B2, ...) - name: offsite-s3 type: s3 host: s3.us-east-1.amazonaws.com path: my-bucket/xo-backups # bucket/directory accessKey: AKIAEXAMPLE secretKey: ${env:S3_SECRET_KEY} region: us-east-1 # protocol: http # for http-only endpoints (e.g. local MinIO) # Directory local to the XO VM (e.g. a mounted USB disk) - name: local-disk type: local path: /mnt/backup-disk backupJobs: # Delta (incremental) backup of every VM tagged "critical", every night - name: nightly-critical mode: delta vms: tag: critical remotes: [nas-backups, offsite-s3] settings: # optional global job settings, passed through to XO concurrency: 2 # timezone: America/New_York # maxExportRate: 104857600 # bytes/s # nRetriesVmBackupFailures: 2 # reportWhen: failure schedules: - name: nightly cron: "0 2 * * *" retention: 14 # backups kept on the remotes # snapshotRetention: 3 # snapshots kept on the pool # timezone: America/New_York # enabled: false # schedules are enabled by default # Weekly full backup of specific VMs, selected by name - name: weekly-full mode: full compression: zstd vms: names: [dc-01, mail-01] # ...or select by uuid: uuids: [770aa52a-fd42-8faf-f167-8c5c4a237cac] # ...or pass a raw XO smart-mode pattern for anything more complex: # raw: # type: VM # tags: # __or: [[prod]] remotes: [nas-backups] schedules: - name: weekly cron: "0 3 * * 0" retention: 8 # Disaster Recovery / Continuous Replication: instead of (or in addition to) # remotes, target one or more SRs. mode:full => DR, mode:delta => CR. - name: dr-critical mode: full # delta = Continuous Replication vms: tag: critical srs: [4991d4aa-ed84-599b-7d19-97f2f943a366] # target SR UUID(s) # remotes: [] # SR-only is fine; may be combined with remotes schedules: - name: hourly-dr cron: "0 * * * *" retention: 3 # replicas kept on the SR # Metadata backups: pool metadata and/or XO's own configuration. metadataBackups: - name: xo-config xoMetadata: true # back up XO's own config pools: [939ed551-fbd6-9868-52d8-d3997b7bf7da] # pool UUID(s) for pool metadata remotes: [nas-backups] schedules: - name: daily cron: "0 21 * * *" xoRetention: 7 # XO metadata backups kept poolRetention: 7 # pool metadata backups kept # Mirror backups: copy an existing remote's backups onto other remote(s), # e.g. push local backups offsite to S3. mirrorBackups: - name: offsite-mirror mode: full # full or delta, to match the source backups sourceRemote: nas-backups remotes: [offsite-s3] schedules: - name: nightly-mirror cron: "0 5 * * *" retention: 14 # Sequences: run backup schedules one after another. Each step names a job and # one of its schedules (from any job kind above, or already in XO). The sequence # has its own cron for when the whole chain runs. sequences: - name: nightly-then-metadata steps: - { job: nightly-critical, schedule: nightly } - { job: xo-config, schedule: daily } cron: "0 22 * * *" # enabled: false # sequences are enabled by default # timezone: America/New_York # Local users (XO's internal auth provider). Users created by an external auth # plugin (LDAP/SAML/GitHub) are NOT managed here and are never pruned. # # Passwords are write-only: XO never returns them, so a real password can't be # exported or diffed — but XO REQUIRES a password to create a user. So `export` # writes the placeholder `password: ChangeMe` for every user. # ⚠️ CHANGE these before importing into a real XO (or use a ${env:...} ref), # otherwise every new user is created with the password "ChangeMe". # The password is only used when a user is CREATED; for an existing user it is # ignored (change it in the XO UI). Only `permission` is compared for drift. users: - email: ops@example.com password: ChangeMe # required by XO; change before import permission: admin # none | read | write | admin (default: none) # Local groups. Members are referenced by email and resolved to ids at apply # time; each member must be a user defined above or already present in XO. groups: - name: operators users: [ops@example.com]

Here's the short version: this looks like the known "VDIs show up as snapshots" bug, and you haven't done anything wrong. What happens is a metadata flag on the VDI, is-a-snapshot, gets flipped to true after some of the recent 8.3 host updates, so XO (both 5 and 6) thinks those disks are snapshots and hides them from the per-VM Disks tab, even though the VDIs are fine and xe / XCP-ng Center still show them. A few older threads track the same thing, the main one is https://xcp-ng.org/forum/topic/11715/vdi-not-showing-in-xo-5-from-source . From what's said there, the fix is expected in two parts: stopping it from happening again, and a way to repair the flag on VDIs that are already affected. I don't have a timeline and could be off on the details, so please don't take that as official. In the meantime people have made the disks reappear by migrating the affected VDI to another SR, or snapshot then revert then delete the snapshot, but it's a visual fix and can come back. The VDI_IN_USE you're now hitting on start does sound worth its own detailed report, so yes please file it with the xapi/sm versions and that malformed is-a-snapshot field, and it'd be good to loop in @Team-Storage. Sorry you're stuck on this.

Great! I think this is really the best possible approach to make it clean

Hello @mgr42 I can confirm that indeed today AMD iGPU passthrough is not working on XCP-ng. I am currently (slowly) working on it. There are other references about this topic in the Forum and on GitHub FYI: https://xcp-ng.org/forum/topic/11918/amd-barcelo-passthrough-issues-any-success-stories https://xcp-ng.org/forum/topic/8909/issue-to-load-gpu-passthrough-invalid-pci-rom-header-signature-expecting-0xaa55-got-0x4556 https://github.com/xcp-ng/xcp/issues/806 https://github.com/xapi-project/xen-api/commit/930ffb34e42d674b8234c3d8399fdba97efa6b38 On the setup I am working on, the Expansion ROM is provided indeed by firmware via the VFCT ACPI table. I dumped it to a file and provided it to qemu-dm xen-pci-passthrough device via the romfile= argument (via a modified XAPI build). I can now indeed see a ROM bar in the guest but unfortunately its content is not the content of the file therefore nothing works (amdgpu driver is unable to get its ROM and fails). So now I'm debugging qemu to try to understand the issue with the emulated/passed-through ROM BAR. Regards, Yann Kernel & Hypervisor team timemaster5 created this issue in xcp-ng/xcp open Unable to set romfile for xen-pci-passthrough devices #806 0 freddy77 committed to xapi-project/xen-api CA-418993: Enable PCI ROM BAR before attempting to give permissions to the VM Currently the VM is not able to access the ROM as the host ROM BAR is disabled and not set. Signed-off-by: Frediano Ziglio <frediano.ziglio@citrix.com>

@dvdwx said: Hi Anyone already tried to review why Users with User permission cannot have the topology menu to edit on their Self-Service resource page? the have admin permission on their self service resource set but cannot edit this setting: [image: 1783021889506-620b892c-4c04-47ca-8425-064dbceee3e8-image.jpeg] [image: 1783021936003-28d03549-b715-44b6-966f-5d4a5a905825-image.jpeg] If i log with and full admin account on Xen Orchestra it can be edited: [image: 1783022030501-b4af0d96-e6c9-4aec-bd23-5b0869b9b31a-image.jpeg] We are using an HA Pool with 2 hosts to LAB for our NOC/SOC team. I already tried to create an ACL with user/tag/group with advanced permission but it release a full permission on the pool and it cannot be delivered to the team. if anyone know a workaround to fix it.. I appreciate that. As RBAC ACL is implemented for Xen Orchestra 6 you’ll likely find this more possible than currently on Xen Orchestra 5.

Ping @julienXOVates

Ping @julienXOVates

I'm doing some testing on my RAID since it appears to be a RAID VDR52: The consistency check found inconsistent parity data on Virtual Disk 1 on Integrated RAID Controller 1. Buffer I/O error on dev dm-9 Buffer I/O error on dev dm-5 Buffer I/O error on dev dm-7 async page read I will keep everyone posted

@the_jest Not showen in this picutre but this is where the message would be displayed. Next to the name of the host... [image: 1782931263879-screenshot-2026-07-01-144023.png]

@Pilow no.