@christopher-petzel Great news! I will test and report after it is available in a global update. Thank you Christopher.

All hosts are restarted with 9.2.18 loaded now. Will see what happens next. [15:56 ovbh-pprod-xen05 ~]# drbdadm --version DRBDADM_BUILDTAG=GIT-hash:\ 71c8bcff6ea77a022b272a7eba649a774251bac4\ build\ by\ @buildsystem\,\ 2025-11-03\ 10:21:36 DRBDADM_API_VERSION=2 DRBD_KERNEL_VERSION_CODE=0x090212 DRBD_KERNEL_VERSION=9.2.18 DRBDADM_VERSION_CODE=0x092100 DRBDADM_VERSION=9.33.0

So I ended up going with the Windows B&R as i could net setup the windows mount server when setting up the repo for the backups. It would setup the linux one but not the windows one. This was with using the veeam console from a windows client. So i ended up with the Windows one. Now after a few backups i am see this warning / error.... 8/5/2026 5:34:50 PM Warning : Failed to use CBT: [Task e75c4247-2ee1-7b51-088c-b970ace60f3d (Async.VDI.list_changed_blocks) failed: . SR_BACKEND_FAILURE_460. . Failed to calculate changed blocks for given VDIs. [opterr=Source and target VDI are unrelated] First i thought it was because i maped the new job to older backups from Beta v1. So i purged all old backups and started fresh. The full backups were successful. Now on first delta 4 out of 7 vms have that warrning. This a Veeam issue or xcp-ng?

@andriy.sultanov @andriy.sultanov said: @pandusen As Teddy said above, you can't passthrough a PCI bridge, so there's no PCI devices xapi shouldn't omit here. I am not trying to pass through the bridge only the end points. The Intel arc's have 2 end points: The GPU and the Sound device. "xe pci-list" only reveals the GPU, not the sound device. (this works for nvidia and AMD) But "going the xen-cmdline way" shouldn't break anything, that's what xe pci-disable-dom0-access does behind the scenes. What issues did you see? Which steps did you follow? the sound device is available in the lspci list and can be passed through using CLI. But doing so, (using CLI for passtrough) undoes everything done using xe or the passthrough gui in XO. and results in this: https://xcp-ng.org/forum/topic/10609/xcp-ng-8.3-pci-passthrough-issue so yes, its does break something.

Déja une piste intéressante, on dirait que pour une raison quelconque l'iso est mount deux fois [image: 1786437651889-14d66748-1d54-411a-b62a-a3691d6decd9-image.jpeg]

@acebmxer all good and thank you again for sharing your discovery on the forums

Good news that it sorted itself out. Both changes went in at once and the next run came back clean on the ADDS VMs too, so I don't think there's anything left to pin down here, and I'd rather say that than invent an explanation after the fact. Your question about the snapshot modes was the useful thing to come out of this thread. I went looking, they genuinely aren't documented anywhere, and that's written up on our side now If the fall back to a full ever comes back on the ADDS VMs specifically, that would deserve its own thread with the SMlog output @tjkreidl asked for, since nobody got to look at that part.

@olivierlambert Happy to hear that they are providing results to be looked at. If there is anything else I can provide, please don't hesitate to ask. And I'll do my very best to be helpfull

Excellent! It's really cool to see how XO Backup and Replication can help you to migrate easily

@dinhngtu said: @john.c I'll consider adding relevant GPOs for this. There's no real complexity difference compared to normal Registry settings. As for event logs, what information do you need? The various guest agents already report via Event Log exclusively. @dinhngtu That is fantastic news regarding the GPOs—having native administrative template support will be a huge win for enterprise deployments! Regarding the Event Logs, my apologies for the ambiguity; I am aware the agents use Event Logs internally. The issue from a fleet management perspective is that when everything is running normally, the logs are quiet, but when a breakdown happens (like a VM losing its IP reporting, or an auto-update breaking xenflt), finding why it failed from a central monitoring system is difficult. To make the Event Logs highly actionable for SIEMs, monitoring agents, and automated alerting, having explicit Event IDs for the following critical lifecycle and failure states would be incredibly useful: Management Lifecycle & Heartbeats Service Lifecycle: Distinct Event IDs for a clean initialisation/handshake with the hypervisor vs. an unexpected service termination. IP/Network Metrics Reporting: An error or warning Event ID raised if the agent fails to bind to the guest network stack or fails to transmit the IP metrics to Xen Orchestra after a specific timeout window. Backup & VSS Operations Quiesced Snapshots: Dedicated Event IDs tracking when a VSS / quiesced snapshot request is received from the host, whether it succeeded, or a specific error ID if it times out or fails (which often causes backup failures on the XO side). Driver & Filtering Health UpperFilters Validation: A warning or error Event ID if the agent detects that its driver filters (like XENFLT) are missing from the registry or have been overwritten by a Windows Update cycle. Upgrade & Missing Tool States “Installed but Inactive" Detection: An alert Event ID if the agent service is running but cannot detect active underlying PV drivers. This is critical for catching scenarios where major Windows or Windows Server feature updates silently strip out driver components without running a proper uninstaller, leaving sysadmins caught flat-footed with a broken stack. Providing explicit, documented Event IDs for these states would allow sysadmins to build standard Windows Scheduled Tasks or monitoring rules (e.g., automatically restarting the Xen Guest Agent service the moment a specific "Failed to report metrics" Event ID is caught).

@MajorP93 said: What you are describing must be a storage live migration then (XenMotion). That should work across pools, yes. I stand corrected because I did not factor in this variant in my initial answer. Yes, that's it - sorry - I should have been more explicit, even if 'lumbers along' wasn't a hint //EDIT: also there is no such thing as non-pool hosts. Even a single host setup has it's own pool. Pedantic, but true I technically meant 'not hosts in the source pool' I guess.

Got this issue since 2026-07-12.

@acebmxer Thank you for your quick feedback.

Hello @anthoineb, We captured another occurrence on the same VM, os-ott-data-3-2 (172.30.52.193), on the same hypervisor and QCOW2 VDI as the incident reported on 2026-07-25. This occurrence is useful because xen_blkfront.max_ring_page_order=3 was definitely active: xenstore showed ring-page-order=3 with eight ring references; the guest queue depth was 256; 255 write requests remained in flight; all 256 blk-mq scheduler tags were busy; wbt_lat_usec=0; I/O PSI full was approximately 98%; completed xvdb counters did not advance between samples. The backend state was nevertheless the same as in the previous capture: tapdisk was sleeping in scheduler_wait_for_events(); tap-ctl reported reqs_outstanding=0; tap request counters were 0/0; req_prod=0 and rsp_prod=0; no tap, image, VBD, map or xenbus errors were reported. One detail may be relevant: GDB still printed n_reqs=32 and n_reqs_free=32, although the frontend had negotiated an order-3 ring and the guest exposed 256 tags. Is this expected because n_reqs represents a separate fixed tapdisk request pool, or could it indicate that tapdisk is not observing the complete multi-page ring? The controller captured GDB and the guest/hypervisor state before reboot. After the ten-minute diagnostic window, the same signature was confirmed three more times and the VM was rebooted. It rejoined the cluster successfully. I have attached: the complete GDB/tap-ctl capture; the guest blk-mq, diskstats, PSI, D-state stack and kernel-journal snapshot; the corresponding tapdisk/xenstore, daemon.log and SMlog excerpt; a README and SHA-256 checksums. The important new result appears to be that increasing max_ring_page_order from 0 to 3 did not prevent the stall; the same mismatch recurred with 255/256 frontend requests/tags occupied while the backend ring and tapdisk remained empty. 20260730-005220-gdb-172.30.52.193.txt 20260730-pre-reboot-guest-172.30.52.193.txt 20260730-pre-reboot-hypervisor-172.30.50.191.txt README.txt SHA256SUMS.txt

@pierrebrunet Hi! I'm on vacation now, so sorry for the delay. In the meantime I've been analyzing these issues and it seems these are somewhat expected and still happen with minpartsize set to 100M. My next step is to increase the AWS_MAX_ATTEMPTS from the default 3 attempts to 10. This might reduce the number of failed VMs and avoid the need to retry the full VM backup. From AI research, it seems rclone uses 10 attempts instead of the AWS SDK default. Here are some stats of my weekly full backup to B2: "00 Weekly Full Backblaze" ran xxxx → xxxxx — 34.1 hours total, moving ~2.86 TiB across 89 VMs. Final status: failure (more on that below, because it's misleading) -> this is related to XO reporting the backup as a failure even though the follow up retry succeded which is a known issue. Every one of the 7 first-attempt errors shows attempts: 3 in the SDK metadata — the default budget was exhausted in all cases -> 7 of 89 VMs failed on the initial attempt. Error mix: 5× Backblaze InternalError 500, 1× ECONNRESET, 1× EPIPE The 7 failed first attempts burned ~7.1 hours of cumulative transfer time Even though XO does not expose AWS_MAX_ATTEMPTS parameter I might be able to set it using an environment variable. I'll give a try when I'm back at the office. Tks, Pedro

@yannsionneau , may this link helps if you not anyway know it https://medium.com/@timemaster5/getting-amd-phoenix-igpu-passthrough-working-on-xen-xcp-ng-8856593f9c0d I will give it a try this weekend

@TrapoSAMA said: @john.c I have this iso but how to know what is de package? Thx It’s going to be an msi installer package or on the guest tools iso, which is on the XCP-ng or Xen Server your running likely in Local Storage SR or Shared Storage SR (if moved). Named the same as the file linked to above, likely. Anyway did you know that the UEFI SecureBoot certificates (at least one of them from 2011) issued by Microsoft are expiring fully during October 2026. Around that time likely at sometime after then those certificates will be placed into dbx. This means the Windows Server 2012 R2 if operating as a UEFI SecureBoot VM, will if restarted (or reboot following a crash), fail to boot. So your client will go from having a working app that they can use, but not upgrade at the moment to one which won’t work and can’t be used! They really need to upgrade or migrate from Windows to Linux (released during 2025 or 2026) along with the app if needed. Alternatively upgrade to Windows Server 2022 or 2025 and the app!

I put this on a spare host rather than guess at it twice. The delay runs after the start, not before it. With start-delay=20 the VM was up in about a second and the call then sat there for the rest of the twenty before returning. So the VM you set it on is already running, and what waits is whatever was going to happen next. Ordering does exist, but it belongs to vApps: an appliance groups VMs and order decides which goes first (https://docs.xcp-ng.org/vms/vm-lifecycle#vapps). The part that surprised me is that nothing starts an appliance when the host boots. I put both test VMs in one, rebooted, and they stayed down. Autostart is a single bulk start of everything flagged for auto power on, it walks them one at a time, and it ignores order. I set order=2 on a VM and it still went first. So a delay does hold up the VMs behind it at boot, which is probably what you were seeing, but you cannot pick which one goes first. For your NAS and seedbox I do not think there is a clean answer today. There is an open request for exactly that, a VM that waits on another VM: https://github.com/vatesfr/xen-orchestra/issues/8433, and your case would be a good one to add to it. What I still cannot explain is why yours stopped starting at all. A delay stalling the ones behind it would explain a late start, not one that never comes up.

XO 6.7.0 went out on 30 July, and the changelog carries Bastien's fix under Backups as "Fix failed status on successful retry", pointing at PR https://github.com/vatesfr/xen-orchestra/pull/10129 and back at this thread: https://xen-orchestra.com/blog/xen-orchestra-6-7/. Given what you said about keeping the backup XOA on a version you trust, I would not rush it, but that is the version to look for when you do move. I have not exercised the retry path myself, so if a retried VM still reports the job as failed after you upgrade, it is worth saying so here.