I converted the topic to a question, then marked it solved. Thanks!

Quick one to close this off: florent's fix for the encrypted-remote mirror alignment landed in a numbered release now, XO 6.6.2 (2026-07-09). The PR was https://github.com/vatesfr/xen-orchestra/pull/10061, so it's out of master and into something you can just update to. Did your retest come good on it? I'm also curious whether you still need the minPartSize=100000000 setting with the fix in place, or whether that's redundant now, since it'd help us work out whether it's worth writing into the B2 docs. fbeauchamp opened this pull request in vatesfr/xen-orchestra closed fix(fs): compute the output stream precisely when encrypted #10061

The timing evidence here looks strong to me. Your second Grafana window is doing a lot of work: linstor-satellite.service jumps to roughly 8,700 log lines in the three minutes around the panic, well above linstor-controller at 926 and xapi at 460, and the priority chart goes red at the same moment. A backup that fans out volume creates and deletes across three replicas, landing on a DRBD race, fits what you are seeing. Throttling Velero should tell you a lot. If the crashes stop with clientQPS: 3 and itemBlockWorkerCount: 1, that narrows it to concurrency rather than anything about those particular volumes. For the logs question, a xen-bugtool --yestoall bundle from a host that has panicked is usually the thing people ask for first (https://docs.xcp-ng.org/troubleshooting/log-files), since it sweeps up the kernel side alongside the storage logs you already have in Loki. I don't know DRBD internals well enough to say which trace matters most, so it might be worth a mention to @Team-Storage. They can say what they actually need rather than have you guess, and a panic that reproduces on a schedule is a good deal easier for them to chase than most.

Before you go further down the bridge tree, redakula called this one early in the thread and it's worth rereading. Intel Arc cards put an internal PCIe bridge between the slot and the GPU, and lspci on Linux only reports the top of that chain. His own A310 shows PCIe x8 4.0 at x4 4.0 in GPU-Z under Windows while Linux insists it is x1. That is the same bridge topology you're now staring at on 81, 82 and 83. TeddyAstie said much the same when he called it mostly a display issue, and he later checked that resizable BAR does work on UEFI guests since the OVMF build supports it. So the Gen1x1 reading may just be lspci describing the bridge rather than the card. I could easily be wrong, and the people already in this thread know this hardware far better than I do.

Your write-up is the kind of thing that saves the next person a weekend. One thing worth knowing before you commit to the whole-controller route. The passthrough page has a "Passing through Keyboards and Mice" section further down, and it says XCP-ng ships /etc/xensource/usb-policy.conf with DENY rules for mice and keyboards by default. You edit those to ALLOW, then refresh with /opt/xensource/libexec/usb_scan.py -d followed by xe pusb-scan host-uuid=<host_uuid>. It's at https://docs.xcp-ng.org/compute/ under USB Passthrough. I have no idea whether that covers your USB-to-serial adapter, which is a different device class, and passing the whole controller may still be the cleaner setup for two discrete workstations anyway. Might be worth a mention to @Team-Hypervisor-Kernel on the display question, because that one still puzzles me.

I had a look at your screenshots. The Topology dropdown is greyed out with a tooltip saying Requires admin permissions, so this looks deliberate rather than broken: the field seems gated on being a full XO admin, not on being admin of your own resource set. I couldn't find an existing report asking for it to respect resource-set admin instead, so feedback.vates.tech is probably the right place to raise it. It would carry more weight coming from you, with those screenshots, than from me. I don't know whether the ACL rework changes any of this, so I wouldn't count on it until someone who works on it says so. Might be worth a mention to @Team-XO-Backend, since where that permission gate lives is really their call.

You may have worked it out yourself already. A consistency check reporting inconsistent parity on Virtual Disk 1, plus Buffer I/O error on several dm- devices, is the storage layer underneath XCP-ng telling you something is wrong down there. The VDI_IO_ERROR is mostly XCP-ng saying it could not read the disk, not the cause itself. I would be careful about anything that writes to that array until someone who knows hardware RAID recovery better than I do has looked at it. I honestly don't know whether a rebuild helps or makes things worse from this state, and I'd rather say that than guess with your data. Might be worth a mention to @Team-Storage.

Thanks Greg, that's a useful data point. If your clocks are within a second across all three hosts and you're still seeing it, that makes me doubt the timezone angle as the root cause, even if the way dom0 displays the time is confusing. The thing I keep coming back to is the split you and I both see: full backups fail while the delta jobs on the same VMs never do. That's the same pattern in https://github.com/vatesfr/xen-orchestra/issues/9181, which points at large VMs or VMs with a lot of free disk space rather than anything clock-related. I'm not sure that's your case, but it might be worth checking whether the VMs that fail are the ones carrying the most free space inside the guest. MajorP93 created this issue in vatesfr/xen-orchestra open Full backup fails on large VMs / VMs with lots of free disk space #9181

I don't fully follow the mac-seed side of this, but a couple of things in the thread stand out. Tristis Oris's workaround looks like the practical unblock for now: removing the halted CR copy on the target host lets the migration go through, presumably because that replica VM is what collides on the mac-seed. Since you, KPS and Tristis Oris are all hitting the same DUPLICATE_MAC_SEED migrating into a replica target, this feels like something worth a GitHub issue on xen-orchestra with your XO commit, the exact steps, and whether a halted CR copy is present each time. It might also be worth a mention to @Team-XAPI-Network, since they'd know whether a CR replica is supposed to share its source's mac-seed. I could be wrong on the mechanism, so take that with a pinch of salt.

@poddingue You're most welcome, sir.

@PessimistTech You can try disabling ARI in your bios as a quick next step. I'm out for a few days so will attempt to assist further when I'm back if issues continue

@poddingue Hello, sorry for the delayed response. We had some issues with the Supermicro IPMI and were waiting for a resolution. Here are the two log files: dmesg_xcp-ng-8320250606-aacraid-legacy.log dmesg_xcp-ng-8320250606-aacraid-uefi.log The alternative kernel did not work with your xcp-ng-8.3.0-20250606-aacraid.iso. I am also providing the log after installing the standard xcp-ng-8.3.0-20250606.iso: The files can be found at https://drive.google.com/drive/folders/1H-0FEruGK26oclD2WInDSOBJhVW2k8Xz?usp=sharing, as I wasn't able to upload the log files directly to the forum. I am also ready to participate in any further testing of new images, up to and including XCP-ng 9.0.

@jedimarcus solution work, thanks, We had server working 100% since last two weeks but error didn't disappear, after clean logs was ok thank you.

Hi @simonp. The problem is gone with commit 44e5b!!! Up-to-date XCP-NG host and XenOrchestra VM were restarted after updating XenOrchestra. I just initiated a full backup and all VMs - including my problematic machine - were backed up successfully without error. Awesome! I will wait with closing until the scheduled delta backup is completed successfully, too. Edit: Scheduled backup ran without issue. The problem seems to have been resolved! Thanks for all your work and assistance Regards, Marc

So if i set VMs to Best-effort or disabled for HA i get not enough memory. @pkgw Do you have Twinstore configured and/or HA enabled? If no I will continue my issues at the twinstore post. If i set to restart i get this.... host.setMaintenanceMode { "id": "35233210-4e37-4703-9bf6-9e8a9c24df9f", "maintenance": true } { "code": "HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN", "params": [], "call": { "duration": 5, "method": "host.disable", "params": [ "* session id *", "OpaqueRef:d9bd976f-445c-8244-1b56-879f446efc12" ] }, "message": "HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN()", "name": "XapiError", "stack": "XapiError: HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN() at XapiError.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12) at file:///opt/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21 at runNextTicks (node:internal/process/task_queues:65:5) at processImmediate (node:internal/timers:472:9)" }

OK so that resource was a pain and did not want to give up, so I restarted the host. Migration is running good now.

The 2026-05-28 build still being good narrows the window a lot more than "sometime since November". What I keep coming back to is the shape of the failure: 14 out of 15, then 5 out of 6, so it is always exactly one that falls over and never the whole run. I don't know whether that points at a per-VM timeout or at something the last task in a run does differently, and someone on the XO team will read that better than me. @pierrebrunet still needs /var/log/xensource.log from the pool master covering one failed run's window, so even a slice from a job where only one VM failed should be enough.

Indeed, it should be detected/reported correctly to avoid losing time finding the problem. Let me ping @julienXOVates