our solution to the problem we want to share a post-incident analysis of a data-loss event on XCP-ng 8.3 LTS (shared block storage over FC, Veeam B&R 13.1 with CBT enabled on the pool) — consolidated from our own incident, topic 12402, the CBT feedback thread 9268, and the still-open blktap PR #17. The pieces fit together into one coherent failure chain, and we believe it may be worth a sticky/KB article. A note on the trigger, to be fair and complete: in our case the interrupted jobs (step 2) were caused by expired/invalid worker tokens on the XCP-ng hosts, so the backups died mid-run with the locks in place. We consider that our availability problem. However, an interrupted backup must never be able to damage production VM data — cleaning up VDI locks on job abort is the hypervisor's job, and that is the part that turned a backup hiccup into guest data loss. Failure chain (as we understand it) Veeam backup job with CBT runs and sets locks on VDIs — paused: true + host_OpaqueRef:xxx: RW entries in the VDI sm_config (XAPI state.db). The job is interrupted (timeout / crash / restart). The locks are never cleaned up → stale paused: true remains in sm_config. These entries are MRO, so xe vdi-param-remove can't clear them. A later leaf coalesce / commit runs into a cbtlog disk: per PR #17, tapdisk_vbd_first_image returns the cbtlog disk on td_commit, and the cbtlog driver has no commit action → commit fails early. Result: broken VDI chains, CBT metadata VDIs without a vhd parent, hundreds of orphaned VDIs, and .cbtlog files hanging coalesces (as reported in thread 9268). SR rescan believes a GC is already running and aborts; a host reboot was the only way to force the coalesce through (also reported in 9268). Storage cleaning freezes the VM disk briefly, but the un-freeze fails on the stale lock → failed to unpause tapdisk ... VMs using this tapdisk have lost access to the corresponding disk(s). The guest keeps writing on a frozen/lost disk → NTFS corruption inside the guest and, in our case, actual SQL Server data loss. Step 5 matches exactly the theory Veeam R&D is currently investigating ("storage cleaning freezes VM disks briefly during backup and sometimes fails to un-freeze them"). The stale paused:true lock appears to be the missing "why" behind the failed unpause. What helped us recover Patching the stale lock out of XAPI state.db (stop xapi, backup state.db, remove paused + host_OpaqueRef entries from the affected VDI's sm_config, start xapi). Then: reset CBT on the affected VDIs and trigger a full backup so CBT re-initializes cleanly — otherwise the next interrupted job re-creates the same situation. For the coalesce backlog: with the affected VMs powered off and CBT disabled, snapshot-create-then-delete to kick the GC, watch SMlog, iterate. (Same recipe a user documented in 9268.)

I went and checked a couple of the factual bits here rather than take them on trust, and the xscontainer one holds up: xscontainer-10.0.4-1.xcpng8.3.noarch.rpm is still sitting in the 8.3 base repo, and back in topic 6845 Olivier said it was meant to be removed at some point, which evidently hasn't happened. The wiki page is the bit I couldn't reproduce. There's no Docker or container page in the current docs that I can find, and nothing matching in the docs repo either, so I might be looking in the wrong place. On the proposal, I'm not the right person to say whether a containers collector fits the agent's scope. Two things I can tell you. xen-guest-agent lives on GitLab rather than GitHub, so the design conversation would need to happen there. And there's no Feeder entry for any of this yet, which surprised me given how far back the requests go; worth putting one up so the votes have somewhere to land. That's my read on where it should go rather than on whether it's a good idea, and someone closer to the agent will correct me if I've sent you the wrong way.

Rocky 10 is affected and has no fix in it. I pulled the source RPM for kernel-6.12.0-211.16.1.el10_2.0.1, which is your -211: jiffies.c still ends on core_initcall(init_jiffies_clocksource) with no cs_jiffies_registered, so f24df84cbe05 hasn't landed, and max_raw_delta sits in clocksource.h with nothing setting it early, so the regression is still there. So tsc_mode=2 and nomigrate stay the answer on Rocky until Red Hat picks it up. One more thing, because this thread reads like an AMD problem if you skim it. @dvinni measured the same stalled clock on Intel Xeon Gold in the sibling thread, and the upstream fix came from @teddyastie bisecting it on a Xen HVM guest. I read source rather than booting a Rocky VM, so if you have one behaving differently I'd like to hear it.

Question for @teddyastie but I'm not 100% sure about Intel Coffee Lake iGPU passthrough (iGPU is always far more difficult to passthrough than a discrete GPU)

@poddingue Here is what is in the Settings -> Servers [image: 1787162145156-1233d201-1ffc-4be8-b18e-666f53e00ee9-image-resized.jpeg] It looks like the migration is working now. I'm not positive what caused it. I did make some changes after I found a discrepancy with the time between the 3 systems. It was 30-50 seconds different between them.

@poddingue Thanks for running the -31 numbers — good to have it confirmed that the ttyS0 removal stays worth ~3-4s even with the clock fixed. Agreed on not rushing -proposed to production; we'll pick up -31 when it promotes and keep the cloud-init tweak permanently.