Small follow-up on the NBD connections idea, because I asked internally and it turns out that was not a false lead at all. The number of NBD connections is capped per host, so 20 per disk is well past what the host will actually hand out, and it is worth turning down whatever else is going on in this thread. I do not have the exact ceiling yet. I have asked the XAPI side for the real number and I will post it here once I have it, because guessing at it would not help anyone. @Andrew, that fits your side too as far as I can tell, since you are at 1 and have seen this once rather than constantly. probain, I will send you an upload link by direct message so you have somewhere to put the logs. I still do not know whether the fallback itself is new this week or simply newly visible, so please do not read anything into my message on that part.

@poddingue After fighting with AI a bit I think this is root cause..... The commit that broke it: 76b45a9b3b9ba636ba3d2148f8575ac4d9f57ff2 — PR #10133, "fix(xo-server): ensure import task progress is correctly updated", merged Mon Jul 27 10:23:29 2026 +0200 by fbeauchamp. https://github.com/vatesfr/xen-orchestra/commit/76b45a9b3b9ba636ba3d2148f8575ac4d9f57ff2 It was in my draft, but buried in a git log code block halfway down. Here's the reply rewritten with it as the opening line. This is a regression from a specific commit, not the certificate issue from #7496. Broken by: commit 76b45a9b3b9ba636ba3d2148f8575ac4d9f57ff2 (PR #10133, "fix(xo-server): ensure import task progress is correctly updated", 27 Jul 2026). That commit added @xen-orchestra/xapi/_watchUploadProgress.mjs, which breaks all cloud-init VM creation. There is no follow-up commit to that file on master, so current master is still broken. Why it breaks: createCloudInitConfigDrive builds the config drive as an in-memory Buffer and passes it to importContent. The new code in importContent calls watchUploadProgress(stream, stream.length, …), which calls stream.on('data', …). A Buffer has .length, so it passes the existing if (stream.length === undefined) guard, but it has no .on() method — so it throws. Every other caller of importContent passes a real stream; the cloud-init config drive appears to be the only Buffer caller. The error (only visible in journalctl -u xo-server — it's swallowed by a .catch() and never shown in the UI): 2026-07-29T21:54:49.406Z xo:xapi WARN importVdiContent: { error: TypeError: stream.on is not a function at watchUploadProgress (file:///opt/xen-orchestra/@xen-orchestra/xapi/_watchUploadProgress.mjs:25:10) at Xapi.importContent (file:///opt/xen-orchestra/@xen-orchestra/xapi/vdi.mjs:295:7) at Xapi.createCloudInitConfigDrive (file:///opt/xen-orchestra/packages/xo-server/src/xapi/index.mjs:1545:5) at Xapi.createCloudInitConfig (file:///opt/xen-orchestra/@xen-orchestra/xapi/vm.mjs:435:14) at Xo.<anonymous> (file:///opt/xen-orchestra/packages/xo-server/src/api/vm.mjs:191:26) at Task.runInside (/opt/xen-orchestra/@vates/task/index.js:204:22) at Task.run (/opt/xen-orchestra/@vates/task/index.js:188:20) at Api.#callApiMethod (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:475:18) Result: the failure is caught and logged as a WARN, so the deferred VDI cleanup never runs and execution continues to VBD_create. The VM gets a config drive attached that was never written — physical_utilisation: 3584 against a virtual_size of 10485760. cloud-init finds no user-data, hence no IP, no DHCP request, unusable console. VMs created during the failure can't be recovered by fixing XO; they must be recreated. Not a permissions issue. I switched the xo-server service account to root and rebuilt — identical error. It's a type error in the code, independent of the account. Affects: XO from sources on any build since 27 Jul. XOA lags master, which is why the same cloud-init config works there. Environment: xo-server v5.206.0, XO from sources, XCP-ng pool. Suggested fix: watchUploadProgress should return early when handed a non-stream, or createCloudInitConfigDrive should wrap the Buffer in a Readable.

I removed the start-delay parameter from one of the VMs (only this one has it) and now it works again. I don't get it. No idea how to achieve a startup delay for a specific VM though. There's also "auto_poweron_delay" and I don't understand what the difference from "start-delay" is.

Yes, you can force start and remove the backup job and that's it.