Incremental backup fails on a 3 TB qcow2 disk: value is out of range ... <= 4294967295 in computeVhdBatAndFileSize

[Razielh]

Hi,

An incremental (delta) backup job fails on a VM that has a 3 TB qcow2 disk. The run is a “full” (base) export, and the transfer fails after about 4 minutes.

Environment

XCP-ng 8.3.0, sm-3.2.12-25.1.xcpng8.3
Xen Orchestra built from sources (ronivay installer), commit 3923756ff (2026-10-22, latest master at the time of testing)
Backup type: incremental / delta (this run: Type: full)
Remote: SMB share on a Synology NAS
VM: 1 VDI of 3 TB in qcow2 format

What happens

The snapshot succeeds.
The transfer to the remote fails after about 4 minutes. The same error is reported at the transfer, remote and job levels:
Error: The value of "value" is out of range. It must be >= 0 and <= 4294967295. Received 4_294_971_137

Stack trace (xo-server)

xo:backups:AbstractVmRunner WARN writer step failed {
error: RangeError [ERR_OUT_OF_RANGE]: The value of "value" is out of range. It must be >= 0 and <= 4294967295. Received 4_294_971_137
at checkInt (node:internal/buffer:78:11)
at writeU_Int32BE (node:internal/buffer:808:3)
at Buffer.writeUInt32BE (node:internal/buffer:821:10)
at DiskConsumerVhdStream.computeVhdBatAndFileSize (packages/vhd-lib/disk-consumer/BaseVhd.mjs:73:11)
at DiskConsumerVhdStream.toStream (packages/vhd-lib/disk-consumer/DiskConsumerVhdStream.mjs:62:36)
at toVhdStream (packages/vhd-lib/disk-consumer/index.mjs:31:19)
at RemoteAdapter.writeVhd (@xen-orchestra/backups/RemoteAdapter.mjs:695:28)
at asyncEach.concurrency (@xen-orchestra/backups/_runners/_writers/IncrementalRemoteWriter.mjs:284:45)
at next (@vates/async-each/index.js:90:37) {
code: 'ERR_OUT_OF_RANGE'
},
step: 'writer.transfer()',
writer: 'IncrementalRemoteWriter'
}

This is followed by VDI_IN_USE (destroy) retries on the snapshot cleanup, which look like a side effect of the failed transfer.

Analysis
The crash happens while computing the VHD BAT. The BAT stores block offsets as 32-bit sector numbers (512-byte units), so a VHD cannot address data beyond ~2 TiB. The qcow2 disk is 3 TB, and the stream’s block offsets go past that limit (4,294,971,137 sectors ≈ 2 TiB + 1.9 MiB).

In the meantime, I’m protecting this VM’s data with a file-level backup agent, but I’d like to go back to XO for the whole VM.

Thanks!

[poddingue]

Hi @Razielh,

I think that error may be the 2 TB VHD ceiling, and it should go away if the backup repository is in block mode.

The same thing came up in July on GitHub (https://github.com/vatesfr/xen-orchestra/issues/10193, a 6 TB qcow2 disk), where @Florent explained that a repository not in block mode writes each disk as one whole VHD file, which can't go past 2 TB.
The reporter made a new share on their NAS, added it as a remote with "Store backup as multiple data blocks instead of a whole VHD file" ticked, and the 6 TB VM backed up after that.
The docs say it too, though as a bullet in a list rather than as a hard limit: https://docs.xen-orchestra.com/backups-and-dr/backup-features-and-settings#remotes.

While you're in there, incremental backups of qcow2 disks also need NBD on the job, or every run falls back to a full one (https://docs.xen-orchestra.com/backups-and-dr/backup-types/incremental_backups#nbd-enabled-backups); I haven't seen your job settings, so ignore that if it's already on.

I'm not sure an existing remote can be switched to block mode in place, so a fresh share like in that issue seems the safer bet. Might also be worth a mention to @Team-XO-Backend that XO could say this up front instead of throwing a RangeError mid-transfer.

[Razielh]

Hello,

Thank you for the feedback. Indeed, by enabling the option "Store backup as multiple data blocks instead of a whole VHD file," the backup is now in progress. However, given the low-end NAS being used, it will take a long time, as the process is saturating its CPU.

Thanks!

[Razielh]

Update: switching the remote to “multiple data blocks” fixed the 2 TiB overflow. The full backup now succeeds (≈1.1M blocks, 13 h) and file restore works.

However, the next incremental always falls back to a full

The delta export request gets no data for exactly 5 minutes, then undici’s body timeout fires. So “can’t compute delta” is actually a timeout. My guess is that the host takes more than 5 min to compute the qcow2 delta of a 3 TB disk before sending the first byte (single SATA SMR disk, local EXT SR).
XapiQcow2StreamSource.close() throws because the source was never initialized. In earlier attempts, this may have left tapdisks for the qcow2 snapshots stuck and attached to dom0 (tap-ctl close timing out, SR_BACKEND_FAILURE_1200). I had to kill them manually before the VBDs could be unplugged.

CBT is disabled on all VDIs.

Is there a way to increase this timeout for qcow2 delta exports, or is a fix planned (e.g. keep-alive while the delta is computed)?
Is the close() error on an uninitialized source a known bug?

Same environment as above (XO from sources 3923756ff, XCP-ng 8.3.0, sm-3.2.12-25.1).