There are any commands that allow me to verify the integrity of the backup files?

Bastien Nollet

Currently there's no way of performing the vhd-cli check command on an encrypted remote, but this is also annoying for us and it shouldn't be too hard to fix, so I'll fix it.

Bastien Nollet

Hi @cbaguzman,

We just merged a PR that should allow you to use vhd-cli check on encrypted remotes: https://github.com/vatesfr/xen-orchestra/pull/9235

Please note that, as stated in the PR description, this PR changes the parameters you need to give to several vhd-cli commands, including the parameters of vhd-cli check. You can use vhd-cli check --help for details.

Here's an example of a command I ran on an encrypted remote:

vhd-cli check --chain 'file:///localRemoteEncryptedTEST?useVhdDirectory=true&encryptionKey=%2219876543210987654321098765432109%22' 'xo-vm-backups/695550e5-16e4-4728-6669-ac39e79f19e2/vdis/2656e4f4-efbc-47a6-95aa-0ba88632e545/7a1b144e-1ec8-44e8-9f5e-2d39119a228d/20251125T095729Z.alias.vhd'

b-Nollet opened this pull request in vatesfr/xen-orchestra

closed feat(vhd-cli): accept all types of remotes #9235

cbaguzman

Thanks @Bastien-Nollet and @olivierlambert .
I'm going to do some tests and write the results. Thank you so much.

cbaguzman

Hello @olivierlambert, @bastien-nollet, Hello Everyone !!

I used vhd-cli with its new arguments and it worked ok.

vhd-cli check --chain 'file:///run/xo-server/mounts/11934fec-f3a1-4f7f-a78d-00eeb1b39654?useVhdDirectory=true&encryptionKey=%22O1xt1ZTRE%5Qq%3X%3D1NZ%26%3ZZQo8%2DD%29s5xt%3OOko%22' 'xo-vm-backups/abc3e130-923e-619e-4fdc-59bcda088586/vdis/cecc4489-722a-4f79-b5e6-c050f63f2761/d489173c-028a-46f0-b712-f4a7c5594c8f/20260108T160552Z.alias.vhd'
ok: xo-vm-backups/abc3e130-923e-619e-4fdc-59bcda088586/vdis/cecc4489-722a-4f79-b5e6-c050f63f2761/d489173c-028a-46f0-b712-f4a7c5594c8f/20260108T160552Z.alias.vhd

But I discovered that using vhd-cli in this way only checks the physical structure of the alias.vhd file. It doesn't check the integrity of the blocks or the total number of blocks to detect a missing block. (It doesn't work like it does with huge monolithic .vhd files, where it detects errors with the slightest modification.)

So I opted to split the solution:

I check the integrity of the filesystem where the files are stored (using btrfs scrub).
I verify that the number of blocks written in the backup is correct (that no blocks are missing).
I verify that the written blocks haven't been modified since they were written.

For point 2, I found (AI) that I could use "vhd-cli info" to find out the total number of blocks in the backup (I don't have much information about it), but when I run it, I get the following error. I suspect it doesn't work with (Encrypt all new data sent to this remote):

vhd-cli info --chain 'file:///run/xo-server/mounts/11934fec-f3a1-4f7f-a78d-00eeb1b39654?useVhdDirectory=true&encryptionKey=%22O1xt1ZTRE%5Qq%3X%3D1NZ%26%3ZZQo8%2DD%29s5xt%3OOko%22' 'xo-vm-backups/abc3e130-923e-619e-4fdc-59bcda088586/vdis/cecc4489-722a-4f79-b5e 6-c050f63f2761/d489173c-028a-46f0-b712-f4a7c5594c8f/20260108T160552Z.alias.vhd'
✖ Unhandled remote type
Error: Unhandled remote type 
at getHandler (/usr/lib/node_modules/vhd-cli/node_modules/@xen-orchestra/fs/dist/index.js:48:11) 
at getSyncedHandler (/usr/lib/node_modules/vhd-cli/node_modules/@xen-orchestra/fs/dist/index.js:54:19) 
at Object.info (/usr/lib/node_modules/vhd-cli/commands/info.js:98:24) 
at Object.runCommand (/usr/lib/node_modules/vhd-cli/index.js:32:13) 
at /usr/lib/node_modules/vhd-cli/node_modules/exec-promise/index.js:57:13 
at new Promise (<anonymous>) 
at execPromise (/usr/lib/node_modules/vhd-cli/node_modules/exec-promise/index.js:56:10) 
at Object.<anonymous> (/usr/lib/node_modules/vhd-cli/index.js:41:1)

at Module._compile (node:internal/modules/cjs/loader:1706:14)

at Object..js (node:internal/modules/cjs/loader:1839:10)

Could you please help me with this? I'm not sure if I'm on the right track. Perhaps there are other ways I'm missing.

Bastien Nollet

Hi @cbaguzman,

I tested on my own and I got the same result as you, but then I realized the AI you used both tricked us into thinking that the --chain was a valid option for the info command (it's not).

I removed this option and the command worked properly.
Can you try the same command without this option?

cbaguzman

@Bastien-Nollet
I used vhd-cli info without the --chain option and it worked correctly. Thank you.

But vhd-cli info isn't the solution for me.

From what I've researched so far, it seems that to have cross-control of the backups, I'll have to use xo-fuse-vhd to mount the backup aliase.vhd and then run dd if=/mnt/vhd0 of=/dev/null bs=4M status=progress to verify the consistency of all the blocks in the backup.

The problem with this mechanism is that each time the verification is run, it checks the entire VDI and not just the incremental blocks of the aliase.vhd file, making it very resource-intensive for daily use.

Any suggestions would be welcome.

olivierlambert

But how one could check the integrity of a few blocks without context of the parent?

florent

@cbaguzman I am not sure checking a vhd without its chain is giving much benefits

you may be able to use vhd-cli raw <vhd> <output> it will generate a raw disk of the source, you should be able to write it to /dev/null and it will fail if any block is unreadable

cbaguzman

Thank you @florent and @olivierlambert

One of the initial alternatives I considered was verifying the integrity of each incremental backup. The idea was that the first backup file would verify all the blocks that make up the backed-up VDI, and subsequent delta backups would only verify the new blocks belonging to the delta. (But I couldn't find a way to do this.)

For now, I think I can only verify the integrity of an .alias.vhd file from a delta backup, which will also include the blocks from the parent backup, by mounting alias.vhd with xo-fuse-vhd.

Now I'm going to test the suggestion that @florent sent me.

Any suggestions are welcome.

Thank you very much.

cbaguzman

@florent , I tried the option of running "vhd-cli raw alias.vhd /dev/null".

I was reading that vhd-cli raw copies the entire VM disk from the backup to /dev/null, reading the incremental backup file and the parent backup.

But something seems off here.

The VM disk in the backup has 133GB of block files within /data. And when I ran the tool, it took 7 seconds. The backup is on a USB hard drive (HDD), so I suspect the read speeds aren't higher than 30-60MB/s.

Just reading all the blocks should take at least 40 minutes.

Did I misunderstand how vhd-cli raw works?

Tank you for your attention.