There are any commands that allow me to verify the integrity of the backup files?

cbaguzman

Thanks @Bastien-Nollet and @olivierlambert .
I'm going to do some tests and write the results. Thank you so much.

cbaguzman

Hello @olivierlambert, @bastien-nollet, Hello Everyone !!

I used vhd-cli with its new arguments and it worked ok.

vhd-cli check --chain 'file:///run/xo-server/mounts/11934fec-f3a1-4f7f-a78d-00eeb1b39654?useVhdDirectory=true&encryptionKey=%22O1xt1ZTRE%5Qq%3X%3D1NZ%26%3ZZQo8%2DD%29s5xt%3OOko%22' 'xo-vm-backups/abc3e130-923e-619e-4fdc-59bcda088586/vdis/cecc4489-722a-4f79-b5e6-c050f63f2761/d489173c-028a-46f0-b712-f4a7c5594c8f/20260108T160552Z.alias.vhd'
ok: xo-vm-backups/abc3e130-923e-619e-4fdc-59bcda088586/vdis/cecc4489-722a-4f79-b5e6-c050f63f2761/d489173c-028a-46f0-b712-f4a7c5594c8f/20260108T160552Z.alias.vhd

But I discovered that using vhd-cli in this way only checks the physical structure of the alias.vhd file. It doesn't check the integrity of the blocks or the total number of blocks to detect a missing block. (It doesn't work like it does with huge monolithic .vhd files, where it detects errors with the slightest modification.)

So I opted to split the solution:

I check the integrity of the filesystem where the files are stored (using btrfs scrub).
I verify that the number of blocks written in the backup is correct (that no blocks are missing).
I verify that the written blocks haven't been modified since they were written.

For point 2, I found (AI) that I could use "vhd-cli info" to find out the total number of blocks in the backup (I don't have much information about it), but when I run it, I get the following error. I suspect it doesn't work with (Encrypt all new data sent to this remote):

vhd-cli info --chain 'file:///run/xo-server/mounts/11934fec-f3a1-4f7f-a78d-00eeb1b39654?useVhdDirectory=true&encryptionKey=%22O1xt1ZTRE%5Qq%3X%3D1NZ%26%3ZZQo8%2DD%29s5xt%3OOko%22' 'xo-vm-backups/abc3e130-923e-619e-4fdc-59bcda088586/vdis/cecc4489-722a-4f79-b5e 6-c050f63f2761/d489173c-028a-46f0-b712-f4a7c5594c8f/20260108T160552Z.alias.vhd'
✖ Unhandled remote type
Error: Unhandled remote type 
at getHandler (/usr/lib/node_modules/vhd-cli/node_modules/@xen-orchestra/fs/dist/index.js:48:11) 
at getSyncedHandler (/usr/lib/node_modules/vhd-cli/node_modules/@xen-orchestra/fs/dist/index.js:54:19) 
at Object.info (/usr/lib/node_modules/vhd-cli/commands/info.js:98:24) 
at Object.runCommand (/usr/lib/node_modules/vhd-cli/index.js:32:13) 
at /usr/lib/node_modules/vhd-cli/node_modules/exec-promise/index.js:57:13 
at new Promise (<anonymous>) 
at execPromise (/usr/lib/node_modules/vhd-cli/node_modules/exec-promise/index.js:56:10) 
at Object.<anonymous> (/usr/lib/node_modules/vhd-cli/index.js:41:1)

at Module._compile (node:internal/modules/cjs/loader:1706:14)

at Object..js (node:internal/modules/cjs/loader:1839:10)

Could you please help me with this? I'm not sure if I'm on the right track. Perhaps there are other ways I'm missing.

Bastien Nollet

Hi @cbaguzman,

I tested on my own and I got the same result as you, but then I realized the AI you used both tricked us into thinking that the --chain was a valid option for the info command (it's not).

I removed this option and the command worked properly.
Can you try the same command without this option?

cbaguzman

@Bastien-Nollet
I used vhd-cli info without the --chain option and it worked correctly. Thank you.

But vhd-cli info isn't the solution for me.

From what I've researched so far, it seems that to have cross-control of the backups, I'll have to use xo-fuse-vhd to mount the backup aliase.vhd and then run dd if=/mnt/vhd0 of=/dev/null bs=4M status=progress to verify the consistency of all the blocks in the backup.

The problem with this mechanism is that each time the verification is run, it checks the entire VDI and not just the incremental blocks of the aliase.vhd file, making it very resource-intensive for daily use.

Any suggestions would be welcome.

olivierlambert

But how one could check the integrity of a few blocks without context of the parent?

florent

@cbaguzman I am not sure checking a vhd without its chain is giving much benefits

you may be able to use vhd-cli raw <vhd> <output> it will generate a raw disk of the source, you should be able to write it to /dev/null and it will fail if any block is unreadable

cbaguzman

Thank you @florent and @olivierlambert

One of the initial alternatives I considered was verifying the integrity of each incremental backup. The idea was that the first backup file would verify all the blocks that make up the backed-up VDI, and subsequent delta backups would only verify the new blocks belonging to the delta. (But I couldn't find a way to do this.)

For now, I think I can only verify the integrity of an .alias.vhd file from a delta backup, which will also include the blocks from the parent backup, by mounting alias.vhd with xo-fuse-vhd.

Now I'm going to test the suggestion that @florent sent me.

Any suggestions are welcome.

Thank you very much.

cbaguzman

@florent , I tried the option of running "vhd-cli raw alias.vhd /dev/null".

I was reading that vhd-cli raw copies the entire VM disk from the backup to /dev/null, reading the incremental backup file and the parent backup.

But something seems off here.

The VM disk in the backup has 133GB of block files within /data. And when I ran the tool, it took 7 seconds. The backup is on a USB hard drive (HDD), so I suspect the read speeds aren't higher than 30-60MB/s.

Just reading all the blocks should take at least 40 minutes.

Did I misunderstand how vhd-cli raw works?

Tank you for your attention.

florent

@cbaguzman it will read only the allocated blocks of the delta , so it should be quite fast, depending on how full is this disk delta

Bastien Nollet

@cbaguzman for information, I made some changes on vhd-cli so in the future we'll get a more explanatory error message when a command failed because we passed an incorrect argument: https://github.com/vatesfr/xen-orchestra/pull/9386

b-Nollet opened this pull request in vatesfr/xen-orchestra

There are any commands that allow me to verify the integrity of the backup files?

closed feat(vhd-cli): prevent using invalid options #9386