NBD now causes excessive backup data size

Andrew

@olivierlambert @florent @julien-f Since the big XO update push (after bee0eb9), backup (Continuous Replication and Delta) using NBD now has huge minimum data sizes (now running on master b0e00).

VMs that are 'off' normally have an update size of a few KBytes. Now (with NBD enabled) the update is 256MBytes for no reason. Turning off NBD returns backups to the normal size.

It may not see like a lot, but it adds up in both storage and transfer time.

olivierlambert

Weird Maybe @florent knows why.

florent

@olivierlambert I can reproduce it . I will look for the cause ( and a fix)

florent

@Andrew Once more, thanks Andrew for your observation skills.

There was an error when skipping the batmap, which lead to misinterpreting part of the batmap as the BAT, and not reading the end of the BAT here https://github.com/vatesfr/xen-orchestra/commit/fc1357db937ecc458e1b7d780a434c09f5463ccc#diff-cbe0b4ffbcc4f57db4dbbb7b00a7e1f333faf8c098f4137f86c7f8a75be1fee1L50

Can you try this branch, it revert the header table offset computation at the right step
https://github.com/vatesfr/xen-orchestra/pull/7264 ?

I will investigate more tomorrow to see the impacts on the transferred backups, as they may be broken due to not reading the right block addresses.

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(vhd-lib/createStreamNbd): skip original table offset before overwriting #7264

0

fbeauchamp committed to vatesfr/xen-orchestra

refactor(nbd-client): remove unused method and default iterator

Andrew

@florent I see it has been merged into master. Yes, transfer size seems to be back to before. I have not checked for damage. At worst I can just force a full backup.