Our future backup code: test it!

olivierlambert

A big change is coming!
As we prepare to add qcow2 support for backups, we took the opportunity to redesign major parts of the backup engine. The result? A much more flexible and abstracted system that can better handle various scenarios like V2V, qcow2, VHD, and more.

We're also moving from traditional streams to Node generators, adding backup throttling, and laying the groundwork for future improvements.

It's not production-ready yet, but that’s where you come in!
If you're working from sources, you can test it by switching to the branch: feat_generator_backups. Bug reports and feedback are more than welcome!

More tests from you means we could put it in production sooner

For reference, the PR is here: https://github.com/vatesfr/xen-orchestra/pull/8432

Adding directly @florent in the loop because he's the guy to talk to

fbeauchamp opened this pull request in vatesfr/xen-orchestra

open feat(backups): use generator instead of streams for backup and replication #8432

Delgado

@olivierlambert I just tried to build the install with that branch and got the following error.

• Running build in 22 packages
• Remote caching disabled
@xen-orchestra/disk-transform:build: cache miss, executing c1d61a12721a1a1b
@xen-orchestra/disk-transform:build: yarn run v1.22.22
@xen-orchestra/disk-transform:build: $ tsc
@xen-orchestra/disk-transform:build: src/SynchronizedDisk.mts(2,30): error TS
2307: Cannot find module '@vates/generator-toolbox' or its corresponding type
 declarations.
@xen-orchestra/disk-transform:build: error Command failed with exit code 2.

OS: Rocky 9
Yarn: 1.22.22
Node: v22.14.0

Let me know if you need anymore information.

olivierlambert

It's a bit early, it should be ready for Monday

florent

@Delgado relevant packet aren't published yet, and we are testing it as extensively as possible before letting you play with it

Delgado

Thanks! Looking forward to trying it out!

flakpyro

@olivierlambert @stormi this looks interesting! What will be the main advantages of using "Node generators" instead of streams for backups?

john.c

Yes it definitely looks really good. Will it be able to perform File Level restore, on multiple disk images in LVM LVs in a LVM VG on a single VM?

The reason being a lecturer on a Linux System Administration course, who's lecturing based on experience. The lecturer's experience is from over 30 Years of System Administration at a wide variety of Corporations including the Fortune 500.

During the course the lecturer has the student's setup VMs using LVM based drive pools for storage. Then have them learn how to add more drives to the LVM VG and increase the size of LVM LV and the filesystem formatted on the logical volume.

If the current and proposed backup engine can't support File Level restore on LVM could it be added please?

olivierlambert

No change in feature in the end, however the problem with LVM could be easily solved with "Instant restore" feature, coming somewhere in 2025.

john.c

@olivierlambert said in Our future backup code: test it!:

No change in feature in the end, however the problem with LVM could be easily solved with "Instant restore" feature, coming somewhere in 2025.

Thanks for the reply and all of your company's hard work. When the date gets closer I'll let the Linux System Administration lecturer know about using VMs via XCP-ng, XOA and Instant Restore.

CodeMercenary

Will this add the ability to control which disks on the VM will be backed up? I'd love to be able to select specific disks on a VM to backup and leave others out.

Maybe configured at the VM level rather than the backup level. Flag a disk as not needing backup and then the regular backup procedure would ignore it. However, I could also see why it might be better to control it by creating a specific backup for that VM so you could have different backup schedules, some that backup those extra disks and some that don't. I have no need to ever backup the extra disks at the moment though.

flakpyro

@CodeMercenary I think what your asking for is already possible using [nobak] and [nosnap]
https://xen-orchestra.com/blog/xen-orchestra-5-96/

CodeMercenary

@flakpyro Oh thank you. I try to go through every update post but I must have missed that one.

It worked

Makes a big difference for this VM if I back up a partially filled 250GB disk vs a largely full 8,250GB set of disks when 8TB of that doesn't need to be backed up.

florent

@CodeMercenary that is exactly the use case of the [NOBAK]
the best way to speed up a transfer is to skip the transfer

florent

@john.c no, but this is high on our backlog, but there is no easy solution to autodetect the settings without booting the VM

Still, we have a cli to mount a VHD as a raw disk : https://github.com/vatesfr/xen-orchestra/tree/master/%40vates/fuse-vhd , this can allow you to then mount any complex setup . This hide the complexity of the vhd chain, then encryption at rest, vhd blocks, and storage difference.

florent

@flakpyro today the backup code use binary stream in a the vhd format. This format is limited, by design , to 2TB disks
xcp-ng team introduce the qcow2 format to handle bigger disk
By using a independant format, we'll be able to handle both vhd and qcow2 on the backup side without multiplying complexity. We'll also be able to build the adapter to handle the various vmdk sub format (rax, cowd, sesparse and stream optimized) used by v2v and import bigger disks directly

Tristis Oris

Could i test it on prod, along with real backups? or better to setup 2nd XO instance.

olivierlambert

It's your call, but there's no guarantee it won't break for now, as the code is relatively young.

Tristis Oris

@olivierlambert but could it change old backup tasks\compability or something else? tests with real data obviously more helpful.

olivierlambert

I will let @florent answering this one

Davidj 0

I'm still getting "Cannot find module '@vates/generator-toolbox'" errors when I try to build it. I'm using the 'feat_generator_backups" branch.