Our future backup code: test it!
-
@olivierlambert @stormi this looks interesting! What will be the main advantages of using "Node generators" instead of streams for backups?
-
Yes it definitely looks really good. Will it be able to perform File Level restore, on multiple disk images in LVM LVs in a LVM VG on a single VM?
The reason being a lecturer on a Linux System Administration course, who's lecturing based on experience. The lecturer's experience is from over 30 Years of System Administration at a wide variety of Corporations including the Fortune 500.
During the course the lecturer has the student's setup VMs using LVM based drive pools for storage. Then have them learn how to add more drives to the LVM VG and increase the size of LVM LV and the filesystem formatted on the logical volume.
If the current and proposed backup engine can't support File Level restore on LVM could it be added please?
-
No change in feature in the end, however the problem with LVM could be easily solved with "Instant restore" feature, coming somewhere in 2025.
-
@olivierlambert said in Our future backup code: test it!:
No change in feature in the end, however the problem with LVM could be easily solved with "Instant restore" feature, coming somewhere in 2025.
Thanks for the reply and all of your company's hard work. When the date gets closer I'll let the Linux System Administration lecturer know about using VMs via XCP-ng, XOA and Instant Restore.
-
Will this add the ability to control which disks on the VM will be backed up? I'd love to be able to select specific disks on a VM to backup and leave others out.
Maybe configured at the VM level rather than the backup level. Flag a disk as not needing backup and then the regular backup procedure would ignore it. However, I could also see why it might be better to control it by creating a specific backup for that VM so you could have different backup schedules, some that backup those extra disks and some that don't. I have no need to ever backup the extra disks at the moment though.
-
@CodeMercenary I think what your asking for is already possible using [nobak] and [nosnap]
https://xen-orchestra.com/blog/xen-orchestra-5-96/ -
@flakpyro Oh thank you. I try to go through every update post but I must have missed that one.
It worked
Makes a big difference for this VM if I back up a partially filled 250GB disk vs a largely full 8,250GB set of disks when 8TB of that doesn't need to be backed up.
-
@CodeMercenary that is exactly the use case of the [NOBAK]
the best way to speed up a transfer is to skip the transfer -
@john.c no, but this is high on our backlog, but there is no easy solution to autodetect the settings without booting the VM
Still, we have a cli to mount a VHD as a raw disk : https://github.com/vatesfr/xen-orchestra/tree/master/%40vates/fuse-vhd , this can allow you to then mount any complex setup . This hide the complexity of the vhd chain, then encryption at rest, vhd blocks, and storage difference.
-
@flakpyro today the backup code use binary stream in a the vhd format. This format is limited, by design , to 2TB disks
xcp-ng team introduce the qcow2 format to handle bigger disk
By using a independant format, we'll be able to handle both vhd and qcow2 on the backup side without multiplying complexity. We'll also be able to build the adapter to handle the various vmdk sub format (rax, cowd, sesparse and stream optimized) used by v2v and import bigger disks directly -
Could i test it on prod, along with real backups? or better to setup 2nd XO instance.
-
It's your call, but there's no guarantee it won't break for now, as the code is relatively young.
-
@olivierlambert but could it change old backup tasks\compability or something else? tests with real data obviously more helpful.
-
I will let @florent answering this one
-
I'm still getting "Cannot find module '@vates/generator-toolbox'" errors when I try to build it. I'm using the 'feat_generator_backups" branch.
-
Thanks for your feedback, @florent will take a look.
-
@Davidj-0 Are you using it from source or through a thrid party script ?
because this dependency is present in the branch, but still not published on npm
-
@Tristis-Oris said in Our future backup code: test it!:
@olivierlambert but could it change old backup tasks\compability or something else? tests with real data obviously more helpful.
It could gives us more informaitons, but in the other hand we are still in the validation phase.
-
F florent referenced this topic on
-
same error as above
@xen-orchestra/disk-transform:build: cache miss, executing 6b8daba7020cf148 @xen-orchestra/disk-transform:build: yarn run v1.22.22 @xen-orchestra/disk-transform:build: $ tsc @xen-orchestra/disk-transform:build: src/SynchronizedDisk.mts(2,30): error TS2307: Cannot find module '@vates/generator-toolbox' or its corresponding type declarations. @xen-orchestra/disk-transform:build: error Command failed with exit code 2. @xen-orchestra/disk-transform:build: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. @xen-orchestra/disk-transform:build: ERROR: command finished with error: command (/opt/xo/xo-builds/xen-orchestra-202504071027/@xen-orchestra/disk-transform) /tmp/yarn--1744010914016-0.7668824784043369/yarn run build exited (2) @xen-orchestra/disk-transform#build: command (/opt/xo/xo-builds/xen-orchestra-202504071027/@xen-orchestra/disk-transform) /tmp/yarn--1744010914016-0.7668824784043369/yarn run build exited (2) Tasks: 24 successful, 28 total Cached: 0 cached, 28 total Time: 11.251s Failed: @xen-orchestra/disk-transform#build ERROR run failed: command exited (2) error Command failed with exit code 2. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. + rm -rf /opt/xo/xo-builds/xen-orchestra-202504071027 -
@Tristis-Oris did you run yarn from the root of the project to install the new dependencies ?
