Why does the backup use snapshots and not CBT
-
@rtjdamen said in Why does the backup use snapshots and not CBT:
@olivierlambert indeed, what about xenserver compatible backups solutions, can they work on xcp as well? I see we could maybe use agent based backups inside the vms for the larger ones, i will test this with one vm, but need a proper solution in the future, i believe if we can get this to work in xoa that would be the best solution.
You should look into something that is Application Aware (SQL etc) for these systems as they are likely sensitive to interruptions.
I don't know of any opensource agent based Application Aware backup solutions that would work, (edit) easily enough and not cause other headaches. Of course you would likely only be backing up your database with one of these separate tools and then use XO to restore the guest as a whole and restore whatever database there-after.
Off the top of my head a few backup solutions I've seen and used
- Bacula
- UrBackup
- Duplicati
- BackupPC
I know Bacula is/can be application aware in their paid version but I'm not sure if the opensource edition offers it.
-
@olivierlambert
No. A Proxy-VM (Windows) has to be deployed in the Pool. Nothing in Dom0.To my understanding, Commvault mounts the Snapshot-VDIs to the Proxy-VM and does the backup this way. They keep track of the block changes on the Proxy-VM.
In the simplest Setup the CommVault Server-VM itself can be the Proxy-VM. The only requirement is that it has access to all SRs where the VMs to backup reside on.
You can read the commvault docs on the internet. it is described very well.
-
@rfx77 can you provide a link please? Might worth a look
-
@olivierlambert i am currently looking at Quadric Software and storware, Quadric Software is doing the cbt backup exactly as needed, i got there description on the workflow yesterday
*When you run an ABD/hypervisor backup, the A3 will snapshot the VM for the duration of the backup data acquisition phase. After the datea is collected, the snapshot is immediately deleted.
In the case of CBT backups, the snapshot is deleted with the special XAPI command vdi-data-destroy, which frees data associated with the snapshot, but retains metadata, including the block allocation for the disk. This allows us to perform a delta backup the next time, with extremely negligible storage overhead.*
This is what we need
-
-
https://documentation.commvault.com/2024/essential/citrix_hypervisor_xenserver.html
Its in the price-range of veeam.
-
lol even XCP-ng is listed officially
-
@olivierlambert yes, maybe the documentation on xcp website can be updated to list them as well.
What is your experience with these backup tools, do u know of companies using them?
-
No experience but we'll contact them and reference them officially
-
@olivierlambert ok i will share my experience as well.
-
We are using it exclusively on all our customers. Enterprise level. Not as easy to configure as Veeam. More complex but much more capable.
We are mostly doing VM-Backups for DR and additional agent-based backups because the Application-Agents are very good and support a lot of different Applications and do not cost additional licenses when used in VMs.
I am not a big fan of AppAware Backups which only rely on the VM-Backups.
But to be honest I would really like to see a CBT enabled XOA Backup for many tiny customers which cannot afford a CommVault installation.
For now i coded my own backup which does Xen Vm Snapshots and does raw backups of the VDIs to kopia (via stdin), which does the dedup to S3. So we have dedup and save a lot of space. we have to read the whole VDI for the incrementals anyway.
-
I have my configuration for quadricsoftware up and running, so far so good, works as promised. i will let it run for a few weeks and let u know the results. the CBT is a real game changer so far!
-
I wanted to share my first experiences so far, for 1 single disk vm it works perfect, cbt is created and left only for metadata till the next backup.
For vms with 2 disks i run into some strange behavior, after the bacup job is see one of the 2 disks snapshot (random) at the health page as not connected to a vm and when i run the backup job again it gives an error "source and target are unrelated", the orphan vdi section.
not shure if this is an issue on XCP side or on a bug inside there software.
as i cannot seem to reproduce this issue on single disk vm's i feel like this is an issue with multiple disks. i am contacting there support to see if this is a bug on there end or if this is an issue inside XCP itself.
Keep u all posted on the progess of this tests.
-
@rtjdamen said in Why does the backup use snapshots and not CBT:
@olivierlambert i am currently looking at Quadric Software and storware, Quadric Software is doing the cbt backup exactly as needed, i got there description on the workflow yesterday
*When you run an ABD/hypervisor backup, the A3 will snapshot the VM for the duration of the backup data acquisition phase. After the datea is collected, the snapshot is immediately deleted.
In the case of CBT backups, the snapshot is deleted with the special XAPI command vdi-data-destroy, which frees data associated with the snapshot, but retains metadata, including the block allocation for the disk. This allows us to perform a delta backup the next time, with extremely negligible storage overhead.*
This is what we need
we did a poc of it
we still need a snapshot ( or to stop the VM), bu at least we won't keep it
the hard part is that we need to change the way we store metadata : today it is stored at the VM snapshot level, but this snapshot is deleted after a CBT enabled backup
Hopefully it will be done this year -
@florent quadric currently keeps the snapshot as well but it only contains the cbt metadata, i believe very similar as we are currently running the backups with XOA
-
Yes, you can keep it but just remove the snap data while keeping metadata (IIRC, it's been a while I took a look)
-
@olivierlambert ok, is there a way we can keep an eye on this devepment? shall i open a feature request on this with support or what is the best way to proceed?
-
That might help, yes
-
@olivierlambert i have created a feature request for this over support.
in the meantime i will let u know the results with the current software we run, if this is working as required this can be a good addition for the environment document as well.
-
Hi all,
Just an update on our progress, i have been doing backups for around 10 larger vms now on CBT, i have seen this is working pretty well but it is not the wonderfull solution we had in mind.
First of all, the Alike backup software we now use to make CBT backups does have issues with multi disk vms, for some reason now and then the CBT got disabled by XEN on one of the disks resulting in a new baseline that needs to be created the next day, this is causing higher and longer backup times, still workable but it seems there are still issues with CBT in XCP-NG, i can determine if this is an issue of the alike software or that this is being caused by XCP-Ng itself. I have currently no other backup solution using the CBT so we can’t compare the results there.
Also a new issue can occur, when u create a CBT snapshot during peak hours it can cause issues with coalesce as well, the snapshot is deleted and it has to be coalesced into the base disk, but when the vm is under higher i/o load it can cause an endless process of coalesce. This is basically a limitation of the XEN garbarge collection process but it is giving issues with backups from now and then.
From what i understand XOA is working on CBT as well, i believe some fundamental changes are needed to get this all working in the same way as it currently is in vmware and other products. I think it would be much better and much more reliable if this kind of coalesces are done direct at the vm level and not limited to 1 task per SR (as i understand there currenlty is).
I am curious if anybody else allready is doing tests with this and does have some feedback that could help us on this problems.
