Why does the backup use snapshots and not CBT
-
@rtjdamen said in Why does the backup use snapshots and not CBT:
@olivierlambert ok would be great if we can get this working, will be a huge improvement.
I concur, I've gotten my backup window down to around an hour, but still the amount of delta in each backup nightly is like 50GB, which doesn't seem feasible given the amount of delta is actually occurring.
-
Are there any 3th party backup solutions on xcp that support this?
-
Reading the doc is a good first step: https://docs.xcp-ng.org/project/ecosystem/
-
@olivierlambert indeed, what about xenserver compatible backups solutions, can they work on xcp as well? I see we could maybe use agent based backups inside the vms for the larger ones, i will test this with one vm, but need a proper solution in the future, i believe if we can get this to work in xoa that would be the best solution.
-
We are using CommVault to backup our XenServer infrastructure. it also does not use CBT but it does not keep the snapshots after the backup. they do some internal tracking. Works very well.
But clearly a CBT solution would be the best way. there is a github repo where you can see how it can be implemented with some simple python scripts.
For now keeping snapshots of all our VMs the whole day to do delta backups is not an option. we are using a iSCSI Storage and dont want to buy that extra SSDs.
Greetings, Franz
-
I suppose there's an agent to install in the Dom0 with CommVault for the tracking? I don't see another solution outside CBT to do it.
-
@rtjdamen said in Why does the backup use snapshots and not CBT:
@olivierlambert indeed, what about xenserver compatible backups solutions, can they work on xcp as well? I see we could maybe use agent based backups inside the vms for the larger ones, i will test this with one vm, but need a proper solution in the future, i believe if we can get this to work in xoa that would be the best solution.
You should look into something that is Application Aware (SQL etc) for these systems as they are likely sensitive to interruptions.
I don't know of any opensource agent based Application Aware backup solutions that would work, (edit) easily enough and not cause other headaches. Of course you would likely only be backing up your database with one of these separate tools and then use XO to restore the guest as a whole and restore whatever database there-after.
Off the top of my head a few backup solutions I've seen and used
- Bacula
- UrBackup
- Duplicati
- BackupPC
I know Bacula is/can be application aware in their paid version but I'm not sure if the opensource edition offers it.
-
@olivierlambert
No. A Proxy-VM (Windows) has to be deployed in the Pool. Nothing in Dom0.To my understanding, Commvault mounts the Snapshot-VDIs to the Proxy-VM and does the backup this way. They keep track of the block changes on the Proxy-VM.
In the simplest Setup the CommVault Server-VM itself can be the Proxy-VM. The only requirement is that it has access to all SRs where the VMs to backup reside on.
You can read the commvault docs on the internet. it is described very well.
-
@rfx77 can you provide a link please? Might worth a look
-
@olivierlambert i am currently looking at Quadric Software and storware, Quadric Software is doing the cbt backup exactly as needed, i got there description on the workflow yesterday
*When you run an ABD/hypervisor backup, the A3 will snapshot the VM for the duration of the backup data acquisition phase. After the datea is collected, the snapshot is immediately deleted.
In the case of CBT backups, the snapshot is deleted with the special XAPI command vdi-data-destroy, which frees data associated with the snapshot, but retains metadata, including the block allocation for the disk. This allows us to perform a delta backup the next time, with extremely negligible storage overhead.*
This is what we need
-
-
https://documentation.commvault.com/2024/essential/citrix_hypervisor_xenserver.html
Its in the price-range of veeam.
-
lol even XCP-ng is listed officially
-
@olivierlambert yes, maybe the documentation on xcp website can be updated to list them as well.
What is your experience with these backup tools, do u know of companies using them?
-
No experience but we'll contact them and reference them officially
-
@olivierlambert ok i will share my experience as well.
-
We are using it exclusively on all our customers. Enterprise level. Not as easy to configure as Veeam. More complex but much more capable.
We are mostly doing VM-Backups for DR and additional agent-based backups because the Application-Agents are very good and support a lot of different Applications and do not cost additional licenses when used in VMs.
I am not a big fan of AppAware Backups which only rely on the VM-Backups.
But to be honest I would really like to see a CBT enabled XOA Backup for many tiny customers which cannot afford a CommVault installation.
For now i coded my own backup which does Xen Vm Snapshots and does raw backups of the VDIs to kopia (via stdin), which does the dedup to S3. So we have dedup and save a lot of space. we have to read the whole VDI for the incrementals anyway.
-
I have my configuration for quadricsoftware up and running, so far so good, works as promised. i will let it run for a few weeks and let u know the results. the CBT is a real game changer so far!
-
I wanted to share my first experiences so far, for 1 single disk vm it works perfect, cbt is created and left only for metadata till the next backup.
For vms with 2 disks i run into some strange behavior, after the bacup job is see one of the 2 disks snapshot (random) at the health page as not connected to a vm and when i run the backup job again it gives an error "source and target are unrelated", the orphan vdi section.
not shure if this is an issue on XCP side or on a bug inside there software.
as i cannot seem to reproduce this issue on single disk vm's i feel like this is an issue with multiple disks. i am contacting there support to see if this is a bug on there end or if this is an issue inside XCP itself.
Keep u all posted on the progess of this tests.
-
@rtjdamen said in Why does the backup use snapshots and not CBT:
@olivierlambert i am currently looking at Quadric Software and storware, Quadric Software is doing the cbt backup exactly as needed, i got there description on the workflow yesterday
*When you run an ABD/hypervisor backup, the A3 will snapshot the VM for the duration of the backup data acquisition phase. After the datea is collected, the snapshot is immediately deleted.
In the case of CBT backups, the snapshot is deleted with the special XAPI command vdi-data-destroy, which frees data associated with the snapshot, but retains metadata, including the block allocation for the disk. This allows us to perform a delta backup the next time, with extremely negligible storage overhead.*
This is what we need
we did a poc of it
we still need a snapshot ( or to stop the VM), bu at least we won't keep it
the hard part is that we need to change the way we store metadata : today it is stored at the VM snapshot level, but this snapshot is deleted after a CBT enabled backup
Hopefully it will be done this year
