XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Why does the backup use snapshots and not CBT

    Scheduled Pinned Locked Moved Backup
    50 Posts 6 Posters 11.2k Views 7 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R Offline
      rtjdamen @olivierlambert
      last edited by

      @olivierlambert ok would be great if we can get this working, will be a huge improvement.

      D 1 Reply Last reply Reply Quote 0
      • D Offline
        DustinB @rtjdamen
        last edited by

        @rtjdamen said in Why does the backup use snapshots and not CBT:

        @olivierlambert ok would be great if we can get this working, will be a huge improvement.

        I concur, I've gotten my backup window down to around an hour, but still the amount of delta in each backup nightly is like 50GB, which doesn't seem feasible given the amount of delta is actually occurring.

        R 1 Reply Last reply Reply Quote 0
        • R Offline
          rtjdamen @DustinB
          last edited by

          Are there any 3th party backup solutions on xcp that support this?

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Online
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Reading the doc is a good first step: https://docs.xcp-ng.org/project/ecosystem/

            R 1 Reply Last reply Reply Quote 0
            • R Offline
              rtjdamen @olivierlambert
              last edited by

              @olivierlambert indeed, what about xenserver compatible backups solutions, can they work on xcp as well? I see we could maybe use agent based backups inside the vms for the larger ones, i will test this with one vm, but need a proper solution in the future, i believe if we can get this to work in xoa that would be the best solution.

              D 1 Reply Last reply Reply Quote 0
              • R Offline
                rfx77
                last edited by

                We are using CommVault to backup our XenServer infrastructure. it also does not use CBT but it does not keep the snapshots after the backup. they do some internal tracking. Works very well.

                But clearly a CBT solution would be the best way. there is a github repo where you can see how it can be implemented with some simple python scripts.

                For now keeping snapshots of all our VMs the whole day to do delta backups is not an option. we are using a iSCSI Storage and dont want to buy that extra SSDs.

                Greetings, Franz

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Online
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  I suppose there's an agent to install in the Dom0 with CommVault for the tracking? I don't see another solution outside CBT to do it.

                  R 1 Reply Last reply Reply Quote 0
                  • D Offline
                    DustinB @rtjdamen
                    last edited by DustinB

                    @rtjdamen said in Why does the backup use snapshots and not CBT:

                    @olivierlambert indeed, what about xenserver compatible backups solutions, can they work on xcp as well? I see we could maybe use agent based backups inside the vms for the larger ones, i will test this with one vm, but need a proper solution in the future, i believe if we can get this to work in xoa that would be the best solution.

                    You should look into something that is Application Aware (SQL etc) for these systems as they are likely sensitive to interruptions.

                    I don't know of any opensource agent based Application Aware backup solutions that would work, (edit) easily enough and not cause other headaches. Of course you would likely only be backing up your database with one of these separate tools and then use XO to restore the guest as a whole and restore whatever database there-after.

                    Off the top of my head a few backup solutions I've seen and used

                    • Bacula
                    • UrBackup
                    • Duplicati
                    • BackupPC

                    I know Bacula is/can be application aware in their paid version but I'm not sure if the opensource edition offers it.

                    1 Reply Last reply Reply Quote 0
                    • R Offline
                      rfx77 @olivierlambert
                      last edited by rfx77

                      @olivierlambert
                      No. A Proxy-VM (Windows) has to be deployed in the Pool. Nothing in Dom0.

                      To my understanding, Commvault mounts the Snapshot-VDIs to the Proxy-VM and does the backup this way. They keep track of the block changes on the Proxy-VM.

                      In the simplest Setup the CommVault Server-VM itself can be the Proxy-VM. The only requirement is that it has access to all SRs where the VMs to backup reside on.

                      You can read the commvault docs on the internet. it is described very well.

                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Online
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        @rfx77 can you provide a link please? Might worth a look 🙂

                        R R 2 Replies Last reply Reply Quote 0
                        • R Offline
                          rtjdamen @olivierlambert
                          last edited by

                          @olivierlambert i am currently looking at Quadric Software and storware, Quadric Software is doing the cbt backup exactly as needed, i got there description on the workflow yesterday

                          *When you run an ABD/hypervisor backup, the A3 will snapshot the VM for the duration of the backup data acquisition phase. After the datea is collected, the snapshot is immediately deleted.​

                          In the case of CBT backups, the snapshot is deleted with the special XAPI command vdi-data-destroy, which frees data associated with the snapshot, but retains metadata, including the block allocation for the disk. This allows us to perform a delta backup the next time, with extremely negligible storage overhead.*

                          This is what we need

                          florentF 1 Reply Last reply Reply Quote 0
                          • olivierlambertO Online
                            olivierlambert Vates 🪐 Co-Founder CEO
                            last edited by

                            @rtjdamen we are aware of that, @florent already did a complete PoC with it.

                            1 Reply Last reply Reply Quote 0
                            • R Offline
                              rfx77 @olivierlambert
                              last edited by

                              @olivierlambert

                              https://documentation.commvault.com/2024/essential/citrix_hypervisor_xenserver.html

                              Its in the price-range of veeam.

                              1 Reply Last reply Reply Quote 0
                              • olivierlambertO Online
                                olivierlambert Vates 🪐 Co-Founder CEO
                                last edited by

                                lol even XCP-ng is listed officially

                                R 1 Reply Last reply Reply Quote 1
                                • R Offline
                                  rtjdamen @olivierlambert
                                  last edited by

                                  @olivierlambert yes, maybe the documentation on xcp website can be updated to list them as well.

                                  What is your experience with these backup tools, do u know of companies using them?

                                  R R 2 Replies Last reply Reply Quote 0
                                  • olivierlambertO Online
                                    olivierlambert Vates 🪐 Co-Founder CEO
                                    last edited by

                                    No experience but we'll contact them and reference them officially 🙂

                                    R 1 Reply Last reply Reply Quote 0
                                    • R Offline
                                      rtjdamen @olivierlambert
                                      last edited by

                                      @olivierlambert ok i will share my experience as well.

                                      1 Reply Last reply Reply Quote 0
                                      • R Offline
                                        rfx77 @rtjdamen
                                        last edited by rfx77

                                        @rtjdamen

                                        We are using it exclusively on all our customers. Enterprise level. Not as easy to configure as Veeam. More complex but much more capable.

                                        We are mostly doing VM-Backups for DR and additional agent-based backups because the Application-Agents are very good and support a lot of different Applications and do not cost additional licenses when used in VMs.

                                        I am not a big fan of AppAware Backups which only rely on the VM-Backups.

                                        But to be honest I would really like to see a CBT enabled XOA Backup for many tiny customers which cannot afford a CommVault installation.

                                        For now i coded my own backup which does Xen Vm Snapshots and does raw backups of the VDIs to kopia (via stdin), which does the dedup to S3. So we have dedup and save a lot of space. we have to read the whole VDI for the incrementals anyway.

                                        1 Reply Last reply Reply Quote 0
                                        • R Offline
                                          rtjdamen @rtjdamen
                                          last edited by

                                          I have my configuration for quadricsoftware up and running, so far so good, works as promised. i will let it run for a few weeks and let u know the results. the CBT is a real game changer so far!

                                          R 1 Reply Last reply Reply Quote 0
                                          • R Offline
                                            rtjdamen @rtjdamen
                                            last edited by rtjdamen

                                            I wanted to share my first experiences so far, for 1 single disk vm it works perfect, cbt is created and left only for metadata till the next backup.

                                            For vms with 2 disks i run into some strange behavior, after the bacup job is see one of the 2 disks snapshot (random) at the health page as not connected to a vm and when i run the backup job again it gives an error "source and target are unrelated", the orphan vdi section.

                                            7478ac16-7466-4b90-9843-4a29f06038f7-image.png

                                            not shure if this is an issue on XCP side or on a bug inside there software.

                                            as i cannot seem to reproduce this issue on single disk vm's i feel like this is an issue with multiple disks. i am contacting there support to see if this is a bug on there end or if this is an issue inside XCP itself.

                                            Keep u all posted on the progess of this tests.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post