XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Change the size of the backup file

    Scheduled Pinned Locked Moved Backup
    11 Posts 3 Posters 313 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • a573A
      a573
      last edited by

      Hi ALL,

      How do I change the size of the XO backup file?

      I want to increase the size of the backup file that XO creates from 2MB to about 200MB.

      I back up from XO to QNAP. Replicating from QNAP to cloud at low frequency.
      Because of the large number of files, the number of API calls is wasted. The purpose is to increase the file size and reduce the number of files.

      ls -plah fe96dbe9-0d78-50a9-4624-7ea0f9ccb1aa/vdis/19217d2e-d234-4f2c-85c1-ecf49cb2389f/045c97cf-ab7f-4915-ab3e-a79165674dbc/data/83945dbd-9580-4d01-92c6-a1cf6e12891a.vhd/blocks/0
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 649
-rw-r--r--  1 admin administrators 1.8K 2024-06-19 22:54 65
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 650
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 651
-rw-r--r--  1 admin administrators 2.0M 2024-06-19 22:54 652
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 653
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 654
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 655
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 656
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 657
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 658
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 659
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 660
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 661
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 662
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 663
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 664
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 665
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 666
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 667
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 668
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 669
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 670
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 671
-rw-r--r--  1 admin administrators 2.1M 2024-06-19 22:54 672
      ForzaF 1 Reply Last reply Reply Quote 0
      • olivierlambertO
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        2MiB is not a random value, it's the size of a VHD block. That's how we can just rename or remove blocks on the fly to reduce the burden of doing a coalesce on the backup repo.

        If we group blocks with larger size, it means we'll need to get far bigger quantity of non-relevant blocks to merge (100 more with 200MiB file size).

        I'm not sure to understand exactly your issue in the first place, can you be more explicit about the "why" of your question?

        a573A 1 Reply Last reply Reply Quote 0
        • a573A
          a573 @olivierlambert
          last edited by

          @olivierlambert said in Change the size of the backup file:

          My problem is that I get too many "api list file names called" calls to Backblaze.

          My structure is a little complicated. Incremental backup from XO to QNAP. Using QNAP's cloud sync function, I'm slowly backing up my XO backup files to Backblaze.

          The my goal was to get the backup done fast locally, and the cloud transfer done slowly with limited internet bandwidth.

          At this time, we noticed that the number of file list calls was abnormally high and Backblaze charges a lot during the cloud transfer.

          I think this is because QNAP's cloud sync function keeps checking the number of files on the cloud side. I thought it would be solved by changing the block size that XO outputs.

          If you back up to the cloud using an S3-compatible backup as a remote server on the XO, does that significantly reduce the number of file references? I haven't experimented yet.

          1 Reply Last reply Reply Quote 0
          • olivierlambertO
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Question for @florent but I would advise to use mirror backup with S3 from your "fast" local backup repository, it's exactly the use case.

            a573A 1 Reply Last reply Reply Quote 0
            • a573A
              a573 @olivierlambert
              last edited by

              @olivierlambert said in Change the size of the backup file:

              Thank you olivierlambert and florent.

              I'm experimenting with direct backups from XO to Backblaze.
              I will report the results of the experiment again.

              a573A 1 Reply Last reply Reply Quote 0
              • a573A
                a573 @a573
                last edited by

                I did a cloud backup of 52 VMs from XO to Backlaze.
                The number of files in the cloud reached about 620,000 files and the capacity reached 580GB.
                After the initial upload was complete, about 30GB/day of Delta was sent.

                The bill in August was $5.65.
                The XO is likely to provide cost-effective backup.

                I hope this helps you.

                1 Reply Last reply Reply Quote 0
                • olivierlambertO
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  Thank you @a573 for your feedback! This is helpful and could assist other community member to decide to go that route or not.

                  Next release (on Monday) will also add backup chaining, so you can do a mirror backup to S3 just after finishing your regular on-site backup 🙂

                  1 Reply Last reply Reply Quote 0
                  • ForzaF
                    Forza
                    last edited by

                    Isn't it dangerous to copy the backup files directly? How can you ensure that you have a complete set?

                    Using XO to mirror the backups seems a safer option.

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by

                      @Forza I'm not sure it is what @a573 said. He simply said he made backup with XO to S3 directly (ie without a mirror job but a regular backup job)

                      1 Reply Last reply Reply Quote 0
                      • ForzaF
                        Forza @a573
                        last edited by

                        I was referring to this:

                        @a573 said in Change the size of the backup file:

                        Replicating from QNAP to cloud at low frequency.

                        I read it as copying the files directly from storage.

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO
                          olivierlambert Vates 🪐 Co-Founder CEO
                          last edited by olivierlambert

                          I don't know if it's still what he does 🤔

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post