XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    10GB backup with XOA (XCP to windows NFS)

    Scheduled Pinned Locked Moved Xen Orchestra
    30 Posts 5 Posters 5.8k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • olivierlambertO Offline
      olivierlambert Vates 🪐 Co-Founder CEO
      last edited by

      Note that you can scale up with concurrency (see https://xen-orchestra.com/docs/backups.html#backup-concurrency).

      Also, if you have different pools, you could use XOA Proxies (alternatively, multiple XO from sources but it's less elegant) so you can accumulate all pool speed to get to a decent global speed.

      1 Reply Last reply Reply Quote 0
      • nikadeN Offline
        nikade Top contributor
        last edited by

        We see about the same speed, just below 1Gbit/s no matter how many concurrent backups and what not. The bursting is probably due to buffering in the node-process.

        I think this is a limitation in XAPI just like @olivierlambert wrote earlier and nothing much you can do in XO to speed it up.

        We've been trying to speed things up for a long time, we ended up using the Delta-backup and Continious Replication which are both incremental. This allows us to backup all our VM's over night to our NFS-machines.

        1 Reply Last reply Reply Quote 0
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          We'll get rid of potential stream buffer limitation in a relatively short time frame, but again, I'm pretty sure the export speed on the host is the real bottleneck.

          As we are also working on XCP-ng storage performance in parallel, I'm confident we'll be able to improve that 🙂

          nikadeN 1 Reply Last reply Reply Quote 1
          • A Offline
            andrea.mazzoni
            last edited by

            I confirm what @nikade say. Even if i use a 10Gb and set the concurrent backup to 10, the speed will be always the same.
            If we use proxy XOA or similar idea i think we can speed the backup but not to a point we would like.
            We usually make a differential backup, but for us (67 VM = 300 GB transfer and 300 GB merged - diff backup, XOA take from 4 to 6 Hours).

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              I'm confident we can improve this for you before the end of the year 🙂 @julien-f is working hard on that!

              1 Reply Last reply Reply Quote 0
              • nikadeN Offline
                nikade Top contributor @olivierlambert
                last edited by

                @olivierlambert said in 10GB backup with XOA (XCP to windows NFS):

                We'll get rid of potential stream buffer limitation in a relatively short time frame, but again, I'm pretty sure the export speed on the host is the real bottleneck.

                As we are also working on XCP-ng storage performance in parallel, I'm confident we'll be able to improve that 🙂

                I think the storage performance-improvements will contribute to increasing the export-storage as well as working on the XAPI export/import performance.
                I know the XCP team is aware of this issue and I am also confident that it will be a priority 🙂

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  First thing on the calendar will be to use the XO proxy code in threads directly in XOA for "normal" backups. It will allow more XOA usage and we'll see if it helps (at least it will help to scale).

                  nikadeN 1 Reply Last reply Reply Quote 0
                  • nikadeN Offline
                    nikade Top contributor @olivierlambert
                    last edited by

                    @olivierlambert said in 10GB backup with XOA (XCP to windows NFS):

                    First thing on the calendar will be to use the XO proxy code in threads directly in XOA for "normal" backups. It will allow more XOA usage and we'll see if it helps (at least it will help to scale).

                    Sounds like a step in the right direction, any ETA?

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by olivierlambert

                      With summer in the middle, hard to tell. But I have the feeling end of Q3 is doable 🙂 At least in a specific release channel

                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        I hope it's fine for you @julien-f 😛

                        1 Reply Last reply Reply Quote 0
                        • julien-fJ Offline
                          julien-f Vates 🪐 Co-Founder XO Team
                          last edited by

                          That sounds feasible indeed 🙂

                          1 Reply Last reply Reply Quote 1
                          • First post
                            Last post