<![CDATA[How fast are your backups]]>Hi!

I am trying to optimize my backup-speeds and I am interested in how fast you can to.

My backups and deltas are not faster than 250-280 MiB/s although my NFS-remote is much faster, everything is on 10 GbE, etc.

Before spending lots of time in trying to optimize further: Are you able to backup "faster"?

Thank you and best wishes
KPS

]]>https://xcp-ng.org/forum/topic/7754/how-fast-are-your-backupsRSS for NodeWed, 13 May 2026 09:44:22 GMTMon, 18 Sep 2023 07:17:36 GMT60<![CDATA[Reply to How fast are your backups on Fri, 10 Nov 2023 13:54:51 GMT]]>@KPS Hi,
that is some nice performance, I guess the hardware under is beefy++

Did you try if NBD increase the speed ?
You need to enable NBD on the backup network, and then enable it in the delta backup :
f556d425-d66f-41b5-bf8b-c6c769fdbcb5-image.png

We have a few tasks in the pipeline to increase performance, leveraging our knowledge of both XCP and XO

Florent

]]>https://xcp-ng.org/forum/post/67306https://xcp-ng.org/forum/post/67306Fri, 10 Nov 2023 13:54:51 GMT<![CDATA[Reply to How fast are your backups on Fri, 10 Nov 2023 09:14:13 GMT]]>Not bad 🙂 More than 2Gbit/s!

]]>https://xcp-ng.org/forum/post/67287https://xcp-ng.org/forum/post/67287Fri, 10 Nov 2023 09:14:13 GMT<![CDATA[Reply to How fast are your backups on Fri, 10 Nov 2023 09:05:09 GMT]]>I'm backing up delta's from 5 pools to a lustre filesystem with a concurrency of 10.
040a4e1f-3f00-4b93-b7c9-9f0ac5191429-image.png

]]>https://xcp-ng.org/forum/post/67286https://xcp-ng.org/forum/post/67286Fri, 10 Nov 2023 09:05:09 GMT<![CDATA[Reply to How fast are your backups on Thu, 21 Sep 2023 19:49:19 GMT]]>@rjt AWS isn't using XAPI. Only XenServer and XCP-ng are using XAPI. It's mostly unrelated to Xen performance (at least not at this point).

]]>https://xcp-ng.org/forum/post/65792https://xcp-ng.org/forum/post/65792Thu, 21 Sep 2023 19:49:19 GMT<![CDATA[Reply to How fast are your backups on Thu, 21 Sep 2023 19:28:36 GMT]]>@rjt The setting for backup concurrency within the XOA backup settings.

]]>https://xcp-ng.org/forum/post/65791https://xcp-ng.org/forum/post/65791Thu, 21 Sep 2023 19:28:36 GMT<![CDATA[Reply to How fast are your backups on Thu, 21 Sep 2023 19:02:04 GMT]]>@olivierlambert Amazon must have patched their Xen / XAPI to have much higher speeds?

]]>https://xcp-ng.org/forum/post/65790https://xcp-ng.org/forum/post/65790Thu, 21 Sep 2023 19:02:04 GMT<![CDATA[Reply to How fast are your backups on Thu, 21 Sep 2023 19:00:22 GMT]]>@planedrop what concurrency configuration settings are you referring to?

  • XOA config?

  • host network card config?

  • SMB concurrency?

]]>https://xcp-ng.org/forum/post/65789https://xcp-ng.org/forum/post/65789Thu, 21 Sep 2023 19:00:22 GMT<![CDATA[Reply to How fast are your backups on Thu, 21 Sep 2023 03:32:20 GMT]]>I'm using SMB not NFS but the max I see is around 100MiB/s, so this is REALLY good speeds (I'm also on 10GbE across the board and on a NAS that can easily take in 2000MiB/s), very impressive speeds here.

Do you have any concurrency configured though? I believe the default is 2 so that may be why you're seeing close to 2x what I am (I have it set to 1, don't really need more than that so I like to decrease overall load).

Also on pretty beefy hosts.

]]>https://xcp-ng.org/forum/post/65763https://xcp-ng.org/forum/post/65763Thu, 21 Sep 2023 03:32:20 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 08:01:13 GMT]]>This might be dangerous for your SR in real life scenario: what if a backup is taking all the SR resources? We have to find a balance 🙂 But anyway, you can always restrict the backup speed in XO, so we'll continue to find ways to speed things up!

]]>https://xcp-ng.org/forum/post/65685https://xcp-ng.org/forum/post/65685Mon, 18 Sep 2023 08:01:13 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 07:56:13 GMT]]>@olivierlambert
"Concurrency" is my next step. I first wanted to find a good setup for "singe-stream-performance", but I am always happy to test new approaches!
Plugin sounds good. Other backup-providers are taking that way, too.

250MiB/s is good for small VMs, but not for big ones. My "non-XCP-setup" can do backups at 8-9 GiB/s

]]>https://xcp-ng.org/forum/post/65684https://xcp-ng.org/forum/post/65684Mon, 18 Sep 2023 07:56:13 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 07:41:17 GMT]]>XO tasks are currently here to solve this. It's still early and not entirely displayed, but this will come soon 🙂

Also, 250-280MiB/s is already a great speed. What's your concurrency? Are you using NBD insecure or secure?

We might also have something else for you to test at some point, using a "plugin" inside the host.

]]>https://xcp-ng.org/forum/post/65683https://xcp-ng.org/forum/post/65683Mon, 18 Sep 2023 07:41:17 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 07:39:47 GMT]]>@olivierlambert
Yes, I am using NBD.
The big problem with NBD is, that there is currently no visibility about the backup-progress, as there is no "task". I know, that this is a big point for XO7, but the same is the case for "mirror-jobs".

]]>https://xcp-ng.org/forum/post/65682https://xcp-ng.org/forum/post/65682Mon, 18 Sep 2023 07:39:47 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 07:30:47 GMT]]>What matters (in terms of bottleneck) is the XAPI itself. If you backup from one single pool at this speed, I can only assume you are already doing NBD backups, right? (because VHD export can't go at this speed)

]]>https://xcp-ng.org/forum/post/65681https://xcp-ng.org/forum/post/65681Mon, 18 Sep 2023 07:30:47 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 07:27:40 GMT]]>@olivierlambert
Currently, I am only testing backups of one VM (did not parallelize, yet)

...but my hosts are quite beefy

]]>https://xcp-ng.org/forum/post/65680https://xcp-ng.org/forum/post/65680Mon, 18 Sep 2023 07:27:40 GMT<![CDATA[Reply to How fast are your backups on Mon, 18 Sep 2023 07:25:59 GMT]]>

not faster than 250-280 MiB/s

That sounds already more than a XAPI can handle. Are you doing backups on multiple pools at once?

]]>https://xcp-ng.org/forum/post/65679https://xcp-ng.org/forum/post/65679Mon, 18 Sep 2023 07:25:59 GMT