-
not faster than 250-280 MiB/s
That sounds already more than a XAPI can handle. Are you doing backups on multiple pools at once?
-
olivierlambert
Currently, I am only testing backups of one VM (did not parallelize, yet)...but my hosts are quite beefy
-
What matters (in terms of bottleneck) is the XAPI itself. If you backup from one single pool at this speed, I can only assume you are already doing NBD backups, right? (because VHD export can't go at this speed)
-
olivierlambert
Yes, I am using NBD.
The big problem with NBD is, that there is currently no visibility about the backup-progress, as there is no "task". I know, that this is a big point for XO7, but the same is the case for "mirror-jobs". -
XO tasks are currently here to solve this. It's still early and not entirely displayed, but this will come soon
Also, 250-280MiB/s is already a great speed. What's your concurrency? Are you using NBD insecure or secure?
We might also have something else for you to test at some point, using a "plugin" inside the host.
-
olivierlambert
"Concurrency" is my next step. I first wanted to find a good setup for "singe-stream-performance", but I am always happy to test new approaches!
Plugin sounds good. Other backup-providers are taking that way, too.250MiB/s is good for small VMs, but not for big ones. My "non-XCP-setup" can do backups at 8-9 GiB/s
-
This might be dangerous for your SR in real life scenario: what if a backup is taking all the SR resources? We have to find a balance
But anyway, you can always restrict the backup speed in XO, so we'll continue to find ways to speed things up! -
I'm using SMB not NFS but the max I see is around 100MiB/s, so this is REALLY good speeds (I'm also on 10GbE across the board and on a NAS that can easily take in 2000MiB/s), very impressive speeds here.
Do you have any concurrency configured though? I believe the default is 2 so that may be why you're seeing close to 2x what I am (I have it set to 1, don't really need more than that so I like to decrease overall load).
Also on pretty beefy hosts.
-
planedrop what concurrency configuration settings are you referring to?
-
XOA config?
-
host network card config?
-
SMB concurrency?
-
-
olivierlambert Amazon must have patched their Xen / XAPI to have much higher speeds?
-
rjt The setting for backup concurrency within the XOA backup settings.
-
rjt AWS isn't using XAPI. Only XenServer and XCP-ng are using XAPI. It's mostly unrelated to Xen performance (at least not at this point).
-
O olivierlambert moved this topic from Xen Orchestra on
-
I'm backing up delta's from 5 pools to a lustre filesystem with a concurrency of 10.
-
Not bad
More than 2Gbit/s! -
KPS Hi,
that is some nice performance, I guess the hardware under is beefy++Did you try if NBD increase the speed ?
You need to enable NBD on the backup network, and then enable it in the delta backup :
We have a few tasks in the pipeline to increase performance, leveraging our knowledge of both XCP and XO
Florent
