Remove backup bottleneck: please report
For me, switching to http seems to have increased the backup speed by about 3.5 times as well. However I feel like it's still pretty slow. I went from a reported transfer speed average of 3.26 MiB/s to 11.59 MiB/s across 16 VMs running on a pool consisting of two 8.1 XCP-ng hypervisors and a 5.61 Xen Orchestra Community VM backing up to an NFS share on a Synology RackStation over a 10 Gb/s storage network. When testing the speed with which I am able to write to the NFS share from the XO VM with dd it's averaging write speeds of about 95 MB/s.
Anything else I can do to try and figure out where the next bottleneck is?
That's slow indeed
It's not trivial. A lot of things could cause that. However, you can test by:
- changing backup concurrency to find a sweet spot (one VM, then 2 VM etc.)
- give more memory to the dom0
- give more memory to the XO VM
- having an SSD on the backup storage
- having an SSD on the storage repository
I will for sure attempt and see how fast it is to backup only one VM at a time... actually I guess I don't know the answer to this but when XO does a delta backup to an NFS remote does it attempt to transfer all VM snapshots at the same time or does it do it sequentially one after the other?
Thank you so much for this info, I can't believe I didn't see this in all of my research. With the information from that document I take it back. I am getting full bandwidth since 12 machines get backed up at a time. The individual VM transfer speed in the backup reports was a bit misleading to me since I didn't realize that 12 are uploaded to storage at a time.
My initial finding still stands though, going http completely removed the one bottleneck that we had in our backup infrastructure and made it 3.5 times faster then before.
To get the complete speed, just take a look at XOA network stats while the backup is happening in… Xen Orchestra
I'm very interested in trying this out.
Except I can't find how I should change the master URL?
In XO when I click on the disconnect button in the pool view I receive an error.
ChuckNorrison last edited by ChuckNorrison
@CIT same here. I was able to switch to http at settings -> server. Remove the server and add with
Current Backup Speed without change on XCP-ng 8.2:
Duration: 28 minutes Size: 43.85 GiB Speed: 26.86 MiB/s
Speed should be limited to WD Red or my Gigabit Bandwidth to my NFS. The meassured speed is far below. x3 is expected (~80-100 MiB/s)
@ChuckNorrison Ah, thnx for updating... I edited my servers, hope to see some speed improvements!