Mirror Backup Testing, So Far Awesome!!
-
Today's blog post mentioned Mirror Backups, a new way to backup an entire Backup Repository instead of the VMs themsevles: https://xen-orchestra.com/blog/xen-orchestra-5-83/?utm_campaign=mail_5.83&utm_term=logo
I decided to go ahead and update a XOA instance to Latest instead of Stable to test this, and so far it's been amazing.
For some reference, my XOA only has 8GB of RAM and is maxing it out with these backups, so the speeds I mention may not be the fastest it can go. This server has a 10GbE link to an SMB share for backups and then a 2GbE link WAN to backup to Backblaze.
I am seeing speeds in the 200Mbps range (megabits, not BYTES) per second, which is over double what I was seeing with doing individual VM backups to Backblaze B2, this is super awesome and doing it with full backups means there are no delays due to merge times either.
Just wanted to start a post to say how awesome this feature is, I love the idea of it and thank you Vates team for making this happen, it's a very unique way to do things and is excellent!
-
Thanks a lot for your feedback @planedrop
I'm sure @florent will be delighted to read this
-
@olivierlambert No problem, what an awesome feature! I'll do more testing too with a beefier XOA install with more CPU and RAM and see if performance goes even higher, but 200 Mbps is enough for even pretty large VM setups to backup overnight!
-
I suppose that most of the bottleneck was coming from XAPI exports, with Mirror Backup your are doing a transfer from BR to BR (or remote to remote). So without that bottleneck, you have a lot of better perfs. We spotted that behavior during the feature dev.
-
@olivierlambert Makes total sense, this is super awesome.
I did want to ask, after running for about an hour speeds dropped off (only a little though) but I'm seeing a huge spike in CPU usage on XOA along with higher RX than TX (whereas before RX and TX were about the same and CPU wasn't working very hard), is this normal? Just curious since it's a new feature trying to provide as much feedback as I can.
I'll still test this all again with some more test VMs once I boost XOA up to a more powerful VM.
-
@planedrop As a quick update to this, I can also verify the Rx traffic is all coming from the SMB share of the NAS, so it's not something else causing higher Rx traffic than Tx.
-
Really hard to tell, we hope to have better details in future releases to investigate what's going on exactly and where potential bottlenecks are
-
@olivierlambert Sounds good, I'll do more testing too. If there is anything you all in specific want me to test let me know!
-
I'm adding to @planedrop, this feature is awesome!
This is the first time I see using all NIC transfer features, both ends of the NFS tags are on two QNAP SATA RAID 5 disks (not SSD) connected at 1Gb/s and the transfer was steady at 104MB/s .
I used the function to synchronize the two QNAPs, so on the first boot I used one NFS(A)->NFS(B) job and then a second NFS(B)->NFS(A) job and now I am perfectly synchronized, without concerns which snapshot the bacukps are from.
I stopped backing up VMs to two NFS targets simultaneously, this new feature is more performant.
If possible, in future versions it wouldn't be a bad idea to see the progress of the copy inside the TASKS !
Thanks for the great job!
-
Yes, it's planned to use XO tasks (cf our latest blogpost release about it)
@florent will be happy to hear your feedback, so pinging him
-
@ptavanti yeah .
the progress bar inside the task is near, most of the preliminary work is done, we will soon be able to plug the new task system, probably this summer.
-
This is all so cool, love to see a platform under this level of development.
