Spread the backup load across servers
-
I have a backups server which also hosts XOA which has a 10GBit NIC. I have around 30 xcp-ng servers which need to be backed up to that server - these all have 1GBit NICs.
It would be good if when doing a Delta Backup XO would, where possible, choose a different physical server for each of the concurrent backups it is doing. Thus reducing load on the physical host and meaning that I can get a much greater throughput as the backups server is capable of accepting the output from 10x servers.
Instead I've just started a backup with a concurrency of 10 and it started all of those backups on the same server.
-
This is only possible if you have local SRs on your hosts. In a shared SR case, it's always the master that doing all the work.
-
Hi,
This isn't servers in a pool. They are all single node xcp-ng installs connected seperately to XO. All using their own local storage.I'm using smart backup to back up from some 23 individual servers. Yet when I started the backup all 10 backup jobs started backing up the same server.
I'm not sure if it is by design or not, just a suggestion that it would be better that it chose different servers whereever possible.
-
In this case, you might want to use XO proxies. Each proxy will handle n pool, so the backup transfer speed can be spread around those.
However, it will work better if you have a remote that's fast enough (otherwise, it might become the new bottleneck at some point).
Proxy in beta will come very soon (it's a matter of days)
-
It does perhaps just require a bit of logic in the backups to choose another xcp-ng server if there's already a backup going on with one of them. Proxies are good but perhaps overkill for solving this issue.
-
For the moment I've had to create several backup jobs instead of one big backup it's e.g. servers01-06, 07-12, 13-18. Which isn't ideal as it's harder to manage, but is what it is!
