How Best to Achieve Higher Transfer Speeds for Backup Jobs
-
Good-day everyone,
Just looking for some advice here. I've been playing around with backups in my lab, as part of my Proof-of-Concept in an Air-Gapped environment. I have a 10Gbps Storage Network between the hosts and the NAS, but the Management network is at 1Gbps. To that end, I am noticing that XOA seems to be the bottleneck during any backup job.
As I'm sure there's a perfectly valid reason for why Vates went with this architecture of running all backup jobs through XO, I'm not questioning that. However, I want to ask how others are managing to achieve higher transfer speeds on their backup jobs. Is it as simple as putting the Management Network on a 10Gbps link?
Looking forward to your responses, thanks.
-
@kagbasi-ngc
Yes, I put XOA on a 10G (or faster) connection and make sure it has enough resources to process the backups. -
@lawrencesystems Thanks for the response, I suspected that would be the remedy. I take it that means the entire management network is 10Gbps or faster?
... and make sure it has enough resources to process the backups.
Although, I'm curious about what you meant by the last part of your response. Does this mean I ought not accept the default CPU, RAM, and Disk sizes of XOA? I suppose one could customize these parameters when building XOCE (from sources), but aren't we supposed to be using the official XOA as-is?
-
@kagbasi-ngc
Yes, but of course speed varies by setup. I have 10G in my lab since it's so common and many of our clients are running 25G setups and bonded to 50G.You can add more CPU if you need more processing power for XOA for running more concurrent backups. If XOA were to ship with a high memory and or CPU setting by default it would not work on some setup that don't have the resources. Defaults work fine but for people who need more there are options.
-
@lawrencesystems Makes sense.
I always assumed that since XOA was delivered as a service, that one had to leave the settings as-is. Good to know that they allow customization. But for an air-gapped environment, that presents some challenges for when upgrade time comes, and a new appliance is delivered. One has to remember to put the customizations in place on the new appliance before cutting to it. Not a deal breaker, just an annoyance factor I guess...lol.
Thanks again for chiming in.
-
If you update the XOA it will not change the Memory or CPU setting of VM.
-
@lawrencesystems Yes sir I agree, as that is my experience with XOCE. But with XOA (Airgapped), in my PoC Lab, I've received my first upgrade and it was a completely new appliance. So I'm now figuring out how to handle that. I did reach out to Vates support about this and got a response that they are looking into perhaps doing the XO updates kinda like how they handle it now for XCP-ng (i.e., allow us to download the entire repo offline and then do the upgrade that way).
I'm sure you've dealt with many customers who are in my kind of environment, where security/compliance requirements are strict.
-
@kagbasi-ngc said in How Best to Achieve Higher Transfer Speeds for Backup Jobs:
@lawrencesystems Yes sir I agree, as that is my experience with XOCE. But with XOA (Airgapped), in my PoC Lab, I've received my first upgrade and it was a completely new appliance. So I'm now figuring out how to handle that. I did reach out to Vates support about this and got a response that they are looking into perhaps doing the XO updates kinda like how they handle it now for XCP-ng (i.e., allow us to download the entire repo offline and then do the upgrade that way).
I'm sure you've dealt with many customers who are in my kind of environment, where security/compliance requirements are strict.
Wouldn't you export the configuration from your existing XOA and import it into the new one before it's finalized and "onsite"? That would get you a full working config, from which you may only have to adjust the cpu/ram.
-
@DustinB Yep, that's precisely what I did (which went very smoothly, by the way, with the exception of a few plugins which didn't startup automatically so had to start them), then discovered, the config + metadata backup does not include the appliance logs and audit logs - so my security folks are on me about that.
I have a ticket into support now inquiring into what the best approach would be to move these two logs from one appliance to another.
-
@kagbasi-ngc I think you'd have to configure some kind of log shipping ahead of time, I don't know that you'll be able to migrate the logs from one XO to another to keep things consistent.
-
@DustinB Yeah, my thoughts exactly. Would be nice if XO supported logging to a remote syslog server, like XCP-ng does.
I haven't done this yet, but I'm gonna go through /var/log and see if perhaps those logs that are visible in the UI are actually being written to disk. If so, then I could just ship them off and import them into the new appliance.
-
@kagbasi-ngc
Until there is a solution to the log export, You could keep the old XOA with the network disabled -
@ph7 Yep, that's what I've done. Actually, I've powered off the prior XOA instance. Not ideal, but a workable solution.
