@florent As part of the documentation updates for the backup sequencing, can we get a sequence diagram of the actual backup process? Showing what gets sent to where and in what order. Both with and without the XO Proxy.

This would help clarify a lot of the issues, I think. Right now it's not clear what connections are made by the host vs XO and to what SR.

Thanks.

@olivierlambert I was thinking of tags like the anti-affinity ones but the XAPI host affinity would work as well. I'm not sure if it would work for the OP's use case, though.

@olivierlambert The LB plugin is what I'm referring to. I'd like to keep XO on the master but currently there's no way to do that. As you mentioned, if I set boot affinity there's nothing preventing it from getting load balanced to another host.

@olivierlambert There's a current workaround with NBD connections set to 1 so it's not a priority. I was just looking for a way to keep an eye on the status of any work on it so I can help test, etc.

@olivierlambert Is there a specific person we should ping or link to watch to get updates on the status?

@olivierlambert No worries. I really was just asking about the status of affinity tags.

@Greg_E Fans and hard drives eat up more power than most people expect.

@olivierlambert Was this ever added to the roadmap?

I recently had a power failure which took down my entire setup. Upon starting the pool master, I was unable to connect to XO because it had been running on another host. If I had an affinity tag I could set XO to prefer to run on the master.

This way I could more easily check the status of each thing as I brought it back online. Instead, I had to boot the entire pool because the master thought that XO was still running on the other host.

@Greg_E said in Share your HomeLabs:

There is also a cheap GPS/GNSS NTP clock at the top of the rack, just too far to show these other items.

What model and how do you have it set up? I've been tempted to add one to my setup but can't justify the cost.

Noise levels are the reason I've avoided rack mounted gear. Pizza boxes are just too loud.

@McHenry I think you may be running into an XY problem here. https://xyproblem.info/

What targets are you attempting to achieve and what do you foresee the prod environment looking like?

What is your reasoning for aging backups? What use case are you targeting?

If you provide more information about your goals and available resources then we can give you better answers. You also may want to reach out to the Vates sales team as I believe they offer design services.

@Greg_E The current storage API has some limitations which mean that you won't saturate a 10g connection with any single disk operation. Vates has a new version that should fix those issues but it's not here yet.

Even if the migrations happened at the same speed, shared storage will always be faster since it's just memory and not the disk as well. The other way to keep migrations fast is to keep your VMs as small as possible, using network shares for data as much as you can.

@olivierlambert Any progress on the attached disks and multiple NBD connections issue?

Related, should we see any performance difference related to the number of NBD connections? I went from 4 to 1 and my backups are still taking the same amount of time.

@Greg_E What do you mean by speed limits on shared storage? With a shared SR you're just moving memory from host to host and not the disk.

Automatic migration, rolling updates, etc won't work with local storage. That's a part of the reason I moved to shared storage.

I don't have any local storage configured on the new pool. The only drive in each host is a small and cheap m2 sata to run XCPng. That's part of what allows the low power usage.

I started on XCPng with AMD mini PCs for the same reasons you're looking at them. While compute and memory were fine, the lack of faster networking made me switch away from them.

If you're managing things proxmox style with all local storage, then they work well. But that means that you lose out on a lot of the nicety of XCPng. Rebooting a host means that you either wait for a migration to another node or shutdown/suspend the VMs. Using shared storage typically limits you to 1G or 2.5G speeds, which mean your VMs have half the speed of SATA.

I've since moved to using older desktops for my hosts. I can max out their memory relatively cheaply and easily add a 25G or better NIC. While the setup uses more power than the mini PCs, it's actually not bad. IIRC, typical usage was around 20W. But as I mentioned, I'm not doing any heavy compute workloads.

After changing NBD back to 1, I haven't seen any additional attached disks. However, the backup that originally succeeded with an attached disk has now failed. Odd that it will initially work with an attached disk but then fail with an attached disk.

@stormi For the last sentence, do you mean that if we use the ISO to upgrade to RC2 we won't need to use the ISO for final?

@Rudder715 You'll also want to look into how your storage is configured. Can the remote handle that many incoming backups? Both network and disk wise.

It sounds like your VMs are relatively large. Can you change them so that the data is on a network share instead of locally in the VM?

@Rudder715 How long are your backups currently taking? Have you tried adjusting the concurrency up or down and how long did those take?

Also, are you doing full or incremental backups?

Unfortunately, as soon as I bumped the NBD connections up to 2 I got an attached disk. It doesn't seem like the latest changes have fixed the issue.

Xen Orchestra, commit 74e6f

@Rudder715 You could set up a separate backup task for each pool. But keep in mind that they still all go through XO.

There have been requests to be able to set concurrency at a finer level, such as per host, but that hasn't been indicated as a feature that Vates is looking into.

Are you backing up with memory? IME, the suspend operation is what takes the longest and is most effected by concurrency.