Backup / Migration Performance

KPS

I think, we are mixing up some topics

2TB limitation
This is not nice, but can be mostly worked around with LVM/storage-spaces inside the VM with multiple VDIs. 2-10 TB are possible, but file-level restore is not.
backup-speed
backup-speed went up within the last updates, NBD, etc. It could be better, but as backups can be parallelized, this is mostly good
restore-speed
As restores are mostly "one-VM-at-a-time"-jobs, this should be faster. Things like "instant-recover" are missing, so you have to wait for the full copy.
migration-speed
No progress on fast networks, improvements on slow-networks with compression. This should really be better compared to other hypervisors

john.c

This post is deleted!

olivierlambert

Restore speed: you can now enjoy diff restore if you still have the original VM. Otherwise, CR can provide you the instant restore you need. But even with that, if you want a better solution, we could spawn an NFS share in XO directly and mount it as a temporary SR. My fear is that will be really slow, and you'll need to live migrate it out after. Potentially creating more problem than fixing it. CR is the right tool for instant restore

nikade

@olivierlambert said in Backup / Migration Performance:

Restore speed: you can now enjoy diff restore if you still have the original VM. Otherwise, CR can provide you the instant restore you need. But even with that, if you want a better solution, we could spawn an NFS share in XO directly and mount it as a temporary SR. My fear is that will be really slow, and you'll need to live migrate it out after. Potentially creating more problem than fixing it. CR is the right tool for instant restore

With Veeam Instant Recovery the VM is booted off the Veeam storage and then it is migrated to your esxi cluster/host, works pretty well if your Veeam respository has fast storage.

olivierlambert

Yes, as usual "if you have X or Y", but we have so many different infrastructure, I'm already feeling the number of tickets "migration can't be done because I'm writing more on the temporary restore SR than it can be migrated"

KPS

@olivierlambert
That is my current workaround: instead of an NFS server, i did install an additional (licensed) XCP-ng-host, that is ONLY used as CR-target.
Not optimal, but - of course - as fast as instant recovery.

But migrating the VM to the prod cluster is limited by the migration speed of XCP-ng

nikade

@KPS said in Backup / Migration Performance:

@olivierlambert
That is my current workaround: instead of an NFS server, i did install an additional (licensed) XCP-ng-host, that is ONLY used as CR-target.
Not optimal, but - of course - as fast as instant recovery.

But migrating the VM to the prod cluster is limited by the migration speed of XCP-ng

This is probably the best solution tbh, it also offers you the flexibility to "scale" up with more hosts if you'd need more for a faster recovery of many VM's.
One note tho, if im correct you're only allowed to do 4 concurrent migrations, but as long as you can start the VM's fast on the CR-host you could queue the migrations.

KPS

@nikade
I think, this can be handled. The downsides are the inefficient way to save the VMs, which can perhaps be minimized with ZFS storage for some compression, but it is working.

planedrop

@KPS Regarding the 2TiB limitation, it'll definitely be nice when we have SMAPIv3 so we can go over this, but it's worth noting that IMO no VMs should be larger than this anyway. Generally speaking if you need that kind of space it'd be better to just use a NAS/iSCSI setup. Something like TrueNAS can delivery that at high speed, and then handle it's own backups and replication of it.

I know most probably already know this, and all environments are different (I manage one that requires a 7TiB local disk, at least for the time being, plan is to migrate it to a NAS once the software vendor supports it), but it's worth noting anytime I see the 2TiB limit come up, ideally it should be architected around so the VMs are nimble.

I do something similar w/ a pretty massive SMB share and TrueNAS can back this up at whatever speed the WAN can handle, in my case 2 gigabits and it'll maintain that 2 gigabit upload for 8+ hours without slowing down. (and I'm confident even 10 gigabit would be possible with this box)

olivierlambert

@planedrop said in Backup / Migration Performance:

Regarding the 2TiB limitation, it'll definitely be nice when we have SMAPIv3 so we can go over this, but it's worth noting that IMO no VMs should be larger than this anyway.

This. Really, this. Even if SMAPIv1 limit was 4 or 8TiB, with the current export or migration speed, that would have been pretty bad anyway. We should get both a lot faster export/migration, not just getting larger drives. So right now, it's more a protection against more problems (but yeah, obviously, we need to improve all the areas at once, which is a challenge).

andrewperry

@john-c I am seeing an option for Migration compression in XO, under Xen settings on the Advanced tab for a Pool of 8.2.1 servers. Haven't tried it though.

olivierlambert

This is only for memory, not disks.

nikade

@planedrop said in Backup / Migration Performance:

@KPS Regarding the 2TiB limitation, it'll definitely be nice when we have SMAPIv3 so we can go over this, but it's worth noting that IMO no VMs should be larger than this anyway. Generally speaking if you need that kind of space it'd be better to just use a NAS/iSCSI setup. Something like TrueNAS can delivery that at high speed, and then handle it's own backups and replication of it.

I know most probably already know this, and all environments are different (I manage one that requires a 7TiB local disk, at least for the time being, plan is to migrate it to a NAS once the software vendor supports it), but it's worth noting anytime I see the 2TiB limit come up, ideally it should be architected around so the VMs are nimble.

I do something similar w/ a pretty massive SMB share and TrueNAS can back this up at whatever speed the WAN can handle, in my case 2 gigabits and it'll maintain that 2 gigabit upload for 8+ hours without slowing down. (and I'm confident even 10 gigabit would be possible with this box)

We have 1 exception and that is for Windows file servers which is backing our DFS.
Except from those we dont allow VM's larger than 1Tb and if they're that big we do not back them up because it usually breaks and cause all kinds of problems.