Feature Request: Metadata replication like XenCenter DR

flakpyro

One cool feature i was reading about with XenCenter is its "Disaster Recovery" functionality.

https://docs.xenserver.com/en-us/xencenter/current-release/dr.html

From reading about how it works it sounds like it replicates the VM metadata from one pool to another, it then relies on you to use your storage providers replication to replicate the actual SRs from one site to another.

This would be great to have in XO for those of us with with DR sites with matching storage array vendors at each location. Even 2 TrueNAS servers could support this with ZFS replication. Array replication is usually very network efficient, does not create additional load on the server side of the environment and does not rely on VM snapshots. It would need to be made clear that this does not replicate any VM data itself, only the metadata and that its up to you to get the data from Array A to Array B.

Thoughts?

olivierlambert

XS DR mechanism is storing the metadata in an SR, as a special VDI. However, you can already backup that with XO in a backup repo (BR) which is probably better. Since it's a very small amount of data, I would prefer to rely on XO to send that content. We've seen various issues on storing this data as a special VDI, especially on parsing the data (causing various security issues in the past)

flakpyro

@olivierlambert I do already do a metadata backup with XO to a backup repo, however i was under the impression it was only for restoring back to the same or identical pool with the same hardware? (Identical number of hosts, nic layout etc) or is it possible to restore it to a non identical pool?

IE: Pool 1 (Production pool) 5 host cluster, 5 SRs - Pure Storage SAN
Pool 2 (DR pool) 3 Host cluster with same Pure Storage SAN

Using storage replication (Pure, Nimble, Dell, TrueNAS etc) to send all the SRs from Pool 1 to Pool 2 and have XCP-NG on pool 2 see the VMs and be able to power them on in case of a DR scenario. As it stands now if i were to do this XO would see all the VHDs on the replicated SRs at DR but would not know what VMs they belong to nor their VM configs.

Perhaps something like a VM inventory metadata only option?

The reason this is so interesting to me is it gets around having to run multiple snapshots on VMs (1 for backups, one for replication) that grow during the day and require coalescing during backup / replication runs.

I know storage integration is on your long term roadmap and this is very much wishful thinking right now but imagine XO being able to kick off the storage replication job on an array like Pure's similar to what Veeam does with a number of providers. That would be slick!

olivierlambert

With CBT, the growing snapshot isn't a problem anymore, because we do not keep snapshot data anymore. So I think this will solve your functional requirement, right?

Also, metadata replication is exactly like saving pool metadata: the difference is the place you save it (in a BR or a SR). With the exact same limitations (similar hosts etc.). IDK the detailed restore option on XenCenter about the metadata (eg just restoring a SR and not the entire metadata?).

flakpyro

@olivierlambert Yes, once CBT with Snapshot purge is production ready that will totally solve our problem! Perhaps its better to just wait for that.

I wondered if they were doing something like a partial restore of the metadata as well.