Backup and the replication - Functioning/Scale

florent

@fcgo it goes from xapi (= pool/hosts) to xapi , and xapi to remote as of now, you can't chain proxy .

xapi call uses https, that means that replication accross sites uses https . If you do backup, and your backup is in the same site as your NFS, then the xapi<-> proxy path is in https, and the proxy <-> NFS will be local

that is one of the strength ( and complexity ) of xcp-ng infra : everything go through an API ( the xapi ) nobody access host/pool data directly from the outside

fcgo

@florent Hi,

Thank you.

And during backup which host is working hard during the vdi export to XOA (or to the proxy) : is it the host which contains the backed up VM, or is the master also doing some work (other than control channel calls) ? Through which host the network flow is going through ?

florent

@fcgo if the storage is shared : the export is done by one of the host of the pool
If the storage is not the export is done by the host with the storage . Same for the host receiving the data

The command channel, as you said, is always the master to the xoa (and eventually xo-proxy)

So for a replication :

[source SR] => source host =https export call=> xoa / xo-proxy =https import call=> target host => [target SR]

if the xoa is running on the host doing an export, it does not use the physical network
the network use between the host and SR is dependent on the storage used

fcgo

@florent "the export is done by one of the host of the pool"
How is this host selected ? Is it the master one, the one hosting the VM, or the less busy one in case of shared storage ?

fcgo

@florent did you have the the time to check my last question related to how the host (doing the export) is selected ?

Br,

florent

@fcgo AFAIK the host is selected as random maybe @andriy.sultanov know more

andriy.sultanov

@florent @fcgo the host that was queried (with host_ip/export_raw_vdi) will do the export from the shared SR, and will redirect the query to the host that can see the SR otherwise. So it's up to the XO/other orchestrators if they want to distribute the load here.

Vanny

In some third-party backup solutions (for example Vinchin and similar enterprise platforms), proxy pool management and job distribution are fully automated, which helps when managing hundreds or thousands of VMs across sites.

olivierlambert

@vanny you can say you are working for Vinchin, I prefer when it's clearly told

florent

thanks Andryi
We us round robin when using NBD , but to be fair, it does not change the performance a lot in most of the case. The concurrency settings ( multiple connection to the same file ) is helping when there is a high latency between XO and the host.

SO , @fcgo if you have thousand of VMs , you should enable NBD it will consume less resource on the DOM0 and XO , and it will be spread on all the possible hosts.