backing up to rsync.net
-
I am in a bit of a quandary. I need to store backups on rsync.net I have a couple of clients committed to that as a backup repository because they were using it for their off-site backups and other systems previously.
When I raised this issue before, it was suggested I use SSHFS for a borg backup target, which has several problems. It's not guaranteed reliable; the mount point can vanish, and you won't know it. It slows down backup significantly because of the de-duplication and archive validation process.
The structure of SRs is pretty opaque. Looking at the current backup remote file system, It looks like, under xo-vm-backups, a series of directories look like UUIDs. I assume each UUID refers to a specific VM. Under the UUID directories, there are snapshots of each VM (xva), a hash for validating xva file, and a JSON file with parameters for the VM. I suspect that if you have a multi-day set of full backups, there will be multiple trios of files under the same UUID. Looks like I found a case where the backup images are cleaned up properly
I'm assuming (and this may be a bad bet) that the backup data set starting with the UUID is a logically self-contained definition of a preserved VM or VMs. If I place the backup data set. starting with the UUID in an unused remote target, it should be visible via the XO backup restore interface
Would placing a copy of a Borg archive in an unused backup remote target create a restorable image? Assuming my technique works, the next question is how to know that today's backup is done so I can start the remote copy.
Thanks for your help!
-
Putting @florent in the loop
-
hi @ricky222
the structure of the backup repository is documented here https://github.com/vatesfr/xen-orchestra/tree/master/%40xen-orchestra/backups/docs/VM backups .
Checking if a backup is done/running can be done with the XO rest api from https://xen-orchestra.com/docs/restapi.html
You can go confidently with rsync/rclone for the full backups ( the xva files ). For restoration you can even use the import VM function of XO or xo-cli to directly import the xva. If you rsync data back from rsync to a local backup repository, you must delete the local cache.json.gsz to ensure XO see the downloaded backup (xva + hash + json)
The caveat is for the incremental backup and the merging. In XO we are merging older incremental backups, that means modifying older files. This process is offloaded to another node process which does not appear for now in the api. Making a transfer during a merge may lead to an invalid backup on target.
Do you have enough informations ? Also, it could be great if you share your setup when you have a working/tested setup with rsync
Regards