Is my backup configuration sane? (plus a bonus delta backup question)
-
Hi all, now that I actually have real storage on which I can place backups, I've enacted a new strategy for my VMs: a weekly delta backup (on Sundays), and a rolling snapshot every day except Sunday.
Given that, my first question is does this even make sense? I run both operations at 2am, so I decided to skip the snapshot on days I run a delta backup.
Second question: I noticed that even though I have delta backup configured to push backups to my NAS via NFS share, I still have the "original copy" full backups on the same disk as the VM. Is this a requirement for delta backups? I'd rather have all the backup files on the NAS, that way if the VM disk fails I have a full backup to restore elsewhere. Doesn't seem very useful to have only the deltas. I do understand that full backups take a lot of time so I guess the delta is sort of a trade-off.
Final question: assuming that I can't put the full backups on my NFS share, what's the best strategy for maintaining actual full backups? Do I need to do a monthly full backup to the NAS or something?
-
With any backup type of XO (real backup, no rolling snap), you can restore even if you lose entirely your previous hosts. As long you have your "remote" safe.
So I'm not sure to completely get your question. Everything needed is stored on the remote.
-
@olivierlambert here's where I'm confused:
When I look at the various volumes on my local storage SR (disk contained inside the host), I see entries like this for every VM.
This made me believe that the backups were actually being stored on my local storage SR. The disk usage on that SR seems to indicate that's true.
Note that the name of the backup is "weekly backup to truenas", which was a reminder to myself that these backups were supposed to be on the NAS' NFS share.
Here's the remote used for the backup:
Digging into the VM_images share on the NAS, I can see I have large VHDs on there; one particular case has a 15G VHD, and a smaller VHD with it:
root@truenas[.../749c5ba4-08e3-cbca-f677-1e5f8e14d639]# ls -lh vdis/f6170cb2-2569-41f5-aab5-d5a39c19a8b2/1755b623-302b-422b-a0a1-a76621e8b438 total 19393818 -rw-r--r-- 1 root mediacontrol 15G Mar 7 06:15 20210307T070144Z.vhd -rw-r--r-- 1 root mediacontrol 1.8G Mar 14 03:28 20210314T070006Z.vhdGiven the timestamps, it does look like these are the primary backup and the first delta file for whatever VM this represents.
So is XO placing the backups in both locations at the same time? I just don't get why they show up on my local storage as well.
-
The icon you see in the first screenshot is a snapshot (the "camera" icon).
So no, the backup is NOT on your local SR.
Backup != snapshot.
-
In order for XO to make a backup (full or incremental), it always makes a snapshot of the VM. The snapshot is then copied over to your backup location (called Remote).
For an incremental backup, XO then makes a second snapshot, which will then only contain the differences between the first snapshot and the running VM. Then it transfers that small snapshot to your Remote.
-
I think I'm starting to understand.
The weekly backup creates a new or updated snapshot, which is stored on the same volume as the VM disk. After the first delta backup operation, the smaller snapshot of the differences between the original snapshot exported to the NFS share is copied to the NFS share.
So basically: the snapshot created by the backup operation that is stored on my local disk is indeed a full snapshot recreated weekly. The main (large) snapshot on my NFS share will ultimately be older as it was only copied over when I originally created the backup operation, but the subsequent backup delta snapshots provide the "updates" to the original snapshot.
And to answer my main question: yes, I have to have at least one snapshot on the same disk as the VM for all of this to work. Now that I understand it (I think), I can live with that - I just couldn't figure out why my VMs consumed twice as much storage as their actual disks.
-
A snapshot is a "frozen" disk state, that's needed to export it. You can't export a disk while it's used by your VM (because blocks are moving!).
For delta, we export (at first) the full content of a snapshot. This will be the big VHD file on your NAS/remote storage.
Next run, we'll take another snapshot, but this time, we'll export only the diff between original and new snapshot. This is the small VHD you have on the remote. When it's done, we remove the oldest snapshot, and so on.
Also, a snapshot doesn't consume a lot of disk space. It's just a small file that will tell we need to freeze the disk content from there.
If you use ext local SR (or NFS) this will have only a small impact on the space used by your VM disks.
It's different on local LVM and LVM over iSCSI (thick pro).
-
@olivierlambert said in Is my backup configuration sane? (plus a bonus delta backup question):
If you use ext local SR (or NFS) this will have only a small impact on the space used by your VM disks.
It's different on local LVM and LVM over iSCSI (thick pro).
My local SRs are still thick provision, I need to change that but that's a different project.
Thanks for the help once again!
-
@olivierlambert Thank you for this explanation of the snapshot. However, does this snapshot decrease the performance of the virtual machine (as with LVM for example) on which it was generated or does it not affect performance at all ?
-
Short answer: depends. Longer answer: if you have more than 3 snapshots, you will start to notice the difference, mostly in read. Because each read will have to check the block origin in the whole chain. But one or two snap will not impact the performance in a very visible way.
All in all, as it's a best practice to avoid more than 3 snapshots for many reasons, don't over think things
-
@olivierlambert Interesting. So you don't recommend nightly incrementals with a weekly full as it would generate too many snapshots and could start seeing performance degradation?
-
Hmm not at all. 2 different jobs will generate 2 snapshots total, which is perfectly fine
-
@CJ on my installation, a full followed by an incremental or the reverse does not result in 2 snapshots but only one. 2 snapshots appear temporarily only when the backup is in progress.
-
You can also have a composite job, doing 2 things at once (like incremental backup and incremental replication), so it will use just one snap for both. But again, 2 and even 3 snaps are OK
-
@olivierlambert Okay, when you say 3 snapshots, you mean number of jobs, not retention? I tend to think of snapshots in ZFS terms so that may be where the disconnect is coming from.
In my example, there would be enough retention of the nightly diff snapshots so that you could restore to any day during the week. Once the weekend full backup occurs the nightly wouldn't be as needed.
I think I might be conflating snapshot and backup. I'll need to double check my cluster, but thinking about it, each nightly snapshot could be overwritten as it would have been backed up to the SR.
-
I'm talking about VM snapshots (and by transitivity, VDI snapshots). Retention is the number of backup you save on the backup repository, and it's totally unrelated to the number of snapshots when doing full or incremental backups.
