XO Backup Failing - Possibly to due Local Disks and loopback issues?

[MichaelCropper]

Physical Disks Setup;

• 1x RAID Array - for VMs
• 1x RAID Array - for Backups (technically this is 1x JBOD at the minute)

Virtualisation Setup;

• Physical Server
• • VM with XO installed (1TB disk space)
• • • With Backup RAID Array Mounted to File System (10TB size of physical disk, configured with a 1 TB partition which is mounted).
• • VM X
• • VM Y
• • VM Z

Ultimately the backup of the VM with XO installed is failing due to lack of disk space on the RAID Array for Backups.

So I deleted all of the backups on the RAID Array for Backups.

Then re-run the backup job that backs up all VMs. They all worked, with the exception of the VM with XO installed, which failed again due to lack of disk space.

I have a sneaky feeling that the backup of the VM with XO installed is trying to backup the main disk for XO plus the mounted disk for the Backups, hence creating an exponentially growing problem.

I gave this approach a test off the back of this thread, https://xcp-ng.org/forum/topic/6274/backups-to-specific-storage-repository-on-local-possible?_=1664570110880

Is this one of the limitations for this approach? Or have I simply done something dumb and not configured things properly?

cc. @Tristis-Oris as you seem to have worked with this setup before so may have come across the solution to this?

[Tristis Oris]

@MichaelCropper did you put [NOBAK] tag for backup disk?
Next you can check a size of each snapshot via restore menu, and how it compare with real data size on disk.
And i supposed that you doing delta backup?

for example depend of data, delta can be very different.
2nd (zabbix) always have such big delta, even if real data just +1-2Gb\day. But also rotation and a lot other things. 1st just classic web service with database.

[MichaelCropper]

@Tristis-Oris At the minute I'm just playing with full backups. The delta backups are next to dig into.

Re. the [NOBAK] tag you mention. No I've not done that. I've not come across any documentation for how to go about implementing this, or even come across the terminology yet. Do you have any handy reference guides that I can take a look at for how to implement that?

I'll do some Googling on this topic in the mean time. It sounds promising.

Re. Zabbix. Where does this fit into the mix. XCP-ng and XO provide a lot of monitoring info out of the box anyhow. I've come across Zabbix and seen it mentioned a few times online, but I've yet to find the time to dig into the details and have a play and understand where it fits architecturally etc.

[Tristis Oris]

@MichaelCropper i showed that in previous post https://xcp-ng.org/forum/topic/6274/backups-to-specific-storage-repository-on-local-possible/2?_=1664605297176
otherwise you backup the backup disk.

[MichaelCropper]

@Tristis-Oris So you did. I must have missed that on the first few reads

I've added this in now so the backup disk has [NOBAK] at the start of the disk name (I've not rebooted the XO VM or full server server yet, not sure it that could be causing an issue), and I'm getting this error which I can't quite figure out yet

Job canceled to protect the VDI chain

I've isolated the XO VM so that it is running now in it's own independent Full Backup to make sure that none of the other VMs are getting tangled up.

I need to have a read up on this error, so just posting this question while I get back to the docs/Google on what this means and how to solve etc.

[Tristis Oris]

@MichaelCropper not sure about it. Lack of space also can be cause of issues. Or any another problem. Maybe try to remove old backups and start new chain.

[Danp]

@MichaelCropper said in XO Backup Failing - Possibly to due Local Disks and loopback issues?:

Job canceled to protect the VDI chain

This usually indicates that there are VDIs that haven't coalesced. Check the "unhealthy VDIs" section of Dashboard > Health.