A VM without any data on disk, passes Health Check - Bug?

jr-m4

After doing some reshuffling and having to re-create a couple of VMs. I have two with newly created disks, that don't contain any data yet.
Upon doing some checks on the backup-schedules, I notice that these two passed the latest Delta-backup Health Check. However, as I mentioned earlier. These don't have data on their disks, and hence no Guest-Tools can be installed.

"A VM without guest tools will fail its health check."

source: https://docs.xen-orchestra.com/backups-and-dr/backup-features-and-settings#backup-health-check

probain

Confirming this on a newly created VM with empty disk of 2MiB.
XO source: 635f0

poddingue

I tried this on a lab host this morning. XO's health check waits for XAPI's PV-drivers-detected flag on the restored VM.

A test VM with nothing but an empty 2 MiB disk reported that flag as true within 3 seconds when started with UEFI firmware, while the same VM under BIOS just halted and stayed false.

So if your two VMs are UEFI, that would explain the pass, though I'm only guessing at why the firmware sets it. Are they?

The docs line you quoted doesn't hold in that case, and https://xcp-ng.org/forum/topic/11066 ran into a cousin of this with a FreeBSD guest. If you need the check to prove the OS really came up, the xo-backup-health-check-xenstore tag makes it wait for a script inside the VM instead: https://docs.xen-orchestra.com/backups-and-dr/backup-features-and-settings#backup-health-check.

I think it's worth a look from @Team-XO-Backend too, since a passing check on a UEFI VM may say less than people assume.

jr-m4

@poddingue

Thanks for taking a look on your end as well.
I would like to point out that the line from the docs is quite deffinitive. If there are gotchas, such as UEFI affecting things. Then this either should be documented. Or better yet, that outlier of an edge-case probably should be addressed.

One scenario I'm envisioning is that a disk is faulty for some reason, and this goes unoticed due to the VM being rarely used. And since no backup-errors are given, this might go unoticed for quite some time. The Health Check passing is a false signal in that case.

These are my two-cents. And I confirm that this is an edge-case, and even unlikely one. But a theoretical greater than zero risk anyways.

Thank you

poddingue

@jr-m4 since my last post I ran the whole thing through XO to be sure. Two VMs, each with nothing but an empty 2 MiB disk, same delta job: the UEFI one passed its health check in a few seconds, and the BIOS one failed after the 10 minute timeout waiting for the guest tools, which is what the docs promise.

So that sentence only holds for BIOS VMs, and your case of a rarely used VM with a bad disk is where that matters. I've passed both results to the XO team, though I don't know yet whether they'll change the check or the docs. Until then, the xo-backup-health-check-xenstore tag I mentioned earlier is the way to make the check depend on something actually running inside the VM.

jr-m4

@poddingue
Thank you!