@rkelley Thank you for doing all the work for us.

When there is a proper patch, I be happy to test it in our environment as well.

@olivierlambert I have swapped to normal full backups for now.

I have non-emergency need to recover at least one VM and have a couple options on how to do so since the data appears intact.

In all fairness to xcp-ng, the hardware we use is a mix of older machines (it is not a funded enterprise setup). I would be willing to provide your team specific hardware details among other things but I will be slow to gather info.

I will try out a delta backup of a bios VM as soon as I find the time. My only theory, which I have little evidence or ability to test, is that I have triggered a bug in the uefi firmware used by the VM. I can at least narrow down a bit more by comparing uefi versus legacy.

@olivierlambert so I booted into Finnix, and was able to mount the root LVM and verify that the filesystem is intact and the files I expect to see are all there. The chroot environment is functional and I don't see anything worry about.

I was able to mount and rebuild the /boot through efi/EFI paths, I have not yet bothered checking file hashes or anything forensic, but I am seeing all the contents in the backups that I see in the running VM from which this backup originated.

One thing I have found, is that only deltas are failing to restore in bootable form. Full backups are fine. I tried to restore the periodic full snapshots within the delta cycles, but those aren't any better than restoring a later delta.

I will be switching to full backups only until I know why delta is broken in my environment.

@olivierlambert

I got a VM to boot into an Ubuntu live CD and could see the disk and its respective partitions, and LVM setup.

I posted here originally

I have both Delta backups and full backups failing to restore properly from an NFS source. I have them throwing into a UEFI rescue mode. I have Debian Buster images doing this and Centos 8 with xo-server 5.81.2 and XCP-NG 8.2.0.

My original VM's are using Secure Boot with UEFI, I have learned that secure boot isnt supported by XCP-NG, and I have not tested non-UEFI VM's and their backups

I do not know what I am missing, nothing about the restore fails, they just will not boot.

@stormi

Well that's news to me.

I just tested disabling secure boot on a VM and rebooting it. It made no difference, which is what I'd expect if it was unsupported.

I guess we can rule it out as a factor in why our backups don't work.

@ninghe I may be having a similar issue.

I have both Delta backups and full backups failing to restore properly from an NFS source. I have them throwing into a UEFI rescue mode. I have Debian Buster images doing this and Centos 8 with xo-server 5.81.2 and XCP-NG 8.2.0.

My original VM's are using Secure Boot with UEFI, I have not yet tested non-Secure Boot or non-UEFI VM's and their backups.

I do not know what I am missing, nothing about the restore fails, they just will not boot.