Check the /var/log/xen* stuff (on top of my head, @stormi probably knows that better than me)

Have you took a look at Xen logs to see if there's anything? Also, anything in the IPMI? (logs)

I would run a memtest on that machine.

IIRC, everything is compressed using Brotli by default, including the full.

The latest patch planned will help to reduce the false positives doing a full. In your case, using an agent inside the VM is maybe preferable where a VM backup of such volume + those requirements isn't really a good fit.