Potential bug with Windows VM backup: "Body Timeout Error"

olivierlambert

archw

@Hex
Ditto
https://xcp-ng.org/forum/topic/10532/backup-failed-with-body-timeout-error/8

I have it happen on almost every large backup. I had to give up. In the VMs that would not backup, I moved to delta backups.

FWIW, here were my results:

Regular backup to TrueNas, “compression” set to “Zstd”: backup fails.
Regular backup to TrueNas, “compression” set to “disabled”: backup is successful.
Regular backup to vanilla Ubuntu test VM, “compression” set to “Zstd”: backup is successful.
Delta backup to TrueNas: backup is successful.

lsouai-vates

@Hex Hello, thanks for your bug report. I am informing Xen Orchestra team on the subject, and keep you in touch when I have some answers that can help you.
Have a good day.

lsouai-vates

@Hex I have some answers from XO Team that I hope will be able to help you:

"During full backup, XO dowloads an XVA file from the XCP-ng/XenServer host.
This error mean that it did not get an answer after a configured delay (5 mins by default)
There can be an XO issue somewhere but the problem lies most likely on the host's side..."

"The timeout is not the bug, only the failsafe to not lock the process. "

olivierlambert

This means we probably need to check with the XCP-ng team. But just before @Hex , can you try to reproduce the issue with xe CLI. If you do, then it's clearly it's not XO's fault.

archw

@olivierlambert
Since I'm having the same issue, can I give that suggestion a shot? If so, how do you do it from the command line (with xe CLI)?

olivierlambert

xe vm-export filename=export_filename compress=true (for gzip, otherwise use zstd for Zstd compression)

Note: make sure to mount a share with enough space and not export directly on the dom0 root, otherwise you'll fill it.

Hex

@olivierlambert
I have tested several scenarios on 1TB test VM.
Mounted share from same storage where backups from XO are stored.

Export of shutdown VM with zstd - succeeded
Export of snapshot while VM is running with zstd - succeeded
Export of snapshot while VM is running with gzip - succeeded

olivierlambert

That's a very interesting result It means the problem is either an interaction between XO and XAPI, or on XO's side, but not simply an XCP-ng issue as we could have thought initially

Can you check if the XVA file seems to work when importing it? (it case xe fails silently). Use xe vm-import.

Hex

@olivierlambert
Tested on one of the previous exports and import with " xe vm-import" was successful. VM Windows OS starts normally.

olivierlambert

So maybe there's a timeout that's too long for XO. Adding @florent and/or @julien-f in the loop.