Potential bug with Windows VM backup: "Body Timeout Error"
-
aaaand it happens again.
-
Sorry it's back.
Just to line things up for whoever picks this up: the original full-backup Body Timeout was tracked in https://github.com/xapi-project/xen-api/pull/6786 and shipped with the March 2026 8.3 updates, so what you're seeing on metadata backups might be a cousin rather than the same bug.
From Olivier's earlier note the error just means XO got nothing from the pool master for 5 minutes mid-transfer, so the useful question is what the master was doing during that silence.
The fact that a toolstack restart clears it for a while and then it returns is probably itself a clue.
Could you grab the complete logs from one failed metadata run (the ask pierrebrunet made earlier), plus a rough note of whether it lines up with the master being busy or briefly unreachable?
I'm out of my depth on the internals, but that's the piece the team will want.
-
Due to the fact that this is the second XO release in a row that broke backups I do wonder: are all backup job types being tested via automated unit tests? Are those tests being run multiple times in order to catch issues that do not occur during every single test run? There must be a reason for QA not catching these kind of issues…
-
@MajorP93 I am currently working on this issue but to answer you quickly, yes we have automatic tests, unit and integration tests (you can see them in *.integ.mjs and *.test.mjs files, and @xen-orchestra/qatest folder), a long QA to validate at each release. We add a regression test at every bug we encounter.
The issue is we could not reproduce your exact issue yet. -
I'm picking this one up alongside the other Body Timeout thread (9002), since they look (to me, at least) like the same underlying issue.
Pierre from the XO team is on it and has tested metadata backups on the XO side without hitting it, so the current thinking is it may be coming from the XCP-ng side rather than XO.
What would move it along fastest is a/var/log/xensource.logfrom the pool master covering one failed run's window, so if anyone here can reproduce and grab that slice, it'd really help line the timeout up against what xapi was doing.
I know it's especially frustrating when the thing that breaks is your backups, thanks for bearing with us while we chase it.
A mention to @Team-Storage might help route the storage-layer angle. -
Had a failed job this night (commit 3bc70)
Logged in to my host and saw time was 30-40 sec's off.
The time in XOCE VM was OK
Had a few power outage yesterday.
after starting up the host again. I guess thecmos clockwas a little slow.
Could that be the culprit?
Adjusted time and ran a backup manually and it worked fine.As requested below
here are the logs
https://paste.vates.tech/?314d454cd7c52552#4ocXwQksjayBbvUxxxddJRwAQ8YJ3KqvKspJGak8yHF4 -
This post is deleted! -
This post is deleted! -
This post is deleted! -
This post is deleted! -
This post is deleted! -
@ph7 As a suggestion: it would improve readibility if you paste your logs on Vates official pastebin tool: https://paste.vates.tech/ and share the links here instead of posting the logs in multiple messages.
-
@MajorP93
I'l do that, didn't know about it.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login