Backup fails with "Body Timeout Error", "all targets have failed, step: writer.run()"

[bogikornel]

@poddingue httpInactivityTimeout changed the only thing for me, I received the email half an hour later that the metadata could not be saved.

[marcoi]

looks like im getting the xo config backup body timeout error as well.
Xen Orchestra, commit a94eb
Master, commit 9c6d1

  "result": {
    "message": "backup task failed with undefined error",
    "name": "Error",
    "stack": "Error: backup task failed with undefined error\n    at forwardResult (file:///opt/xo/xo-builds/xen-orchestra-202608030700/packages/xo-server/src/_handleBackupLog.mjs:37:25)\n    at handleBackupLog (file:///opt/xo/xo-builds/xen-orchestra-202608030700/packages/xo-server/src/_handleBackupLog.mjs:68:12)\n    at onTaskUpdate (file:///opt/xo/xo-builds/xen-orchestra-202608030700/packages/xo-server/src/xo-mixins/metadata-backups.mjs:133:13)\n    at onTaskUpdate (file:///opt/xo/xo-builds/xen-orchestra-202608030700/@xen-orchestra/mixins/Tasks.mjs:205:23)\n    at Task.onProgress [as _onProgress] (/opt/xo/xo-builds/xen-orchestra-202608030700/@vates/task/combineEvents.js:61:5)\n    at Task.#emit (/opt/xo/xo-builds/xen-orchestra-202608030700/@vates/task/index.js:157:10)\n    at Task.#end (/opt/xo/xo-builds/xen-orchestra-202608030700/@vates/task/index.js:168:15)\n    at Task.run (/opt/xo/xo-builds/xen-orchestra-202608030700/@vates/task/index.js:190:16)\n    at Jobs.runJob (file:///opt/xo/xo-builds/xen-orchestra-202608030700/packages/xo-server/src/xo-mixins/jobs/index.mjs:297:7)\n    at Jobs.runJobSequence (file:///opt/xo/xo-builds/xen-orchestra-202608030700/packages/xo-server/src/xo-mixins/jobs/index.mjs:339:7)\n    at Task.runInside (/opt/xo/xo-builds/xen-orchestra-202608030700/@vates/task/index.js:204:22)\n    at Task.run (/opt/xo/xo-builds/xen-orchestra-202608030700/@vates/task/index.js:188:20)\n    at Api.#callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202608030700/packages/xo-server/src/xo-mixins/api.mjs:475:18)"
  }

[JB]

@christopher-petzel Olá Chris!
Yes, this is a local XCP-ng management network and there are no other backups simultaneous.

[JB]

@poddingue Olá! The failure has a fixed duration of 5 minutes.

[acebmxer]

Just got a failure today but with a different pool.

    "message": "backup task failed with undefined error",
    "name": "Error",
    "stack": "Error: backup task failed with undefined error\n    at forwardResult (file:///usr/local/lib/node_modules/xo-server/src/_handleBackupLog.mjs:37:25)\n    at handleBackupLog (file:///usr/local/lib/node_modules/xo-server/src/_handleBackupLog.mjs:68:12)\n    at onTaskUpdate (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/metadata-backups.mjs:117:13)\n    at onTaskUpdate (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/mixins/Tasks.mjs:205:23)\n    at onLogFct (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/combineEvents.js:61:5)\n    at metadataBackup._executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/metadata-backups.mjs:122:11)\n    at Jobs.runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:297:7)\n    at file:///usr/local/lib/node_modules/xo-server/src/api/schedule.mjs:84:9\n    at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:204:22)\n    at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:188:20)\n    at Xo.runSequence (file:///usr/local/lib/node_modules/xo-server/src/api/schedule.mjs:73:3)\n    at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:204:22)\n    at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:188:20)\n    at Api.#callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:475:18)"
  }
}

[JB]

Every day I update XO-CE hoping this error gets fixed, but so far, no solution!

[olivierlambert]

Is there a way in private message to give access to the XCP-ng host and XO VM in question OR to get all logs from 2 machines somewhere? 5min is clearly a timeout error.

[acebmxer]

@olivierlambert

Mine has not done it since last post but i have opened a support ticket along with tunnel being opened - Ticket#7762089

[JB]

@olivierlambert Yes, I can grant it.

[marcoi]

looks like mine also just started working.

[JB]

Master, commit be960

[olivierlambert]

@JB I'd like to test a different way of investigating the problem. So I won't do it for many people, just send me a private message with a way to SSH to your host and XOA, promise it's not a scam

[JB]

@marcoi Try doing it three times, and the problem comes back.

[marcoi]

@JB seems okay after a few back to back runs.