Delta backups (NBD) stuck on "Exporting content of VDI Backups.." failing after upgrade from 8.3 RC2 to 8.3 final.

[archw]

I recently upgraded from 8.3 RC2 to 8.3 final. Ever since then I've had a terrible time getting backups...especially delta backups (going to NFS server).

1. Under tasks in XO (Xen Orchestra, commit ff118), it says "[XO] Exporting content of VDI blahblah-drive3 using NBD (on host1) 34%". I have tried rebooting the XO VM (after disabling teh backup task) as well as those that has VM with the data. The task will not go away. I've tried restarting the toolstack on every host....its still stuck.

2. If you go to the backup activity, it says interrupted and then "download logs" you get this:

{
  "data": {
    "mode": "delta",
    "reportWhen": "always"
  },
  "id": "1730477627182",
  "jobId": "7863cc0b-601e-440a-8328-1d1ef2a09972",
  "jobName": "HOST1-VM1-delta",
  "message": "backup",
  "scheduleId": "59590d78-6809-4eb8-97fe-ba938a68939e",
  "start": 1730477627182,
  "status": "interrupted",
  "infos": [
    {
      "data": {
        "vms": [
          "5e01568d-3fa9-11f8-e293-e576f15d5cdb"
        ]
      },
      "message": "vms"
    }
  ],
  "tasks": [
    {
      "data": {
        "type": "VM",
        "id": "5e01568d-3fa9-11f8-e293-e576f15d5cdb",
        "name_label": "HOST1-VM1 (20241014T050050Z)"
      },
      "id": "1730477630143",
      "message": "backup VM",
      "start": 1730477630143,
      "status": "interrupted",
      "tasks": [
        {
          "id": "1730477630162",
          "message": "clean-vm",
          "start": 1730477630162,
          "status": "success",
          "end": 1730477630497,
          "result": {
            "merge": false
          }
        },
        {
          "id": "1730477630164",
          "message": "clean-vm",
          "start": 1730477630164,
          "status": "success",
          "end": 1730477630537,
          "result": {
            "merge": false
          }
        },
        {
          "id": "1730477631151",
          "message": "snapshot",
          "start": 1730477631151,
          "status": "success",
          "end": 1730477648388,
          "result": "e252e0e8-7c26-cd1c-3365-e519829f41e3"
        },
        {
          "data": {
            "id": "0d6da7ba-fde1-4b37-927b-b56c61ce8e59",
            "isFull": true,
            "type": "remote"
          },
          "id": "1730477648389",
          "message": "export",
          "start": 1730477648389,
          "status": "interrupted",
          "tasks": [
            {
              "id": "1730477651581",
              "message": "transfer",
              "start": 1730477651581,
              "status": "interrupted"
            }
          ]
        },
        {
          "data": {
            "id": "656f82d9-5d68-4e26-a75d-b57b4cb17d5e",
            "isFull": true,
            "type": "remote"
          },
          "id": "1730477648390",
          "message": "export",
          "start": 1730477648390,
          "status": "interrupted",
          "tasks": [
            {
              "id": "1730477651599",
              "message": "transfer",
              "start": 1730477651599,
              "status": "interrupted"
            }
          ]
        }
      ],
      "infos": [
        {
          "message": "Transfer data using NBD"
        }
      ]
    }
  ]
}

[CodeMercenary]

This happens to me on 8.2 with one specific VM. It doesn't always happen but about once a week (it feels like) the backup will get stuck in Starting and it's always, or almost always, the same VM it's stuck on. I usually reboot the VM running XO and that makes the backup task change to Interrupted and then the next scheduled backup usually works.

Would be nice to know if there's something that's causing this that I can try to fix. Not a world-ending problem but it's a nuisance since it stops all future backups for that Job until it's resolved. I'm happy that all the other VMs in that job get backed up at least.