VDI_IO_ERROR during Backup

[ReeChip]

xcp-ng v7.6.0
xo v5.34.0 (fresh trial)

Hi,

it seems that I got VDI_IO_ERRORs during backup, only for disks on local zfs zpool Storage.
every other vms using my other local storage run the backup without problems.

Thanks in advance!

{
  "data": {
    "mode": "delta",
    "reportWhen": "failure"
  },
  "id": "1557150224133",
  "jobId": "b4dd9a7a-8e25-4128-8152-c8108f1119bb",
  "jobName": "test",
  "message": "backup",
  "scheduleId": "e1fbd723-a44e-4fa9-81cc-c9fb600eb221",
  "start": 1557150224133,
  "status": "failure",
  "tasks": [
    {
      "data": {
        "type": "VM",
        "id": "4894a4d3-f13a-a170-b127-1b6fdc060a2b"
      },
      "id": "1557150224135",
      "message": "Starting backup of nc. (b4dd9a7a-8e25-4128-8152-c8108f1119bb)",
      "start": 1557150224135,
      "status": "failure",
      "tasks": [
        {
          "id": "1557150224140",
          "message": "snapshot",
          "start": 1557150224140,
          "status": "success",
          "end": 1557150226672,
          "result": "12b0d945-4e7a-5aae-7913-31f0fa9b7597"
        },
        {
          "id": "1557150226675",
          "message": "add metadata to snapshot",
          "start": 1557150226675,
          "status": "success",
          "end": 1557150226687
        },
        {
          "id": "1557150226870",
          "message": "waiting for uptodate snapshot record",
          "start": 1557150226870,
          "status": "success",
          "end": 1557150227074
        },
        {
          "id": "1557150227077",
          "message": "start snapshot export",
          "start": 1557150227077,
          "status": "success",
          "end": 1557150227077
        },
        {
          "data": {
            "id": "4a10bddd-09ee-4830-a797-1ab4a145352b",
            "isFull": true,
            "type": "remote"
          },
          "id": "1557150227078",
          "message": "export",
          "start": 1557150227078,
          "status": "failure",
          "tasks": [
            {
              "id": "1557150227113",
              "message": "transfer",
              "start": 1557150227113,
              "status": "failure",
              "end": 1557150231817,
              "result": {
                "code": "VDI_IO_ERROR",
                "params": [
                  "Device I/O errors"
                ],
                "task": {
                  "uuid": "7d3f94fc-ec8a-eb55-0e0e-3827a9044ae0",
                  "name_label": "[XO] VDI Export",
                  "name_description": "nc_bekozu",
                  "allowed_operations": [],
                  "current_operations": {},
                  "created": "20190506T13:43:47Z",
                  "finished": "20190506T13:43:51Z",
                  "status": "failure",
                  "resident_on": "OpaqueRef:d05c0fb6-5654-417b-9e75-0976166bad04",
                  "progress": 1,
                  "type": "<none/>",
                  "result": "",
                  "error_info": [
                    "VDI_IO_ERROR",
                    "Device I/O errors"
                  ],
                  "other_config": {},
                  "subtask_of": "OpaqueRef:NULL",
                  "subtasks": [],
                  "backtrace": "(((process\"xapi @ akroma\")(filename ocaml/xapi/vhd_tool_wrapper.ml)(line 59))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process\"xapi @ akroma\")(filename ocaml/xapi/export_raw_vdi.ml)(line 47))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process\"xapi @ akroma\")(filename ocaml/xapi/export_raw_vdi.ml)(line 54)))"
                },
                "message": "VDI_IO_ERROR(Device I/O errors)",
                "name": "XapiError",
                "stack": "XapiError: VDI_IO_ERROR(Device I/O errors)\n    at Function.wrap (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/_XapiError.js:16:11)\n    at _default (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/_getTaskResult.js:11:28)\n    at Xapi._addRecordToCache (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:795:37)\n    at events.forEach.event (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:816:13)\n    at Array.forEach (<anonymous>)\n    at Xapi._processEvents (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:806:11)\n    at /usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:960:13\n    at Generator.next (<anonymous>)\n    at asyncGeneratorStep (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/dist/index.js:58:103)\n    at _next (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/dist/index.js:60:194)\n    at run (/usr/local/lib/node_modules/xo-server-xoa/node_modules/core-js/modules/es6.promise.js:75:22)\n    at /usr/local/lib/node_modules/xo-server-xoa/node_modules/core-js/modules/es6.promise.js:92:30\n    at flush (/usr/local/lib/node_modules/xo-server-xoa/node_modules/core-js/modules/_microtask.js:18:9)\n    at _combinedTickCallback (internal/process/next_tick.js:132:7)\n    at process._tickCallback (internal/process/next_tick.js:181:9)"
              }
            }
          ],
          "end": 1557150231817,
          "result": {
            "code": "VDI_IO_ERROR",
            "params": [
              "Device I/O errors"
            ],
            "task": {
              "uuid": "7d3f94fc-ec8a-eb55-0e0e-3827a9044ae0",
              "name_label": "[XO] VDI Export",
              "name_description": "nc_bekozu",
              "allowed_operations": [],
              "current_operations": {},
              "created": "20190506T13:43:47Z",
              "finished": "20190506T13:43:51Z",
              "status": "failure",
              "resident_on": "OpaqueRef:d05c0fb6-5654-417b-9e75-0976166bad04",
              "progress": 1,
              "type": "<none/>",
              "result": "",
              "error_info": [
                "VDI_IO_ERROR",
                "Device I/O errors"
              ],
              "other_config": {},
              "subtask_of": "OpaqueRef:NULL",
              "subtasks": [],
              "backtrace": "(((process\"xapi @ akroma\")(filename ocaml/xapi/vhd_tool_wrapper.ml)(line 59))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process\"xapi @ akroma\")(filename ocaml/xapi/export_raw_vdi.ml)(line 47))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process\"xapi @ akroma\")(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process\"xapi @ akroma\")(filename ocaml/xapi/export_raw_vdi.ml)(line 54)))"
            },
            "message": "VDI_IO_ERROR(Device I/O errors)",
            "name": "XapiError",
            "stack": "XapiError: VDI_IO_ERROR(Device I/O errors)\n    at Function.wrap (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/_XapiError.js:16:11)\n    at _default (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/_getTaskResult.js:11:28)\n    at Xapi._addRecordToCache (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:795:37)\n    at events.forEach.event (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:816:13)\n    at Array.forEach (<anonymous>)\n    at Xapi._processEvents (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:806:11)\n    at /usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/index.js:960:13\n    at Generator.next (<anonymous>)\n    at asyncGeneratorStep (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/dist/index.js:58:103)\n    at _next (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/dist/index.js:60:194)\n    at run (/usr/local/lib/node_modules/xo-server-xoa/node_modules/core-js/modules/es6.promise.js:75:22)\n    at /usr/local/lib/node_modules/xo-server-xoa/node_modules/core-js/modules/es6.promise.js:92:30\n    at flush (/usr/local/lib/node_modules/xo-server-xoa/node_modules/core-js/modules/_microtask.js:18:9)\n    at _combinedTickCallback (internal/process/next_tick.js:132:7)\n    at process._tickCallback (internal/process/next_tick.js:181:9)"
          }
        },
        {
          "id": "1557150231818",
          "message": "set snapshot.other_config[xo:backup:exported]",
          "start": 1557150231818,
          "status": "success",
          "end": 1557150231824
        }
      ],
      "end": 1557150231825
    }
  ],
  "end": 1557150231825
}

[olivierlambert]

Hi,

As stated in the official doc: https://github.com/xcp-ng/xcp/wiki/ZFS-on-XCP-ng#limitations

It is not yet possible to import, migrate or export a VDI to/from a ZFS SR.

It's mainly due to the fact that O_DIRECT isn't supported by ZFS.

[ReeChip]

@olivierlambert Thanks did not found that!

[olivierlambert]

We patched some part of SMAPIv1 to "make it work" (our patches were even merged) but VHD export is another (more) complicated story.

That's why we are working on SMAPIv3 now instead wasting resources on the "legacy" storage stack