8.3 beta, crashed Windows11 VM when trying to snapshot with memory
-
Tried to do a snaphot with memory and got this, VM turned stopped cold:
vm.snapshot { "id": "71679fda-1c99-127a-609f-f09edef98abe", "saveMemory": true } { "code": "INTERNAL_ERROR", "params": [ "xenopsd internal error: Device_common.QMP_Error(15, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"An IO error has occurred\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-000123-15\\\"}\")" ], "task": { "uuid": "f61415bc-7682-e230-d567-2364c4efff5a", "name_label": "Async.VM.checkpoint", "name_description": "", "allowed_operations": [], "current_operations": {}, "created": "20240409T18:10:01Z", "finished": "20240409T18:10:04Z", "status": "failure", "resident_on": "OpaqueRef:476ca210-c084-24b9-7d3d-88df3ebd0bd8", "progress": 1, "type": "<none/>", "result": "", "error_info": [ "INTERNAL_ERROR", "xenopsd internal error: Device_common.QMP_Error(15, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"An IO error has occurred\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-000123-15\\\"}\")" ], "other_config": {}, "subtask_of": "OpaqueRef:NULL", "subtasks": [], "backtrace": "(((process xenopsd-xc)(filename ocaml/xenopsd/xc/device_common.ml)(line 563))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename ocaml/xenopsd/xc/device.ml)(line 3204))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename ocaml/xenopsd/xc/device.ml)(line 3614))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 1677))((process xenopsd-xc)(filename ocaml/xenopsd/xc/emu_manager.ml)(line 239))((process xenopsd-xc)(filename ocaml/xenopsd/xc/emu_manager.ml)(line 244))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 1807))((process xenopsd-xc)(filename result.ml)(line 23))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 1800))((process xenopsd-xc)(filename ocaml/xenopsd/xc/xenops_server_xen.ml)(line 2575))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 1828))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 1836))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2381))((process xenopsd-xc)(filename list.ml)(line 121))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2374))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2547))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 1828))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 1836))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3190))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3200))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3220))((process xenopsd-xc)(filename ocaml/xapi-idl/lib/task_server.ml)(line 196))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3352))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3913))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3361))((process xapi)(filename lib/backtrace.ml)(line 210))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3367))((process xapi)(filename ocaml/xapi/xapi_vm_snapshot.ml)(line 121))((process xapi)(filename ocaml/xapi/xapi_vm_snapshot.ml)(line 122))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 143))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 38))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 1702))((process xapi)(filename ocaml/xapi/rbac.ml)(line 189))((process xapi)(filename ocaml/xapi/rbac.ml)(line 198))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 75)))" }, "message": "INTERNAL_ERROR(xenopsd internal error: Device_common.QMP_Error(15, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"An IO error has occurred\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-000123-15\\\"}\"))", "name": "XapiError", "stack": "XapiError: INTERNAL_ERROR(xenopsd internal error: Device_common.QMP_Error(15, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"An IO error has occurred\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-000123-15\\\"}\")) at Function.wrap (file:///opt/xo/xo-builds/xen-orchestra-202404091252/packages/xen-api/_XapiError.mjs:16:12) at default (file:///opt/xo/xo-builds/xen-orchestra-202404091252/packages/xen-api/_getTaskResult.mjs:11:29) at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202404091252/packages/xen-api/index.mjs:1029:24) at file:///opt/xo/xo-builds/xen-orchestra-202404091252/packages/xen-api/index.mjs:1063:14 at Array.forEach (<anonymous>) at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202404091252/packages/xen-api/index.mjs:1053:12) at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202404091252/packages/xen-api/index.mjs:1226:14)" }
-
Are you passing through any PCI or USB devices to this VM?
-
@Danp None.
-
Which version of the VM Tools are you running? I assume the VM has a VTPM, correct?
Does a standard snapshot work? What about a snapshot with memory for a non-Windows 11 VM?
-
@Danp Standard snapshot works. Also after rebooting the VM after the crash the snapshot with memory worked.
Tools are:VM has a VTPM.
-
Interesting. Maybe @stormi will know more about the reason for the crash.
-
@Danp Actually this is not the 1st time this happened as I now remember... Can't recall on which VM though. I 9 out of 10 only do standard snapshots
-
Does it always crash, or just sometimes?
-
@stormi I don't do this frequently or regularly. Since I reported it crashed 1 out of 4 (tests).
-
We'll try to reproduce. If you can upload the tarball created by
xen-bugtool -y
somewhere private, we can try to have a look at it. -
@stormi said in 8.3 beta, crashed Windows11 VM when trying to snapshot with memory:
xen-bugtool -y
I did run the command on the host it occurred. How do I get the file out? -
With
scp
(or a windows equivalent such as WinSCP maybe? I seldom use Windows) -
@stormi Worked with sftp. (On macos).
I'll submit link via PM
-
Actually, I'm not sure checkpoints are supported, or snapshots of a suspended VM which as a vTPM: https://docs.xenserver.com/en-us/xenserver/8/vms/windows.html#constraints
I had already noted we need to investigate what this actually means, but this might explain why it crashes sometimes.
-
@stormi "You cannot snapshot or checkpoint a suspended VM with a vTPM attached."
The VM was running.... And we need snapshots for delta backups.So let me know what you find.
-
@manilx I agree about snapshots. I don't know how we can checkpoint a VM without suspending it (I think it shortly suspends it and them resumes it), so I have trouble understanding XenServer's statement.