Unhealthy VDIs
-
Not sure if this is related to qcow2 beta. I have one linux vm in this format. In my dashboard is see the following error. The backup job is skipping this vm to protect the vdi-chain. If i shut down the vm the unhealthy vdis go away. After next backup its back again.
xo-ce = commit 042c9
Following - https://docs.xen-orchestra.com/backup_troubleshooting#vdi-chain-protection
I get the following errors. It goes on for a long while.
[06:54 xcp-ng-qhfpcnmb ~]# cat /var/log/SMlog | grep -i exception Nov 7 03:08:31 xcp-ng-qhfpcnmb SM: [3534788] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3534799, status=95 Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534785] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3534799, status=95"] Nov 7 03:08:32 xcp-ng-qhfpcnmb SM: [3534596] EXCEPTION while coalescing: Child process exited with error Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] raise util.SMException("Child process exited with error") Nov 7 03:08:34 xcp-ng-qhfpcnmb SM: [3534943] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3534948, status=95 Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534940] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3534948, status=95"] Nov 7 03:08:35 xcp-ng-qhfpcnmb SM: [3534596] EXCEPTION while coalescing: Child process exited with error Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] raise util.SMException("Child process exited with error") Nov 7 03:09:01 xcp-ng-qhfpcnmb SM: [3535379] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3535390, status=95 Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535376] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3535390, status=95"] Nov 7 03:09:02 xcp-ng-qhfpcnmb SM: [3535200] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] raise util.SMException("Child process exited with error") Nov 7 03:09:04 xcp-ng-qhfpcnmb SM: [3535511] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3535527, status=95 Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535508] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3535527, status=95"] Nov 7 03:09:05 xcp-ng-qhfpcnmb SM: [3535200] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] raise util.SMException("Child process exited with error") Nov 7 03:09:31 xcp-ng-qhfpcnmb SM: [3535960] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3535971, status=95 Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535957] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3535971, status=95"] Nov 7 03:09:32 xcp-ng-qhfpcnmb SM: [3535793] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] raise util.SMException("Child process exited with error") Nov 7 03:09:34 xcp-ng-qhfpcnmb SM: [3536092] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3536097, status=95 Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3536089] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3536097, status=95"] Nov 7 03:09:35 xcp-ng-qhfpcnmb SM: [3535793] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] raise util.SMException("Child process exited with error") Nov 7 03:10:01 xcp-ng-qhfpcnmb SM: [3536528] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3536555, status=95 Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536525] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3536555, status=95"] Nov 7 03:10:02 xcp-ng-qhfpcnmb SM: [3536345] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] raise util.SMException("Child process exited with error") Nov 7 03:10:04 xcp-ng-qhfpcnmb SM: [3536676] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3536681, status=95 Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536673] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3536681, status=95"] Nov 7 03:10:05 xcp-ng-qhfpcnmb SM: [3536345] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] raise util.SMException("Child process exited with error") Nov 7 03:10:31 xcp-ng-qhfpcnmb SM: [3537108] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3537119, status=95 Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3537105] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3537119, status=95"] Nov 7 03:10:32 xcp-ng-qhfpcnmb SM: [3536945] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] raise util.SMException("Child process exited with error") Nov 7 03:10:34 xcp-ng-qhfpcnmb SM: [3537240] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3537245, status=95 Nov 7 03:10:34 xcp-ng-qhfpcnmb SMGC: [3537237] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3537245, status=95"] Nov 7 03:10:35 xcp-ng-qhfpcnmb SM: [3536945] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:35 xcp-ng-qhfpcnmb SMGC: [3536945] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:35 xcp-ng-qhfpcnmb SMGC: [3536945] raise util.SMException("Child process exited with error") Nov 7 03:11:01 xcp-ng-qhfpcnmb SM: [3537677] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3537688, status=95 Nov 7 03:11:01 xcp-ng-qhfpcnmb SMGC: [3537674] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3537688, status=95"] Nov 7 03:11:02 xcp-ng-qhfpcnmb SM: [3537499] EXCEPTION while coalescing: Child process exited with error Nov 7 03:11:02 xcp-ng-qhfpcnmb SMGC: [3537499] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:11:02 xcp-ng-qhfpcnmb SMGC: [3537499] raise util.SMException("Child process exited with error") Nov 7 03:11:04 xcp-ng-qhfpcnmb SM: [3537809] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3537814, status=95 Nov 7 03:11:04 xcp-ng-qhfpcnmb SMGC: [3537806] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3537814, status=95"] Nov 7 03:11:05 xcp-ng-qhfpcnmb SM: [3537499] EXCEPTION while coalescing: Child process exited with error Nov 7 03:11:05 xcp-ng-qhfpcnmb SMGC: [3537499] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:11:05 xcp-ng-qhfpcnmb SMGC: [3537499] raise util.SMException("Child process exited with error") Nov 7 03:11:31 xcp-ng-qhfpcnmb SM: [3538230] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3538241, status=95 Nov 7 03:11:31 xcp-ng-qhfpcnmb SMGC: [3538227] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3538241, status=95"]Also this errors...
[07:08 xcp-ng-qhfpcnmb ~]# cat /var/log/SMlog | grep -i error Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534785] raise Failure(result['ErrorDescription']) Nov 7 03:08:32 xcp-ng-qhfpcnmb SM: [3534596] EXCEPTION while coalescing: Child process exited with error Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] raise util.SMException("Child process exited with error") Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534940] raise Failure(result['ErrorDescription']) Nov 7 03:08:35 xcp-ng-qhfpcnmb SM: [3534596] EXCEPTION while coalescing: Child process exited with error Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] raise util.SMException("Child process exited with error") Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535376] raise Failure(result['ErrorDescription']) Nov 7 03:09:02 xcp-ng-qhfpcnmb SM: [3535200] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] raise util.SMException("Child process exited with error") Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535508] raise Failure(result['ErrorDescription']) Nov 7 03:09:05 xcp-ng-qhfpcnmb SM: [3535200] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] raise util.SMException("Child process exited with error") Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535957] raise Failure(result['ErrorDescription']) Nov 7 03:09:32 xcp-ng-qhfpcnmb SM: [3535793] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] raise util.SMException("Child process exited with error") Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3536089] raise Failure(result['ErrorDescription']) Nov 7 03:09:35 xcp-ng-qhfpcnmb SM: [3535793] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] raise util.SMException("Child process exited with error") Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536525] raise Failure(result['ErrorDescription']) Nov 7 03:10:02 xcp-ng-qhfpcnmb SM: [3536345] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] raise util.SMException("Child process exited with error") Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536673] raise Failure(result['ErrorDescription']) Nov 7 03:10:05 xcp-ng-qhfpcnmb SM: [3536345] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] raise util.SMException("Child process exited with error") Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3537105] raise Failure(result['ErrorDescription']) Nov 7 03:10:32 xcp-ng-qhfpcnmb SM: [3536945] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] raise util.SMException("Child process exited with error") Nov 7 03:10:34 xcp-ng-qhfpcnmb SMGC: [3537237] raise Failure(result['ErrorDescription']) Nov 7 03:10:35 xcp-ng-qhfpcnmb SM: [3536945] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:35 xcp-ng-qhfpcnmb SMGC: [3536945] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:35 xcp-ng-qhfpcnmb SMGC: [3536945] raise util.SMException("Child process exited with error") Nov 7 03:11:01 xcp-ng-qhfpcnmb SMGC: [3537674] raise Failure(result['ErrorDescription']) Nov 7 03:11:02 xcp-ng-qhfpcnmb SM: [3537499] EXCEPTION while coalescing: Child process exited with error Nov 7 03:11:02 xcp-ng-qhfpcnmb SMGC: [3537499] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:11:02 xcp-ng-qhfpcnmb SMGC: [3537499] raise util.SMException("Child process exited with error") Nov 7 03:11:04 xcp-ng-qhfpcnmb SMGC: [3537806] raise Failure(result['ErrorDescription']) Nov 7 03:11:05 xcp-ng-qhfpcnmb SM: [3537499] EXCEPTION while coalescing: Child process exited with error Nov 7 03:11:05 xcp-ng-qhfpcnmb SMGC: [3537499] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:11:05 xcp-ng-qhfpcnmb SMGC: [3537499] raise util.SMException("Child process exited with error") Nov 7 03:11:31 xcp-ng-qhfpcnmb SMGC: [3538227] raise Failure(result['ErrorDescription']) Nov 7 03:11:32 xcp-ng-qhfpcnmb SM: [3538069] EXCEPTION while coalescing: Child process exited with error Nov 7 03:11:32 xcp-ng-qhfpcnmb SMGC: [3538069] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:11:32 xcp-ng-qhfpcnmb SMGC: [3538069] raise util.SMException("Child process exited with error") Nov 7 03:11:34 xcp-ng-qhfpcnmb SMGC: [3538359] raise Failure(result['ErrorDescription']) Nov 7 03:11:35 xcp-ng-qhfpcnmb SM: [3538069] EXCEPTION while coalescing: Child process exited with error Nov 7 03:11:35 xcp-ng-qhfpcnmb SMGC: [3538069] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:11:35 xcp-ng-qhfpcnmb SMGC: [3538069] raise util.SMException("Child process exited with error") Nov 7 03:12:01 xcp-ng-qhfpcnmb SMGC: [3538800] raise Failure(result['ErrorDescription']) Nov 7 03:12:02 xcp-ng-qhfpcnmb SM: [3538625] EXCEPTION while coalescing: Child process exited with error Nov 7 03:12:02 xcp-ng-qhfpcnmb SMGC: [3538625] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:12:02 xcp-ng-qhfpcnmb SMGC: [3538625] raise util.SMException("Child process exited with error") Nov 7 03:12:04 xcp-ng-qhfpcnmb SMGC: [3538932] raise Failure(result['ErrorDescription']) Nov 7 03:12:05 xcp-ng-qhfpcnmb SM: [3538625] EXCEPTION while coalescing: Child process exited with error Nov 7 03:12:05 xcp-ng-qhfpcnmb SMGC: [3538625] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:12:05 xcp-ng-qhfpcnmb SMGC: [3538625] raise util.SMException("Child process exited with error") Nov 7 03:12:31 xcp-ng-qhfpcnmb SMGC: [3539350] raise Failure(result['ErrorDescription']) Nov 7 03:12:32 xcp-ng-qhfpcnmb SM: [3539190] EXCEPTION while coalescing: Child process exited with error Nov 7 03:12:32 xcp-ng-qhfpcnmb SMGC: [3539190] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:12:32 xcp-ng-qhfpcnmb SMGC: [3539190] raise util.SMException("Child process exited with error") Nov 7 03:12:34 xcp-ng-qhfpcnmb SMGC: [3539487] raise Failure(result['ErrorDescription']) Nov 7 03:12:35 xcp-ng-qhfpcnmb SM: [3539190] EXCEPTION while coalescing: Child process exited with error Nov 7 03:12:35 xcp-ng-qhfpcnmb SMGC: [3539190] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:12:35 xcp-ng-qhfpcnmb SMGC: [3539190] raise util.SMException("Child process exited with error") Nov 7 03:13:01 xcp-ng-qhfpcnmb SMGC: [3539917] raise Failure(result['ErrorDescription']) Nov 7 03:13:02 xcp-ng-qhfpcnmb SM: [3539740] EXCEPTION while coalescing: Child process exited with error Nov 7 03:13:02 xcp-ng-qhfpcnmb SMGC: [3539740] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:13:02 xcp-ng-qhfpcnmb SMGC: [3539740] raise util.SMException("Child process exited with error") Nov 7 03:13:04 xcp-ng-qhfpcnmb SMGC: [3540049] raise Failure(result['ErrorDescription']) Nov 7 03:13:05 xcp-ng-qhfpcnmb SM: [3539740] EXCEPTION while coalescing: Child process exited with error Nov 7 03:13:05 xcp-ng-qhfpcnmb SMGC: [3539740] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:13:05 xcp-ng-qhfpcnmb SMGC: [3539740] raise util.SMException("Child process exited with error") Nov 7 03:13:31 xcp-ng-qhfpcnmb SMGC: [3540470] raise Failure(result['ErrorDescription']) Nov 7 03:13:32 xcp-ng-qhfpcnmb SM: [3540307] EXCEPTION while coalescing: Child process exited with error Nov 7 03:13:32 xcp-ng-qhfpcnmb SMGC: [3540307] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:13:32 xcp-ng-qhfpcnmb SMGC: [3540307] raise util.SMException("Child process exited with error") Nov 7 03:13:34 xcp-ng-qhfpcnmb SMGC: [3540607] raise Failure(result['ErrorDescription']) Nov 7 03:13:35 xcp-ng-qhfpcnmb SM: [3540307] EXCEPTION while coalescing: Child process exited with error Nov 7 03:13:35 xcp-ng-qhfpcnmb SMGC: [3540307] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:13:35 xcp-ng-qhfpcnmb SMGC: [3540307] raise util.SMException("Child process exited with error") Nov 7 03:14:01 xcp-ng-qhfpcnmb SMGC: [3541037] raise Failure(result['ErrorDescription']) Nov 7 03:14:02 xcp-ng-qhfpcnmb SM: [3540862] EXCEPTION while coalescing: Child process exited with error Nov 7 03:14:02 xcp-ng-qhfpcnmb SMGC: [3540862] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:14:02 xcp-ng-qhfpcnmb SMGC: [3540862] raise util.SMException("Child process exited with error") Nov 7 03:14:04 xcp-ng-qhfpcnmb SMGC: [3541169] raise Failure(result['ErrorDescription']) Nov 7 03:14:05 xcp-ng-qhfpcnmb SM: [3540862] EXCEPTION while coalescing: Child process exited with error Nov 7 03:14:05 xcp-ng-qhfpcnmb SMGC: [3540862] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:14:05 xcp-ng-qhfpcnmb SMGC: [3540862] raise util.SMException("Child process exited with error") Nov 7 03:14:31 xcp-ng-qhfpcnmb SMGC: [3541605] raise Failure(result['ErrorDescription']) Nov 7 03:14:32 xcp-ng-qhfpcnmb SM: [3541442] EXCEPTION while coalescing: Child process exited with error Nov 7 03:14:32 xcp-ng-qhfpcnmb SMGC: [3541442] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:14:32 xcp-ng-qhfpcnmb SMGC: [3541442] raise util.SMException("Child process exited with error") Nov 7 03:14:34 xcp-ng-qhfpcnmb SMGC: [3541742] raise Failure(result['ErrorDescription']) Nov 7 03:14:35 xcp-ng-qhfpcnmb SM: [3541442] EXCEPTION while coalescing: Child process exited with error Nov 7 03:14:35 xcp-ng-qhfpcnmb SMGC: [3541442] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:14:35 xcp-ng-qhfpcnmb SMGC: [3541442] raise util.SMException("Child process exited with error") Nov 7 03:15:01 xcp-ng-qhfpcnmb SMGC: [3542172] raise Failure(result['ErrorDescription']) Nov 7 03:15:02 xcp-ng-qhfpcnmb SM: [3541994] EXCEPTION while coalescing: Child process exited with error Nov 7 03:15:02 xcp-ng-qhfpcnmb SMGC: [3541994] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:15:02 xcp-ng-qhfpcnmb SMGC: [3541994] raise util.SMException("Child process exited with error") Nov 7 03:15:04 xcp-ng-qhfpcnmb SMGC: [3542320] raise Failure(result['ErrorDescription']) Nov 7 03:15:05 xcp-ng-qhfpcnmb SM: [3541994] EXCEPTION while coalescing: Child process exited with error Nov 7 03:15:05 xcp-ng-qhfpcnmb SMGC: [3541994] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:15:05 xcp-ng-qhfpcnmb SMGC: [3541994] raise util.SMException("Child process exited with error") Nov 7 03:15:31 xcp-ng-qhfpcnmb SMGC: [3542761] raise Failure(result['ErrorDescription']) Nov 7 03:15:32 xcp-ng-qhfpcnmb SM: [3542597] EXCEPTION while coalescing: Child process exited with error Nov 7 03:15:32 xcp-ng-qhfpcnmb SMGC: [3542597] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:15:32 xcp-ng-qhfpcnmb SMGC: [3542597] raise util.SMException("Child process exited with error") Nov 7 03:15:34 xcp-ng-qhfpcnmb SMGC: [3542894] raise Failure(result['ErrorDescription']) Nov 7 03:15:35 xcp-ng-qhfpcnmb SM: [3542597] EXCEPTION while coalescing: Child process exited with error Nov 7 03:15:35 xcp-ng-qhfpcnmb SMGC: [3542597] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:15:35 xcp-ng-qhfpcnmb SMGC: [3542597] raise util.SMException("Child process exited with error") Nov 7 03:16:01 xcp-ng-qhfpcnmb SMGC: [3543326] raise Failure(result['ErrorDescription']) Nov 7 03:16:02 xcp-ng-qhfpcnmb SM: [3543146] EXCEPTION while coalescing: Child process exited with error Nov 7 03:16:02 xcp-ng-qhfpcnmb SMGC: [3543146] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:16:02 xcp-ng-qhfpcnmb SMGC: [3543146] raise util.SMException("Child process exited with error") Nov 7 03:16:04 xcp-ng-qhfpcnmb SMGC: [3543458] raise Failure(result['ErrorDescription']) Nov 7 03:16:05 xcp-ng-qhfpcnmb SM: [3543146] EXCEPTION while coalescing: Child process exited with error Nov 7 03:16:05 xcp-ng-qhfpcnmb SMGC: [3543146] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:16:05 xcp-ng-qhfpcnmb SMGC: [3543146] raise util.SMException("Child process exited with error") Nov 7 03:16:31 xcp-ng-qhfpcnmb SMGC: [3543882] raise Failure(result['ErrorDescription']) Nov 7 03:16:32 xcp-ng-qhfpcnmb SM: [3543722] EXCEPTION while coalescing: Child process exited with error Nov 7 03:16:32 xcp-ng-qhfpcnmb SMGC: [3543722] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:16:32 xcp-ng-qhfpcnmb SMGC: [3543722] raise util.SMException("Child process exited with error") Nov 7 03:16:34 xcp-ng-qhfpcnmb SMGC: [3544014] raise Failure(result['ErrorDescription']) Nov 7 03:16:35 xcp-ng-qhfpcnmb SM: [3543722] EXCEPTION while coalescing: Child process exited with error Nov 7 03:16:35 xcp-ng-qhfpcnmb SMGC: [3543722] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:16:35 xcp-ng-qhfpcnmb SMGC: [3543722] raise util.SMException("Child process exited with error") Nov 7 03:17:01 xcp-ng-qhfpcnmb SMGC: [3544446] raise Failure(result['ErrorDescription']) -
More errors...
[07:21 xcp-ng-qhfpcnmb ~]# grep -i coales /var/log/SMlog Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesced size = 6.596G Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesced size = 6.596G Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:08:30 xcp-ng-qhfpcnmb SMGC: [3534596] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesced size = 6.596G Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534596] Coalescing *39355b7e(256.000G/9.938M/8.875M)[qcow2] -> *f35d8b9e(256.000G/6.566G/6.556G)[qcow2] Nov 7 03:08:31 xcp-ng-qhfpcnmb SM: [3534596] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2 Nov 7 03:08:31 xcp-ng-qhfpcnmb SM: [3534785] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:08:31 xcp-ng-qhfpcnmb SM: [3534788] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3534799, status=95 Nov 7 03:08:31 xcp-ng-qhfpcnmb SM: [3534788] return str(cowutil.coalesceOnline(path)) Nov 7 03:08:31 xcp-ng-qhfpcnmb SM: [3534788] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534785] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:08:31 xcp-ng-qhfpcnmb SMGC: [3534785] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:08:32 xcp-ng-qhfpcnmb SM: [3534596] EXCEPTION while coalescing: Child process exited with error Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] self._coalesce(vdi) Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:08:32 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce failed, skipping Nov 7 03:08:33 xcp-ng-qhfpcnmb SMGC: [3534596] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:08:33 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesced size = 51.125M Nov 7 03:08:33 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:08:33 xcp-ng-qhfpcnmb SMGC: [3534596] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesced size = 51.125M Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534596] Coalescing *4dd67111(256.000G/6.750M/5.875M)[qcow2] -> *39355b7e(256.000G/9.938M/8.875M)[qcow2] Nov 7 03:08:34 xcp-ng-qhfpcnmb SM: [3534596] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2 Nov 7 03:08:34 xcp-ng-qhfpcnmb SM: [3534940] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:08:34 xcp-ng-qhfpcnmb SM: [3534943] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3534948, status=95 Nov 7 03:08:34 xcp-ng-qhfpcnmb SM: [3534943] return str(cowutil.coalesceOnline(path)) Nov 7 03:08:34 xcp-ng-qhfpcnmb SM: [3534943] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534940] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:08:34 xcp-ng-qhfpcnmb SMGC: [3534940] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:08:35 xcp-ng-qhfpcnmb SM: [3534596] EXCEPTION while coalescing: Child process exited with error Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] self._coalesce(vdi) Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:08:35 xcp-ng-qhfpcnmb SMGC: [3534596] Coalesce failed, skipping Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesced size = 6.596G Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesced size = 6.596G Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:09:00 xcp-ng-qhfpcnmb SMGC: [3535200] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesced size = 6.596G Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535200] Coalescing *39355b7e(256.000G/9.938M/8.875M)[qcow2] -> *f35d8b9e(256.000G/6.566G/6.556G)[qcow2] Nov 7 03:09:01 xcp-ng-qhfpcnmb SM: [3535200] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2 Nov 7 03:09:01 xcp-ng-qhfpcnmb SM: [3535376] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:09:01 xcp-ng-qhfpcnmb SM: [3535379] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3535390, status=95 Nov 7 03:09:01 xcp-ng-qhfpcnmb SM: [3535379] return str(cowutil.coalesceOnline(path)) Nov 7 03:09:01 xcp-ng-qhfpcnmb SM: [3535379] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535376] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:09:01 xcp-ng-qhfpcnmb SMGC: [3535376] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:09:02 xcp-ng-qhfpcnmb SM: [3535200] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] self._coalesce(vdi) Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:09:02 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce failed, skipping Nov 7 03:09:03 xcp-ng-qhfpcnmb SMGC: [3535200] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:03 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesced size = 51.125M Nov 7 03:09:03 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:09:03 xcp-ng-qhfpcnmb SMGC: [3535200] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesced size = 51.125M Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535200] Coalescing *4dd67111(256.000G/6.750M/5.875M)[qcow2] -> *39355b7e(256.000G/9.938M/8.875M)[qcow2] Nov 7 03:09:04 xcp-ng-qhfpcnmb SM: [3535200] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2 Nov 7 03:09:04 xcp-ng-qhfpcnmb SM: [3535508] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:09:04 xcp-ng-qhfpcnmb SM: [3535511] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3535527, status=95 Nov 7 03:09:04 xcp-ng-qhfpcnmb SM: [3535511] return str(cowutil.coalesceOnline(path)) Nov 7 03:09:04 xcp-ng-qhfpcnmb SM: [3535511] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535508] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:09:04 xcp-ng-qhfpcnmb SMGC: [3535508] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:09:05 xcp-ng-qhfpcnmb SM: [3535200] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] self._coalesce(vdi) Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:09:05 xcp-ng-qhfpcnmb SMGC: [3535200] Coalesce failed, skipping Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesced size = 6.596G Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesced size = 6.596G Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:09:30 xcp-ng-qhfpcnmb SMGC: [3535793] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesced size = 6.596G Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535793] Coalescing *39355b7e(256.000G/9.938M/8.875M)[qcow2] -> *f35d8b9e(256.000G/6.566G/6.556G)[qcow2] Nov 7 03:09:31 xcp-ng-qhfpcnmb SM: [3535793] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2 Nov 7 03:09:31 xcp-ng-qhfpcnmb SM: [3535957] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:09:31 xcp-ng-qhfpcnmb SM: [3535960] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3535971, status=95 Nov 7 03:09:31 xcp-ng-qhfpcnmb SM: [3535960] return str(cowutil.coalesceOnline(path)) Nov 7 03:09:31 xcp-ng-qhfpcnmb SM: [3535960] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535957] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:09:31 xcp-ng-qhfpcnmb SMGC: [3535957] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:09:32 xcp-ng-qhfpcnmb SM: [3535793] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] self._coalesce(vdi) Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:09:32 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce failed, skipping Nov 7 03:09:33 xcp-ng-qhfpcnmb SMGC: [3535793] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:33 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesced size = 51.125M Nov 7 03:09:33 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:09:33 xcp-ng-qhfpcnmb SMGC: [3535793] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesced size = 51.125M Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3535793] Coalescing *4dd67111(256.000G/6.750M/5.875M)[qcow2] -> *39355b7e(256.000G/9.938M/8.875M)[qcow2] Nov 7 03:09:34 xcp-ng-qhfpcnmb SM: [3535793] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2 Nov 7 03:09:34 xcp-ng-qhfpcnmb SM: [3536089] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:09:34 xcp-ng-qhfpcnmb SM: [3536092] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3536097, status=95 Nov 7 03:09:34 xcp-ng-qhfpcnmb SM: [3536092] return str(cowutil.coalesceOnline(path)) Nov 7 03:09:34 xcp-ng-qhfpcnmb SM: [3536092] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3536089] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:09:34 xcp-ng-qhfpcnmb SMGC: [3536089] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:09:35 xcp-ng-qhfpcnmb SM: [3535793] EXCEPTION while coalescing: Child process exited with error Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] self._coalesce(vdi) Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:09:35 xcp-ng-qhfpcnmb SMGC: [3535793] Coalesce failed, skipping Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesced size = 6.596G Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesced size = 6.596G Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:10:00 xcp-ng-qhfpcnmb SMGC: [3536345] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesced size = 6.596G Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536345] Coalescing *39355b7e(256.000G/9.938M/8.875M)[qcow2] -> *f35d8b9e(256.000G/6.566G/6.556G)[qcow2] Nov 7 03:10:01 xcp-ng-qhfpcnmb SM: [3536345] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2 Nov 7 03:10:01 xcp-ng-qhfpcnmb SM: [3536525] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:10:01 xcp-ng-qhfpcnmb SM: [3536528] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3536555, status=95 Nov 7 03:10:01 xcp-ng-qhfpcnmb SM: [3536528] return str(cowutil.coalesceOnline(path)) Nov 7 03:10:01 xcp-ng-qhfpcnmb SM: [3536528] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536525] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:10:01 xcp-ng-qhfpcnmb SMGC: [3536525] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:10:02 xcp-ng-qhfpcnmb SM: [3536345] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] self._coalesce(vdi) Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:10:02 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce failed, skipping Nov 7 03:10:03 xcp-ng-qhfpcnmb SMGC: [3536345] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:03 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesced size = 51.125M Nov 7 03:10:03 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:10:03 xcp-ng-qhfpcnmb SMGC: [3536345] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesced size = 51.125M Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536345] Coalescing *4dd67111(256.000G/6.750M/5.875M)[qcow2] -> *39355b7e(256.000G/9.938M/8.875M)[qcow2] Nov 7 03:10:04 xcp-ng-qhfpcnmb SM: [3536345] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2 Nov 7 03:10:04 xcp-ng-qhfpcnmb SM: [3536673] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:10:04 xcp-ng-qhfpcnmb SM: [3536676] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3536681, status=95 Nov 7 03:10:04 xcp-ng-qhfpcnmb SM: [3536676] return str(cowutil.coalesceOnline(path)) Nov 7 03:10:04 xcp-ng-qhfpcnmb SM: [3536676] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536673] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:10:04 xcp-ng-qhfpcnmb SMGC: [3536673] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:10:05 xcp-ng-qhfpcnmb SM: [3536345] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] self._coalesce(vdi) Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:10:05 xcp-ng-qhfpcnmb SMGC: [3536345] Coalesce failed, skipping Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesced size = 6.596G Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesced size = 6.596G Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] 39355b7e-0735-47ed-97be-8b09864a31ba is coalescable Nov 7 03:10:30 xcp-ng-qhfpcnmb SMGC: [3536945] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesced size = 6.596G Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesce candidate: *39355b7e(256.000G/9.938M/8.875M)[qcow2] (tree height 4) Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3536945] Coalescing *39355b7e(256.000G/9.938M/8.875M)[qcow2] -> *f35d8b9e(256.000G/6.566G/6.556G)[qcow2] Nov 7 03:10:31 xcp-ng-qhfpcnmb SM: [3536945] Remote coalesce for /var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2 Nov 7 03:10:31 xcp-ng-qhfpcnmb SM: [3537105] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2', 'vdi_type': 'qcow2'}Nov 7 03:10:31 xcp-ng-qhfpcnmb SM: [3537108] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3537119, status=95 Nov 7 03:10:31 xcp-ng-qhfpcnmb SM: [3537108] return str(cowutil.coalesceOnline(path)) Nov 7 03:10:31 xcp-ng-qhfpcnmb SM: [3537108] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3537105] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:10:31 xcp-ng-qhfpcnmb SMGC: [3537105] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce Nov 7 03:10:32 xcp-ng-qhfpcnmb SM: [3536945] EXCEPTION while coalescing: Child process exited with error Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] self._coalesce(vdi) Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:10:32 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesce failed, skipping Nov 7 03:10:33 xcp-ng-qhfpcnmb SMGC: [3536945] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:33 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesced size = 51.125M Nov 7 03:10:33 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) Nov 7 03:10:33 xcp-ng-qhfpcnmb SMGC: [3536945] 4dd67111-5da5-485e-a0d0-21aa3d34c07f is coalescable Nov 7 03:10:34 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesced size = 51.125M Nov 7 03:10:34 xcp-ng-qhfpcnmb SMGC: [3536945] Coalesce candidate: *4dd67111(256.000G/6.750M/5.875M)[qcow2] (tree height 4) -
Hi,
Try adjusting your grep to include surrounding lines, ie --
grep -A 5 -B 5 -i exception /var/log/SMlog -
@Danp unfortunately get the beginning of the error. that is as far up as i can scroll. Not sure how much was lost from the beginning.
Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] *********************** Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] * E X C E P T I O N * Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] *********************** Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] self._coalesce(vdi) Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] None, self.sr.uuid, abortTest, VDI.POLL_INTERVAL, 0, prefSig=signal.SIGTERM) Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 200, in runAbortable Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] raise util.SMException("Child process exited with error") Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] Coalesce failed, skipping Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] In cleanup Nov 7 03:46:32 xcp-ng-qhfpcnmb SMGC: [3577661] Starting asynch srUpdate for SR 8743a32f-9325-88cd-a0e1-b89e340fdc66 -- Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] ['lsof', '-t', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] pread SUCCESS Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2']Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] = 95 Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] Query command failed on tapdisk instance 3040201. Raising... Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3577964, status=95 Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] File "/etc/xapi.d/plugins/on-slave", line 184, in commit_tapdisk Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] return str(cowutil.coalesceOnline(path)) Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] TapCtl.commit(pid, minor, QCOW2_TYPE, path) Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577959] File "/opt/xensource/sm/blktap2.py", line 453, in commit -- Nov 7 03:46:34 xcp-ng-qhfpcnmb SM: [3577956] IPCFlag: set 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] *********************** Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] * E X C E P T I O N * Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] *********************** Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3577964, status=95"] Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] File "/opt/xensource/sm/cleanup.py", line 224, in runAbortable Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] if func() == ret: Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] File "/opt/xensource/sm/cleanup.py", line 884, in <lambda> Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce -- Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] raise Failure(result['ErrorDescription']) Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] Nov 7 03:46:34 xcp-ng-qhfpcnmb SMGC: [3577956] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:46:35 xcp-ng-qhfpcnmb SM: [3577661] IPCFlag: clear 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:46:35 xcp-ng-qhfpcnmb SM: [3577661] EXCEPTION while coalescing: Child process exited with error Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] *********************** Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] * E X C E P T I O N * Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] *********************** Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] self._coalesce(vdi) Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] None, self.sr.uuid, abortTest, VDI.POLL_INTERVAL, 0, prefSig=signal.SIGTERM) Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] File "/opt/xensource/sm/cleanup.py", line 200, in runAbortable Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] raise util.SMException("Child process exited with error") Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] Coalesce failed, skipping Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] In cleanup Nov 7 03:46:35 xcp-ng-qhfpcnmb SMGC: [3577661] Starting asynch srUpdate for SR 8743a32f-9325-88cd-a0e1-b89e340fdc66 -- Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] ['lsof', '-t', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] pread SUCCESS Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2']Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] = 95 Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] Query command failed on tapdisk instance 3040201. Raising... Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3578402, status=95 Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] File "/etc/xapi.d/plugins/on-slave", line 184, in commit_tapdisk Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] return str(cowutil.coalesceOnline(path)) Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] TapCtl.commit(pid, minor, QCOW2_TYPE, path) Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578391] File "/opt/xensource/sm/blktap2.py", line 453, in commit -- Nov 7 03:47:01 xcp-ng-qhfpcnmb SM: [3578388] IPCFlag: set 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] *********************** Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] * E X C E P T I O N * Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] *********************** Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3578402, status=95"] Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] File "/opt/xensource/sm/cleanup.py", line 224, in runAbortable Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] if func() == ret: Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] File "/opt/xensource/sm/cleanup.py", line 884, in <lambda> Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce -- Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] raise Failure(result['ErrorDescription']) Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] Nov 7 03:47:01 xcp-ng-qhfpcnmb SMGC: [3578388] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:02 xcp-ng-qhfpcnmb SM: [3578208] IPCFlag: clear 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:02 xcp-ng-qhfpcnmb SM: [3578208] EXCEPTION while coalescing: Child process exited with error Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] *********************** Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] * E X C E P T I O N * Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] *********************** Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] self._coalesce(vdi) Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] None, self.sr.uuid, abortTest, VDI.POLL_INTERVAL, 0, prefSig=signal.SIGTERM) Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 200, in runAbortable Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] raise util.SMException("Child process exited with error") Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] Coalesce failed, skipping Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] In cleanup Nov 7 03:47:02 xcp-ng-qhfpcnmb SMGC: [3578208] Starting asynch srUpdate for SR 8743a32f-9325-88cd-a0e1-b89e340fdc66 -- Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] ['lsof', '-t', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] pread SUCCESS Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2']Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] = 95 Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] Query command failed on tapdisk instance 3040201. Raising... Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3578528, status=95 Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] File "/etc/xapi.d/plugins/on-slave", line 184, in commit_tapdisk Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] return str(cowutil.coalesceOnline(path)) Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] TapCtl.commit(pid, minor, QCOW2_TYPE, path) Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578523] File "/opt/xensource/sm/blktap2.py", line 453, in commit -- Nov 7 03:47:04 xcp-ng-qhfpcnmb SM: [3578520] IPCFlag: set 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] *********************** Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] * E X C E P T I O N * Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] *********************** Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3578528, status=95"] Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] File "/opt/xensource/sm/cleanup.py", line 224, in runAbortable Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] if func() == ret: Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] File "/opt/xensource/sm/cleanup.py", line 884, in <lambda> Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce -- Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] raise Failure(result['ErrorDescription']) Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] Nov 7 03:47:04 xcp-ng-qhfpcnmb SMGC: [3578520] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:05 xcp-ng-qhfpcnmb SM: [3578208] IPCFlag: clear 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:05 xcp-ng-qhfpcnmb SM: [3578208] EXCEPTION while coalescing: Child process exited with error Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] *********************** Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] * E X C E P T I O N * Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] *********************** Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] self._coalesce(vdi) Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] None, self.sr.uuid, abortTest, VDI.POLL_INTERVAL, 0, prefSig=signal.SIGTERM) Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] File "/opt/xensource/sm/cleanup.py", line 200, in runAbortable Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] raise util.SMException("Child process exited with error") Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] Coalesce failed, skipping Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] In cleanup Nov 7 03:47:05 xcp-ng-qhfpcnmb SMGC: [3578208] Starting asynch srUpdate for SR 8743a32f-9325-88cd-a0e1-b89e340fdc66 -- Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] ['lsof', '-t', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] pread SUCCESS Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2']Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] = 95 Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] Query command failed on tapdisk instance 3040201. Raising... Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3578962, status=95 Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] File "/etc/xapi.d/plugins/on-slave", line 184, in commit_tapdisk Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] return str(cowutil.coalesceOnline(path)) Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] TapCtl.commit(pid, minor, QCOW2_TYPE, path) Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578951] File "/opt/xensource/sm/blktap2.py", line 453, in commit -- Nov 7 03:47:31 xcp-ng-qhfpcnmb SM: [3578948] IPCFlag: set 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] *********************** Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] * E X C E P T I O N * Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] *********************** Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3578962, status=95"] Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] File "/opt/xensource/sm/cleanup.py", line 224, in runAbortable Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] if func() == ret: Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] File "/opt/xensource/sm/cleanup.py", line 884, in <lambda> Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce -- Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] raise Failure(result['ErrorDescription']) Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] Nov 7 03:47:31 xcp-ng-qhfpcnmb SMGC: [3578948] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:32 xcp-ng-qhfpcnmb SM: [3578786] IPCFlag: clear 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:32 xcp-ng-qhfpcnmb SM: [3578786] EXCEPTION while coalescing: Child process exited with error Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] *********************** Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] * E X C E P T I O N * Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] *********************** Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] self._coalesce(vdi) Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] None, self.sr.uuid, abortTest, VDI.POLL_INTERVAL, 0, prefSig=signal.SIGTERM) Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 200, in runAbortable Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] raise util.SMException("Child process exited with error") Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] Coalesce failed, skipping Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] In cleanup Nov 7 03:47:32 xcp-ng-qhfpcnmb SMGC: [3578786] Starting asynch srUpdate for SR 8743a32f-9325-88cd-a0e1-b89e340fdc66 -- Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] ['lsof', '-t', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] pread SUCCESS Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2']Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] = 95 Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] Query command failed on tapdisk instance 3040201. Raising... Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3579088, status=95 Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] File "/etc/xapi.d/plugins/on-slave", line 184, in commit_tapdisk Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] return str(cowutil.coalesceOnline(path)) Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] TapCtl.commit(pid, minor, QCOW2_TYPE, path) Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579083] File "/opt/xensource/sm/blktap2.py", line 453, in commit -- Nov 7 03:47:34 xcp-ng-qhfpcnmb SM: [3579080] IPCFlag: set 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] *********************** Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] * E X C E P T I O N * Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] *********************** Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] This exception has occured: EXCEPTION <class 'XenAPI.Failure'>, ['XENAPI_PLUGIN_FAILURE', 'commit_tapdisk', 'CommandFailure', "['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/4dd67111-5da5-485e-a0d0-21aa3d34c07f.qcow2'] failed: errmsg=Operation not supported, pid=3579088, status=95"] Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] File "/opt/xensource/sm/cleanup.py", line 224, in runAbortable Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] if func() == ret: Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] File "/opt/xensource/sm/cleanup.py", line 884, in <lambda> Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] Util.runAbortable(lambda: self._call_plugin_coalesce(hostRef), \ Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] File "/opt/xensource/sm/cleanup.py", line 851, in _call_plugin_coalesce -- Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] raise Failure(result['ErrorDescription']) Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] Nov 7 03:47:34 xcp-ng-qhfpcnmb SMGC: [3579080] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:35 xcp-ng-qhfpcnmb SM: [3578786] IPCFlag: clear 8743a32f-9325-88cd-a0e1-b89e340fdc66:failure Nov 7 03:47:35 xcp-ng-qhfpcnmb SM: [3578786] EXCEPTION while coalescing: Child process exited with error Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] *********************** Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] * E X C E P T I O N * Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] *********************** Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] coalesce: EXCEPTION <class 'util.SMException'>, Child process exited with error Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 2302, in coalesce Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] self._coalesce(vdi) Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 2524, in _coalesce Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] vdi._doCoalesceOnHost(list(host_refs)[0]) Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 885, in _doCoalesceOnHost Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] None, self.sr.uuid, abortTest, VDI.POLL_INTERVAL, 0, prefSig=signal.SIGTERM) Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] File "/opt/xensource/sm/cleanup.py", line 200, in runAbortable Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] raise util.SMException("Child process exited with error") Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] Coalesce failed, skipping Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] In cleanup Nov 7 03:47:35 xcp-ng-qhfpcnmb SMGC: [3578786] Starting asynch srUpdate for SR 8743a32f-9325-88cd-a0e1-b89e340fdc66 -- Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] ['lsof', '-t', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] pread SUCCESS Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2']Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] = 95 Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] Query command failed on tapdisk instance 3040201. Raising... Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] ***** Couldn't coalesce online: EXCEPTION <class 'blktap2.TapCtl.CommandFailure'>, ['/usr/sbin/tap-ctl', 'commit', '-p', '3040201', '-m', '4', '-a', '/var/run/sr-mount/8743a32f-9325-88cd-a0e1-b89e340fdc66/39355b7e-0735-47ed-97be-8b09864a31ba.qcow2'] failed: errmsg=Operation not supported, pid=3579530, status=95 Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] File "/etc/xapi.d/plugins/on-slave", line 184, in commit_tapdisk Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] return str(cowutil.coalesceOnline(path)) Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] File "/opt/xensource/sm/qcow2util.py", line 734, in coalesceOnline Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] TapCtl.commit(pid, minor, QCOW2_TYPE, path) Nov 7 03:48:01 xcp-ng-qhfpcnmb SM: [3579519] File "/opt/xensource/sm/blktap2.py", line 453, in commit -
I also had to cut alot of the bottom of that log just to post it.
-
@Danp it appears the unhealthy vdi are no longer appearing, long after backup completes.
Just to eliminate any issues with the SR i decided to migrate all the vms off that SR onto a tmp SR and recreate the original SR. I am in the process of migrating them back to the newly created SR.
I ran those above command again and this is the output now..
[10:31 xcp-ng-qhfpcnmb ~]# cat /var/log/SMlog | grep -i exception Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] Raising exception [101, The request is missing the serverpath parameter] Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ***** generic exception: sr_probe: EXCEPTION <class 'xs_errors.SROSError'>, The request is missing the serverpath parameter Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ***** NFS VHD and QCOW2: EXCEPTION <class 'xs_errors.SROSError'>, The request is missing the serverpath parameter [10:32 xcp-ng-qhfpcnmb ~]# cat /var/log/SMlog | grep -i error Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ***** generic exception: sr_probe: EXCEPTION <class 'xs_errors.SROSError'>, The request is missing the serverpath parameter Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] raise xs_errors.XenError('ConfigServerPathMissing') Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ***** NFS VHD and QCOW2: EXCEPTION <class 'xs_errors.SROSError'>, The request is missing the serverpath parameter Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] raise xs_errors.XenError('ConfigServerPathMissing') [10:32 xcp-ng-qhfpcnmb ~]# grep -i coales /var/log/SMlog Nov 8 09:23:26 xcp-ng-qhfpcnmb SM: [566243] We have secondary for 2944d0bb-11fe-4a3d-8272-6b3354d07342, blocking leaf coalesce Nov 8 09:23:48 xcp-ng-qhfpcnmb SM: [567385] We have secondary for 01b89462-cae3-4657-b0f1-83eee08dce66, blocking leaf coalesce Nov 8 09:35:29 xcp-ng-qhfpcnmb SMGC: [576746] Got other-config for 2944d0bb(64.000G/136.000K?)[vhd]: {'leaf-coalesce': 'false'} Nov 8 09:35:29 xcp-ng-qhfpcnmb SMGC: [576746] Leaf-coalesce disabled for 2944d0bb(64.000G/136.000K?)[vhd] Nov 8 09:35:34 xcp-ng-qhfpcnmb SMGC: [576990] Coalesced size = 5.992G Nov 8 09:35:34 xcp-ng-qhfpcnmb SMGC: [576990] Leaf-coalesce candidate: 053f72af(64.000G/80.000K/0)[qcow2] Nov 8 09:35:34 xcp-ng-qhfpcnmb SMGC: [576990] Leaf-coalescing 053f72af(64.000G/80.000K/0)[qcow2] -> *b99d6033(64.000G/23.931G/23.926G)[qcow2] Nov 8 09:35:34 xcp-ng-qhfpcnmb SMGC: [576990] Running COW coalesce on 053f72af(64.000G/80.000K/0)[qcow2] Nov 8 09:35:36 xcp-ng-qhfpcnmb SMGC: [576990] Removed leaf-coalesce from 053f72af(64.000G/23.931G/23.926G)[qcow2] Nov 8 09:46:08 xcp-ng-qhfpcnmb SMGC: [584997] Got other-config for 01b89462(128.000G/1.705G?)[vhd]: {'leaf-coalesce': 'false'} Nov 8 09:46:08 xcp-ng-qhfpcnmb SMGC: [584997] Leaf-coalesce disabled for 01b89462(128.000G/1.705G?)[vhd] Nov 8 09:46:10 xcp-ng-qhfpcnmb SMGC: [585279] Got other-config for 01b89462(128.000G/1.705G?)[vhd]: {'leaf-coalesce': 'false'} Nov 8 09:46:10 xcp-ng-qhfpcnmb SMGC: [585279] Leaf-coalesce disabled for 01b89462(128.000G/1.705G?)[vhd] Nov 8 09:46:26 xcp-ng-qhfpcnmb SMGC: [585740] Coalesced size = 19.549G Nov 8 09:46:26 xcp-ng-qhfpcnmb SMGC: [585740] Leaf-coalesce candidate: 0541a5bc(128.000G/117.867M/89.189M)[qcow2] Nov 8 09:46:26 xcp-ng-qhfpcnmb SMGC: [585740] Leaf-coalescing 0541a5bc(128.000G/117.867M/89.189M)[qcow2] -> *906ab99e(128.000G/77.916G/77.904G)[qcow2] Nov 8 09:46:28 xcp-ng-qhfpcnmb SMGC: [585740] Coalescing parent *2e1779a9(128.000G/117.883M/89.193M)[qcow2] Nov 8 09:46:28 xcp-ng-qhfpcnmb SM: [585740] Remote coalesce for /var/run/sr-mount/5ca7d722-6820-6762-d872-b65f6fc7f3f8/2e1779a9-06a0-44cc-bd83-172c82f15f42.qcow2 Nov 8 09:46:28 xcp-ng-qhfpcnmb SM: [586271] DAMS: Calling remote coalesce with: {'path': '/var/run/sr-mount/5ca7d722-6820-6762-d872-b65f6fc7f3f8/2e1779a9-06a0-44cc-bd83-172c82f15f42.qcow2', 'vdi_type': 'qcow2'} Nov 8 09:46:37 xcp-ng-qhfpcnmb SM: [586271] DAMS: Remote coalesce returned 137438953472 Nov 8 09:46:38 xcp-ng-qhfpcnmb SMGC: [585740] Coalesce summary Nov 8 09:46:39 xcp-ng-qhfpcnmb SMGC: [585740] Running COW coalesce on 0541a5bc(128.000G/25.891M/24.750M)[qcow2] Nov 8 09:46:41 xcp-ng-qhfpcnmb SMGC: [585740] Removed leaf-coalesce from 0541a5bc(128.000G/77.916G/77.904G)[qcow2] Nov 8 09:47:48 xcp-ng-qhfpcnmb SM: [587542] Aborting GC/coalesce Nov 8 09:47:56 xcp-ng-qhfpcnmb SM: [587621] Aborting GC/coalesce Nov 8 09:47:56 xcp-ng-qhfpcnmb SM: [587621] Aborting GC/coalesce Nov 8 09:53:48 xcp-ng-qhfpcnmb SM: [591030] Aborting GC/coalesce Nov 8 09:54:52 xcp-ng-qhfpcnmb SM: [592147] We have secondary for 053f72af-867b-499e-8123-4c6897737948, blocking leaf coalesce Nov 8 09:59:14 xcp-ng-qhfpcnmb SMGC: [595908] Got other-config for 053f72af(64.000G/193.000K/0)[qcow2]: {'leaf-coalesce': 'false'} Nov 8 09:59:14 xcp-ng-qhfpcnmb SMGC: [595908] Leaf-coalesce disabled for 053f72af(64.000G/193.000K/0)[qcow2] Nov 8 09:59:18 xcp-ng-qhfpcnmb SMGC: [596131] Coalesced size = 5.991G Nov 8 09:59:18 xcp-ng-qhfpcnmb SMGC: [596131] Leaf-coalesce candidate: 406997bc(64.000G/80.000K/0)[qcow2] Nov 8 09:59:18 xcp-ng-qhfpcnmb SMGC: [596131] Leaf-coalescing 406997bc(64.000G/80.000K/0)[qcow2] -> *e7281aae(64.000G/23.935G/23.926G)[qcow2] Nov 8 09:59:18 xcp-ng-qhfpcnmb SMGC: [596131] Running COW coalesce on 406997bc(64.000G/80.000K/0)[qcow2] Nov 8 09:59:20 xcp-ng-qhfpcnmb SMGC: [596131] Removed leaf-coalesce from 406997bc(64.000G/23.935G/23.926G)[qcow2] Nov 8 10:00:29 xcp-ng-qhfpcnmb SM: [597528] We have secondary for f8a9567f-66a3-4fdc-9313-711d13851cb6, blocking leaf coalesce Nov 8 10:06:28 xcp-ng-qhfpcnmb SM: [602617] We have secondary for 0b3d1b69-0936-4951-be61-3939a96cb462, blocking leaf coalesce [10:32 xcp-ng-qhfpcnmb ~]# grep -A 5 -B 5 -i exception /var/log/SMlog Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] _testHost: Testing host/port: 10.100.2.183,2049 Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] scanning2 (target=10.100.2.183) Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] scanning Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ['/usr/sbin/showmount', '--no-headers', '-e', '10.100.2.183'] Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] pread SUCCESS Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] Raising exception [101, The request is missing the serverpath parameter] Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] lock: released /var/lock/sm/sr Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ***** generic exception: sr_probe: EXCEPTION <class 'xs_errors.SROSError'>, The request is missing the serverpath parameter Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/SRCommand.py", line 113, in run Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] return self._run_locked(sr) Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/SRCommand.py", line 163, in _run_locked Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] rv = self._run(sr, target) Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/SRCommand.py", line 345, in _run -- Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/NFSSR", line 177, in probe Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] self.validate_remotepath(True) Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/NFSSR", line 119, in validate_remotepath Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] raise xs_errors.XenError('ConfigServerPathMissing') Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ***** NFS VHD and QCOW2: EXCEPTION <class 'xs_errors.SROSError'>, The request is missing the serverpath parameter Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/SRCommand.py", line 392, in run Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] ret = cmd.run(sr) Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/SRCommand.py", line 113, in run Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] return self._run_locked(sr) Nov 8 09:53:39 xcp-ng-qhfpcnmb SM: [590827] File "/opt/xensource/sm/SRCommand.py", line 163, in _run_locked -
It seams migration back to new SR is very very slow. in the 1 - 2 hr mark when the initial migration with converting vhd to qcow2 was quicker in the 20 - 40 min mark.
-
A acebmxer referenced this topic
-
Not sure if related or not but I ended up removing the beta qcow2 support. Even though after recreating my SR I was still seeing other odd behaviors.
Backup progress was not shown on vms when qcow2 support enabled.
CBT delete snapshots were having issues
With CBT disabled snapshots were adding up and coalescing issues.I didn't do any official speed testing but overall performance seemed slower.
After disabling qcow2 support all seems better now. Backup process is able to be monitored, CBT backups working as expected and performance is back.
-
You were using qcow2 instead of VHD, right?
-
@olivierlambert Yes I previously was. now back on VHD. This was in my home lab, not work enviorment.
