Large VM Migration fails, host has free disk - Storage_error

ditzy-olive

I am trying to migrate a rather large vm 1.6T of disks in total.
It runs for 15 Hours and then fails with There is insufficient space
The target xcp-ng however has 3.09TB of disk free so I am a bit confused where the insufficient space is coming from.
They hosts are running XCP-ng 8.2.1, other VM could be migrated without issue but struggling with this large one.
I could potentially start splitting the services from the VM into smaller VM which wouldn't be a bad idea anyway, but still would like to understand how to debug this.

"(((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 1564))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 131))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/xapi/rbac.ml)(line 205))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 95)))"
message	"INTERNAL_ERROR(Storage_error ([S(Internal_error);S(Storage_error ([S(Internal_error);S(Storage_error ([S(Internal_error);S(Storage_error ([S(Backend_error);[S(SR_BACKEND_FAILURE_44);[S();S(There is insufficient space);S()]]]))]))]))]))"
name	"XapiError"
stack	"XapiError: INTERNAL_ERROR(Storage_error ([S(Internal_error);S(Storage_error ([S(Internal_error);S(Storage_error ([S(Internal_error);S(Storage_error ([S(Backend_error);[S(SR_BACKEND_FAILURE_44);[S();S(There is insufficient space);S()]]]))]))]))]))\n    at Function.wrap (file:///opt/xo/xo-builds/xen-orchestra-202512060836/packages/xen-api/_XapiError.mjs:16:12)\n    at default (file:///opt/xo/xo-builds/xen-orchestra-202512060836/packages/xen-api/_getTaskResult.mjs:13:29)\n    at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202512060836/packages/xen-api/index.mjs:1073:24)\n    at file:///opt/xo/xo-builds/xen-orchestra-202512060836/packages/xen-api/index.mjs:1107:14\n    at Array.forEach (<anonymous>)\n    at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202512060836/packages/xen-api/index.mjs:1097:12)\n    at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202512060836/packages/xen-api/index.mjs:1270:14)

Pilow

@ditzy-olive Hi,

What are the type of SRs (Source and destination ?) NFS . iSCSI ? local or shared ?
What is the free space on both, before launching the migration ?

ditzy-olive

@Pilow both are local.
the host where it's being moved from has 2.3 TB free the host where it's being migrated to has 3TB Free.
I moved other VM without any issue.

Pilow

@ditzy-olive what bugs me is that error is happening 15h after the start

1.6Tb disks in total, you could detach disks, migrate VM+system, then migrate secondary disks and re attach them ?

or try offline migrating ? perhaps this VM is too i/o intensive and fills up its snapshots ?

source and destination SRs are thin or thick ?

nikade

Are you using EXT or LVM?
I've seen this as well with large VM's and not enough free space on the source host, the only solution is to either free up some space or do an offline copy. Migration wont work, so copy is your only option.