Problem with importing large VMDK disks
-
Hello,
I ran into a problem when importing VMDK disks using Xen Orchestra. Importing a large (100 GiB) VMDK disk runs out of RAM in the Debian virtual machine running Xen Orchestra. The import process basically ends and XO is unavailable or very slow and needs to be restarted. First I had a virtual with 4 GiB RAM for XO, then I tried to increase it to 8 GiB, but the problem happened again. When importing the VHD file (100 GiB as well), the problem does not seem to occur.
EDIT:
This is what the memory usage looks like when importing VHD - no problem, the import seems to complete fine:
-
To conclude this: @florent and I discussed the issue in a private conversation. I provided the vmdk file that was causing the problem. It turned out that the problem was caused by the vmdk thin file from VMware Workstation Pro, which has a slightly different structure than the vmdk files on ESXi hypervisors. Currently XO imports these files fine and the problem can be considered solved. Thank you very much @florent for your cooperation and help.
-
Ping @florent
-
You're importing the VMDK files themselves right, not migrating directly from ESXi? If migrating off ESXi, it may be worth trying the Import from VMware feature.
-
@planedrop These disk files belong to VMware Workstation Pro virtual machines, not ESXi. I converted the files to VHD as a workaround, but it would still be good to get a fix for XO.
-
Yes, that's not a normal behavior. @florent is pretty busy ATM because we are about to release XO 5.92 soon, but after that, he'll come here
-
@olivierlambert Thanks for the good news!
I've used VMware for years. ESXi as a server hypervisor and VMware Workstation Pro on the desktop. Even for some labs and test environments I got by with the free version of ESXi with limited APIs. The new licensing policy does not suit me and my companions at all and as a bonus Broadcom has cut the free version of ESXi. Bravo! I think Broadcom should get the award for best marketing of XCP-ng, H-V, Citrix etc. for 2024
On desktops, I need to keep VMware Workstation Pro for many reasons. That's why I was impressed by the videos @lawrencesystems made about XCP-ng & Xen Orchestra. The capabilities that are close to vSphere and vCentre, the ability to import and export VMDK so easily through XO blew me away. And to top it all off, transferring VMDK disks through XO works noticeably faster than migrating VMs between VMw WS and ESXi!
-
@abudef said in Problem with importing large VMDK disks:
@olivierlambert Thanks for the good news!
I've used VMware for years. ESXi as a server hypervisor and VMware Workstation Pro on the desktop. Even for some labs and test environments I got by with the free version of ESXi with limited APIs. The new licensing policy does not suit me and my companions at all and as a bonus Broadcom has cut the free version of ESXi. Bravo! I think Broadcom should get the award for best marketing of XCP-ng, H-V, Citrix etc. for 2024
On desktops, I need to keep VMware Workstation Pro for many reasons. That's why I was impressed by the videos @lawrencesystems made about XCP-ng & Xen Orchestra. The capabilities that are close to vSphere and vCentre, the ability to import and export VMDK so easily through XO blew me away. And to top it all off, transferring VMDK disks through XO works noticeably faster than migrating VMs between VMw WS and ESXi!
thank you
I tried exporting and importing a vmdk locally , and didn't reproduce the memory leak. I will try again with an other configuration ( a slower XCP ) -
@florent said in Problem with importing large VMDK disks:
thank you
I tried exporting and importing a vmdk locally , and didn't reproduce the memory leak. I will try again with an other configuration ( a slower XCP )How big was the VMDK file you tried to import? Did the RAM usage grow during the import, as seen in the screenshot above?
In my case, the import was from a pc from which I was connected to XO, whose virtual was running directly on the XCP-ng server into whose local storage the file was being imported.
-
@abudef I tested with a 50GB file, and the RAM is not changing notably
-
@florent I reproduced the problem on a different environment. Try using a really large file or importing two at once.
-
@abudef ouch
Would you be able to give me access to this vmdk file ? If you have a XOA, and with your authorization, I could download it through ssh and check here -
To conclude this: @florent and I discussed the issue in a private conversation. I provided the vmdk file that was causing the problem. It turned out that the problem was caused by the vmdk thin file from VMware Workstation Pro, which has a slightly different structure than the vmdk files on ESXi hypervisors. Currently XO imports these files fine and the problem can be considered solved. Thank you very much @florent for your cooperation and help.
-
O olivierlambert marked this topic as a question on
-
O olivierlambert has marked this topic as solved on
