VMware migration tool: we need your feedback!

florent

@severhart the sesparse format is used for disk greater than 2TB , and for all disks after esxi 6.5 , and it's not very documented
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.storage.doc/GUID-88E5A594-DEBC-4662-812F-EA421591C70F.html
we are working on implementing this reader to allow migration during Q1 2023

For now esxi 6.5+ vm are limited to migration of sopped vm without snasphots

severhart

@florent thanks! for the fast reply, I will take the outage and rerun, and let you know outcome.

olivierlambert

Also, 2TiB+ disk can't be imported since it's limited to 2TiB tops on XCP-ng default storage stack.

planedrop

@olivierlambert Would love to see some way to have it import larger than 2TiB disks as multiple disks in XCP-ng, since most OSes just let you span drives anyway.

Just realized I may not be able to leave VMWare with this method since one of the disks on a VM I'm trying to move is over 3TiB.

olivierlambert

We could detect 2TiB+ drives and create a raw disk on our side, but it won't support snap nor live storage migration.

Only another format (used in SMAPIv3) will allow us to solve this.

planedrop

@olivierlambert Gotcha, this makes sense.

Is there any way to skip a specific drive with this migration script? I'm thinking I could skip the larger than 2TiB disk and then just create 2 x 2TiB disks after migration, span them in Windows, and then copy the data manually from the VMWare VM.

olivierlambert

I think we could probably skip it (since it's likely not a system disk) so you can then manually copy the rest the way you prefer. We should probably add an option like "just skip 2TiB+ disk without failing"

planedrop

@olivierlambert Yes, I think that would be great, this would be a good workaround for people that have larger than 2TiB disks.

planedrop

@olivierlambert Also, do you know if the disk is OVER 2TiB thick provisioned but actual data usage on it is like 1TiB, will the script still fail or will it just create the 1TiB disk?

olivierlambert

The problem by doing that automatically is you can have bad surprises. We'll probably just skip it after adding the option.

severhart

@florent none of the drives are over 2tb, largest is 900GB, so I will assume it is due to snapshots?

Drives are
127GB
900GB
325GB
250GB
500GB

olivierlambert

@severhart as Flo said, it's because of your VMware version (6.5+) that is using another "diff" format not yet supported. In your case, you should do a cold migration for now, until we support this diff format

magicker

Total noob here jumping in at the deep end.

I get

 xo-cli vm.importFromEsxi host=xxx.xxx.xxx.xxx user=w...w password='u .... l' sslVerify=false vm=16 network=a1044bf9-4c06-8ae0-060c-e3462dd4524f sr=9b465ed4-e6d2-7a67-b5e0-5edc4915adac stopSource=true thin=true  

✖ Cannot read properties of undefined (reading 'stream')
JsonRpcError: Cannot read properties of undefined (reading 'stream')
    at Peer._callee$ (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/json-rpc-peer/dist/index.js:139:44)
    at tryCatch (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:44:17)
    at Generator.<anonymous> (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:125:22)
    at Generator.next (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:69:21)
    at asyncGeneratorStep (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/asyncToGenerator.js:22:9)
    at /opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/asyncToGenerator.js:27:7
    at new Promise (<anonymous>)
    at Peer.<anonymous> (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/@babel/runtime/helpers/asyncToGenerator.js:19:12)
    at Peer.exec (/opt/xo/xo-builds/xen-orchestra-202302081722/node_modules/json-rpc-peer/dist/index.js:182:20)

olivierlambert

Hey @magicker can you give us more details? Are you using XOA or XO from the sources? At which version? Also on VMware side, what's your ESXi version?

magicker

@olivierlambert said in VMware migration tool: we need your feedback!:

XO from the sources

XO from github (2 days old)

esxi .. 7.0.0 (Build 16324942)

olivierlambert

So the diff for warm migration isn't supported on ESXi 7.0 (yet!). So your VM must be halted first (on the VMware side I mean)

florent

hi @magicker

you should also have additionnal logs in your xo logs

Thank you for testing it

regards

magicker

@olivierlambert ah I see.. other than that!! it works.. Just like magic!! v cool

rochemike

Executing the following command resulted in an error until I powered the VM off:

ESXi host ==> VMware ESXi, 6.5.0, 19092475

Any ideas?

[11:02 10] xoa@xoa:~$ xo-cli vm.importFromEsxi host=xxx.xxx.xxx.xxx user=root password=secret sslVerify=false vm=262 network=13d8ab8a-dfdc-1e5c-0e35-0028af26987a sr=e748751e-02fd-28ae-5fa9-d58f5f0dc50a stopSource=true thin=true
✖ Cannot read properties of undefined (reading 'stream')
JsonRpcError: Cannot read properties of undefined (reading 'stream')
    at Peer._callee$ (/usr/local/lib/node_modules/xo-cli/node_modules/json-rpc-peer/dist/index.js:139:44)
    at tryCatch (/usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:44:17)
    at Generator.<anonymous> (/usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:125:22)
    at Generator.next (/usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:69:21)
    at asyncGeneratorStep (/usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/asyncToGenerator.js:22:9)
    at /usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/asyncToGenerator.js:27:7
    at new Promise (<anonymous>)
    at Peer.<anonymous> (/usr/local/lib/node_modules/xo-cli/node_modules/@babel/runtime/helpers/asyncToGenerator.js:19:12)
    at Peer.exec (/usr/local/lib/node_modules/xo-cli/node_modules/json-rpc-peer/dist/index.js:182:20)
[11:02 10] xoa@xoa:~$

olivierlambert

Same answer than my previous post: since ESXi 6.5, there's a new diff algorithm @florent is working on it, but it's even more complicated than the "legacy" one