VMware migration tool: we need your feedback!

chrispro-21

@brezlord said in VMware migration tool: we need your feedback!:

TypeError

I got something similar on the latest build of XO.
System: ESXi 6.7.0 Update 3 (Build 15160138)

vm.importFromEsxi
{
  "host": "10.20.0.105",
  "network": "1816a2e5-562a-4559-531d-6b59f8ec1a5c",
  "password": "* obfuscated *",
  "sr": "eb23cb22-5b8c-9641-b801-c86397e05520",
  "sslVerify": false,
  "stopSource": true,
  "thin": false,
  "user": "root",
  "vm": "9"
}
{
  "message": "Cannot read properties of undefined (reading 'stream')",
  "name": "TypeError",
  "stack": "TypeError: Cannot read properties of undefined (reading 'stream')
    at file:///home/node/xen-orchestra/packages/xo-server/src/xo-mixins/migrate-vm.mjs:272:30
    at Task.runInside (/home/node/xen-orchestra/@vates/task/index.js:158:22)
    at Task.run (/home/node/xen-orchestra/@vates/task/index.js:137:20)"
}

florent

@chrispro-21 we're not able to import VM with snapshots from esxi 6.5+ , for now

At least the next release will be able to ignore them or give a meaningfull error, if we have not finished the implementation of sesparse reading ( the disk snapshot format for esxi 6.5+)

alexredston

Hi @florent I'm getting the same on ESXi 6.7 - have tried migrating the disk storage within VMWare first to see if it updated. Looking forwards to testing this new feature as soon as disk format sorted - in the mean time will look for a workaround.

vm.importFromEsxi
{
  "host": "172.29.49.11",
  "network": "b4d861c9-a735-b21c-6f58-655911cf8fb2",
  "password": "* obfuscated *",
  "sr": "1216d3f2-a60d-e68b-ea93-0b1b594a53cb",
  "sslVerify": true,
  "stopSource": false,
  "thin": false,
  "user": "administrator@blue.local",
  "vm": "vm-4292"
}
{
  "message": "sesparse Vmdk reading is not functionnal yet gbr-stf-blue-dc/gbr-stf-blue-dc-000001-sesparse.vmdk",
  "name": "Error",
  "stack": "Error: sesparse Vmdk reading is not functionnal yet gbr-stf-blue-dc/gbr-stf-blue-dc-000001-sesparse.vmdk
    at openDeltaVmdkasVhd (file:///opt/xo/xo-builds/xen-orchestra-202303171405/@xen-orchestra/vmware-explorer/openDeltaVmdkAsVhd.mjs:7:11)
    at file:///opt/xo/xo-builds/xen-orchestra-202303171405/packages/xo-server/src/xo-mixins/migrate-vm.mjs:267:27
    at Task.runInside (/opt/xo/xo-builds/xen-orchestra-202303171405/@vates/task/index.js:158:22)
    at Task.run (/opt/xo/xo-builds/xen-orchestra-202303171405/@vates/task/index.js:137:20)"
}

olivierlambert

@alexredston you can workaround it by stopping the VM and that will work

alexredston

@olivierlambert Thanks - already underway.

3 hours elapsed, 3 hours to go 80GB thick provisioned primary domain controller, unfortunately it was on a slow network card 1G, averaging about 42 MB per second during the transfer. Will try the next one on 10G and will see what the speed is like.

RAID 1 on old server to RAID10 ADM triple mirrored and striped.

olivierlambert

Select thin on the XO V2V tool, that might be faster in the end. Patience is key

rochemike

@alexredston I don't know what "old server" means to you, but 42MB/se could be ~320Mbps which would be about the maximum on a SCSI bus. If I'm right and it's Ultra2 SCSI, there's little that you'll be able to speed up by increasing available network bandwidth.

https://en.wikipedia.org/wiki/Parallel_SCSI#:~:text=At 10 MHz with a,rate of 640 MB%2Fs.

Mike

alexredston

@rochemike SSD - so should go a little faster.

alexredston

@olivierlambert Trying another VM now - Linux this time, on ESXi it is 500GB thin provisioned with about 100GB in use - transfer died at around 20 hours + on thick, seemed to get nowhere on thin, but I'm not sure where to get error message to see what happened - just going to try again now after bringing XO up to date with master. Will try thin.

alexredston

@rochemike it did get there in the end - one down 28 to go

olivierlambert

\o/

Next release will have multi-VM import

florent

@olivierlambert yes
and the multi import will runVM two by two

stmboyd

Hello,

I am not even able to connect. When I enter the credentials for my host and hit connect, I get an error when XO tries to iterate over the data:

esxi.listVms
{
  "host": "10.64.54.4",
  "user": "root",
  "password": "* obfuscated *",
  "sslVerify": false
}
{
  "message": "Cannot read properties of undefined (reading 'perDatastoreUsage')",
  "name": "TypeError",
  "stack": "TypeError: Cannot read properties of undefined (reading 'perDatastoreUsage')
    at file:///opt/xo/xo-builds/xen-orchestra-202303302257/@xen-orchestra/vmware-explorer/esxi.mjs:211:55
    at Array.map (<anonymous>)
    at Esxi.getAllVmMetadata (file:///opt/xo/xo-builds/xen-orchestra-202303302257/@xen-orchestra/vmware-explorer/esxi.mjs:209:31)
    at Api.#callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202303302257/packages/xo-server/src/xo-mixins/api.mjs:401:20)"
}

Using XO from sources a41037833c960078d8509fd95e01aa0bb5a3670e
ESXi 6.5.0

Regardless of what the issue ends up being, is there maybe a better way of handling the flow? As in, when I click connect, can there be a landing page that says "connected", and allows for partial loading of info? Might be helpful, and will certainly be a better user experience.

olivierlambert

Either try on latest commit on master or XOA with latest to see if it's better
I don't understand the other part of your comment, if the error is during the connection, there's little more we can do

stmboyd

@olivierlambert

Did you even look at the commit I posted? When I ran this, it was 2 commits behind latest master. Was the commit of you updating a document supposed to make things better? How about the commit to Lite, was that really going to fix anything?
Please at least make some effort here, I posted a trace for a reason.

The failing line is this: https://github.com/vatesfr/xen-orchestra/blob/master/%40xen-orchestra/vmware-explorer/esxi.mjs#L211
For some reason, this property is not available on storage.
That code is called from here: https://github.com/vatesfr/xen-orchestra/blob/1942e55f763bd3d74c7cbc3dc1fc4c0ec070414c/packages/xo-server/src/xo-mixins/migrate-vm.mjs#L164

Look at that function for a second, and then re-read what I said in the earlier comment. Why does this function try and do two things? I'm saying it may be a good idea to split this into 2 parts. First, only connect, and then display a page in XO that shows the connection is live, and then after that query the data. This would make things a bit nicer.

More importantly, there needs to be some way for me to help figure out why the data that comes in is not in the format XO is expecting. That way we could solve a problem.

In the future, please don't make several blog posts and a huge thread asking for "feedback" and then tell me "there's little more we can do" when you have done nothing, it doesn't really feel encouraging.

olivierlambert

As per the documentation, it's asked to be sure you are on the latest commit on master. It solved countless of reports because people (in general) aren't following this simple rule. Since you are not more special than anyone else (there's dozen of new messages per day, I can't check each situation in details as we all offer assistance on our free time here), you should follow it. This is how it works here, and it's not personal, believe me
"Make some effort": this is a community forum and not a paid assistance here. You should adapt your attitude if you want help, because this is really not helping to be rude or ask for help while nobody owe you anything. Being polite is a far more rewarding strategy to have help from others!

Finally, we are fully open source, so you have also the opportunity to bring your knowledge and modify the code, even providing it via a pull request we'll be happy to read/comment and merge. https://xen-orchestra.com/docs/contributing.html for details.

We'll be happy to assist as soon you understand what this place is: a community forum where people assist on their free time and because they want to assist people. Not because they have to. We (I think I can talk for a large part of the community) do not accept people coming and asking for results. You can criticize and report bugs, as long it's constructive and following our doc. If you want a guarantee of efforts or results, you can have pro support with a service you pay for

I hope it's more clear, happy to assist you when it's understood

florent

@stmboyd Hi

I wrote most of the code for v2v migration. Please note that

there is a huge number of things happening in this error, and this error is probably caused earlier during the api call to the ESXI, which is probably linked to a configuration on your part that we should try to handle, or at least identify. And we'll try, but it will take times, to identify it
The real question, is why the storage is missing the soap api call should return it and the code should format it correctly . And that is why we ask for constructive feedback, making different productes talk together reliably is hard.
If the call to the api fail , the connect won't return anything usefull. If it succeed, we won't have much more to show ( maybe a "password/login seems ok ?) That will add one step for everybody , with , in my advice, little additionnal value, but I am not ready to die on this hill if there are other people advocating for it

As @olivierlambert said, as soon as we are working in the same direction to solves this, we'll do our possible.

pedroalvesbatista

No tested yet, as soon some hosts are fine after our PXE enrollments, will test this out.
Any scripts in the house to accelerate a little bit the process ?

florent

@pedroalvesbatista For now we're working on reliability and ease of use, performance won't be easy to improve. The main clue I have, would be to mount the dtastore of the esxi in XO, it may be faster than the HTTP API ( which does a surprising good job given the number of query we make )

But if there are ideas, I would be happy to discuss them.

DangerKlaus

Hi all,
I've hit an issue testing import and not seen the answer anywhere else so here goes.
Setup:
ESX 6.0
XO build from sources

Issue, when importing from vmware I get the following error

vm.importMultipleFromEsxi
{
  "concurrency": 2,
  "host": "x.x.x.x",
  "network": "7286bb98-7207-610c-0519-44bb366d7b46",
  "password": "* obfuscated *",
  "sr": "22c0b692-4837-d527-ec8f-3e258baabbf2",
  "sslVerify": false,
  "stopOnError": true,
  "stopSource": false,
  "thin": true,
  "user": "administrator@vsphere.local",
  "vms": [
    "vm-13048",
    "vm-784"
  ]
}
{
  "succeeded": {},
  "message": "Property description must be an object: undefined",
  "name": "TypeError",
  "stack": "TypeError: Property description must be an object: undefined
    at Function.defineProperty (<anonymous>)
    at Task.onProgress (/opt/xen-orchestra/@vates/task/combineEvents.js:51:16)
    at Task.#emit (/opt/xen-orchestra/@vates/task/index.js:126:21)
    at Task.#maybeStart (/opt/xen-orchestra/@vates/task/index.js:133:17)
    at Task.runInside (/opt/xen-orchestra/@vates/task/index.js:152:21)
    at Task.run (/opt/xen-orchestra/@vates/task/index.js:138:31)
    at asyncEach.concurrency.concurrency (file:///opt/xen-orchestra/packages/xo-server/src/api/vm.mjs:1372:58)
    at next (/opt/xen-orchestra/@vates/async-each/index.js:90:37)"
}

I've tried via host and vCenter ip, via thin and thick, snapshot and no snapshot. Also tried via command line.

Any ideas?