What is the status/roadmap of V2V (Migrating from VMware to XCPng/XO) ?

[afk]

@olivierlambert Essentialy yes, though it would be great to have a recap of the current situation.

[olivierlambert]

This would require an investigation because since all recent VMFS versions, we cannot access the snapshot data anymore. It could be helpful to check if there's other solutions able to do warm migration with VMware since the API approach seems a dead end due to this change

[afk]

@olivierlambert Thanks for the feedback.

Is the limitation only due to VMFS or both the esxi version and VMFS ? Because vsphere8 still supports VMFS5 and we could imagine a 2-step migration by manually moving VMs on a temporary datastore. However, if the issue is the API change with vsphere8 then I understand that it would be difficult indeed.

I'm sure the dev team has already explored the subject to build V2V in the first place but just in case it could help, here are the relevant veeam and vmware docs for vmdks transport modes (V2V is NBD mode if i'm not mistaken):

https://helpcenter.veeam.com/docs/backup/vsphere/transport_modes.html?ver=120
https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vddk-programming-guide/GUID-15395099-5300-4D3F-BCC3-E50DCDC954C2.html

I imagine building a viable alternative is quite a big project in itself.

[olivierlambert]

Food for thoughts @florent

[florent]

@afk the newer VMFS put more lock on the files, locking the full chain of snapshot and base disks instead of locking only the active disk.
Even VMFS5 sometimes lock the full chain.
The more open storage is a NFS, that can additionally be accessed directly by XO, not going through the soap api , giving anice performance boost and not hammering the esxi too much ( only for the metadata )

We don't intend to migrate the VM running in vmware from XO automatically. A user could script it though, by using a combination of the vmware apis and xo-cli.

We never succeed in getting the disks though NBD, but it should be possible ( https://github.com/vexxhost/migratekit/blob/main/internal/vmware_nbdkit/vmware_nbdkit.go#L91 ) and ( https://vdc-download.vmware.com/vmwb-repository/dcr-public/8ed923df-bad4-49b3-b677-45bca5326e85/d2d90bb6-d1b3-4266-8ce5-443680187a9a/vim.vm.device.VirtualDevice.BackingInfo.html )
What we are mostly missing here, is internal knowledge on the Vmware side : how to get the NBD server address,how to authenticate and how to get the exportname of a disk through the soap API .

Since we already have the knowledge of using NBD to read massive volume of data as long as we can connect.

[afk]

Thanks for the details @florent

@florent said in What is the status/roadmap of V2V (Migrating from VMware to XCPng/XO) ?:

the newer VMFS put more lock on the files, locking the full chain of snapshot and base disks instead of locking only the active disk.
Even VMFS5 sometimes lock the full chain.

That explains why I had locking issues trying to restart the source VM on vmware after a migration test.

I'll see if I can find anything on how to use NBD with vmware.

[olivierlambert]

Thanks @afk ! Your knowledge on the "other side" might be very helpful for us!

[olivierlambert]

@afk Have you found anything ?

[afk]

Hi @olivierlambert @florent ,

I didn't have much time to work on this in the last weeks but I finally could dig deeper thanks to the migratekit repo.

Essentially, they are delegating all the work to nbdkit and its vddk plugin (https://gitlab.com/nbdkit/nbdkit and https://libguestfs.org/nbdkit-vddk-plugin.1.html) by spawning an external process (https://github.com/vexxhost/migratekit/blob/a08325d420733e4eb26331d87bf6ef46d8cccd7f/internal/nbdkit/builder.go#L82).

The authentication info is simply the authentication to vCenter/ESXi provided by the end-user if I'm not mistaken and the filename given to nbdkit is indeed gathered from the VirtualDeviceBackingInfo property. They are using the govmomi auto-generated library for this.

For instance, on a snapshot of one of our VMs:

You can see the property path at the top and the fileName property contains the "[datastore-name] filepath" string.

The "device[2000]" part of the path is from the list of devices attached to the VM that can also be accessed following the snapshot moref:

Migratekit is then filtering on the VirtualDisk type in the device list.

Now, the problem in this setup is that nbdkit is using VDDK directly, but the development kit cannot be redistributed without a licence agreement from Broadcom: https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-sdks-tools/8-0/virtual-disk-development-kit-programming-guide/the-virtual-disk-api-and-vsphere/developing-for-vmware-platform-products/redistributing-vddk-components.html

The user would have to download and install VDDK manually.

I hope this helps and let me know if you need more details on all this. I played a bit with pyvmomi 5+ years ago but I never used the SOAP API "manually" though.

[olivierlambert]

Thanks, that's very useful info! @florent will take a look when he can!