Question about migration when creating VM

Kptainflintt

Hi,

I have two questions regarding creation of new VM from a custom template.

To understand :

Custom templates are on a network SR, shared between my three hosts
Resource set are on a local SR

When users launch several VM creation from one template, some of them have an error ("unknown error from the peer").
If I get logs, the real error is "TOO_MANY_STORAGE_MIGRATES"

And they have to try many times.

I see in this article that there is a queue, but it seems that in that situation, there is not.

When I migrate more than 3 VM from one host to another (and to another SR), I don't have this kind of error, and queue seem to work because when one of the three first VM finish, another start.

So, my questions :

Is creating from template and migrating an existing VM is different in "migration" meaning ?
I see that "Fast clone" is active by default, is it the best option in that situation (network SR for source, local SR for destination).

olivierlambert

Hi,

So when you create a new VM with "fast clone", that happens:

The template disk is used as a base (copy on write) and a new active disk is created to write all the diff for that new VM
As the target SR is somewhere else, then, XCP-ng must migrate that active disk and generate an entire non-differential disk to the destination SR.

When you migrate via XO, XO is smart enough to "intercept" the list of migration and creating a queue. Because XCP-ng can't do that (for now). But not doing VM creation, because… it wasn't really requested before.

2 options:

Get XO doing that
Ideally XCP-ng handling it transparently (sounds a better approach)

Let me add @Team-XAPI-Network for discussion about this

Kptainflintt

@olivierlambert Hi,

So, if I understand, VM creation is done, but XO have to create a copy of base disk and move it trough migration to local SR.

I think I didn't explain clearly, because don't understand the options provided.

Just to be clear, the user journey is :

Connect to XO
Create a VM and choose resource set with local SR (nearly 10 user at the same time)
Clic on "Create", seems to run, but fail with "unknown error from the peer"
With admin right, I can see an error of vm.create "TOO_MANY_STORAGE_MIGRATES" (get attention there is no number after in parenthesis)
User clic again and again, until creation is done.

I can effectively see 2 "base copy" disk of the VM wanted by users on the local SR, bur for the same template (called de13-xfce).
So I imagine that copy of base disk is made only once, and other VMs get their differential disk from it.
What I didn't understand is that when they create another VM from template (called deb13-nograph), there is no error and there is no base disk. However, fast clone is used here too...

Sorry for my misunderstanding, I'm pretty new to this kind of use with Xcp/XO (educational use) and I try to understand how it works.

olivierlambert

It's not XO, it's XCP-ng that has to move the VDI to the new SR, doing then a SR migration. It's not XO that is doing the operation.

XO is just asking XCP-ng to create a VM from this template and some info (I want a VM from this template, to be stored there etc.). Then XCP-ng is doing the actual work and migrating the freshly cloned VM to the destination SR.

If you can store your templates directly on the shared SR, the problem will disappear completely.

In your current situation, a fast clone is made (ultra fast) but then the disk has to be "rebuilt entirely" (not just the diff, you cannot diff between 2x SRs) on destination. So it takes time and cannot handle more than 3x disks migrated at once.

Kptainflintt

@olivierlambert

Right, I've now understand all of this stuff !
And since it's explained so well, I can also explain it to users (and especially teachers).

And I imagine that if I don't use "Fast Clone", it's a VM copy (not migrate), it will work but take much more time...

Thank you again.

EDIT : forget it, I've tested multiVM deployement fo the same template to local SR without Fast Clone. A VMcopy is made, and after that, an Async.VM.migrate_send to the local SR. I can see that only 3 migration are done at the same time, the fourth one is "blinking" in task list until one finish, and then start.

olivierlambert

So without fast clone you don't have the issue, right? But the creation time is similar?

Kptainflintt

@olivierlambert
Iiiiiiii can't really say this...

After the end of creation, I see only VM 1, 3 and 4... No VM2
In logs, again : "TOO_MANY_STORAGE_MIGRATES", seems VM 4 started migrating before VM2 ends...

So, you're right, put custom template on the same storage is the best solution.

olivierlambert

I think it is. An improvement would be a real queue on XCP-ng side (XAPI side to be precise). If it's too hard, then we'll see how to do that in XO for this use case

psafont

@olivierlambert Ideally XCP-ng (xapi) could add this to a queue, and wait for some time before cancelling the task because it took too long. This also needs some kind of feedback that can be given to the user / client, which I think currently is quite undercooked (how to report is waiting on other migrations to the same host when a client asks?). For the time I think XO being aware that it can retry the operation would be simpler, especially because it already has code to do it for other operations