@Danp As soon as I read that, I thought "I think I've been down this road before." Sure enough. I gave it 2G of RAM and it wouldn't build. I gave it 4G of RAM and I watched using systat and that final step went to about 52% of RAM usage. It has completed. I routinely run Xen Orchestra on a 1G VM because it's rarely used and it runs fine that way. But building it clearly takes a lot more RAM. I'll update my internal notes. It might be worth it to update the building from source docs to mention having enough RAM.

Thanks for hitting the nail on the head with that advice!

@olivierlambert You expect a system administrator to read documentation!?

Anyways, that was it. Sorry I didn't look more carefully. I rebuilt using node8 and poof it was running. Thanks guys.

I'm willing to contribute some documentation on templates, because I keep looking up how to work with them. Every so often another person posts some basic questions about them, and there isn't one central place to send them.

What I wanted to do was collect the basics of what I know here in this thread, and give people an opportunity to add other things they think should be in a documentation page on templates, and then I'll go write the page and contribute it to the docs.

My thinking is to organize it around "CRUD": Create, Read, Update, Delete.

Create

There's basically 3 ways to create templates.

1. From an existing VM. For example, you can install Linux or FreeBSD or whatever on your VM at some basic level with a few customizations, and then make a template from that.
2. Duplicate one of the default templates and then modify it (subject to the limitations in "Update" below)
3. Create a VM, then convert it to a template. The only difference between this and #1 above is that if you never boot it or install any OS, you create a template that has an unformatted disk. This just gives you customized RAM, CPU and other values. See the section on "Delete" where I talk about diskless templates. Am I understanding it right?

Any other useful ways to create that I haven't mentioned?

Read

How can I see the properties of a template in Xen Orchestra? If I click the hamburger menu, all I can see are tags, CPU and RAM. If I want to see other values, it seems like I have to start to create a VM, choose the template, and then click on the "Advanced Settings" to see what the template sets them to.

Pull from the API? This forum post mentions that. Is that this documentation on the xo command or something else? What should I link to for more information on invoking the API?

From the CLI you can get all the information from a template. If I've created a template called My Debian Template, this is how to see its values.

xe template-list name-label="My Debian Template"
uuid=$(xe template-list name-label="My Debian Template" --minimal)
xe template-param-list uuid=$uuid | less

Update

I can't find a way in Xen Orchestra to modify the parameters of a template. There's a thread here that talks about finding it, clicking the hamburger menu, then editing the values. When I do that, I see a little green check mark next to the value I changed, but the values don't actually change. Is this meant to work? And those are really basic settings. What about changing advanced settings in Xen Orchestra?

I guess there's a cumbersome way to modify a template in XO: I could create a VM from my template, modifying the values to what I want them to be, and then convert that VM to a new template. Then I delete the old template? Does that work?

The only way I have found to update a template is to set values from the command line following this post. For example (assuming $uuid is set from my prior example) this will set memory-static-min to 512Mb.

xe template-param-set memory-static-min=$((1024 * 1024 * 512)) uuid=$uuid

Any other ways to update the properties of an existing template?

Delete

It seems simple enough to find a template in Xen Orchestra, select the box next to it, and click the trash can. According to this post, that will leave orphan VDIs that have to be deleted manually. But this is not true of the default templates, right? So:

1. Deleting a default template: you just find it in the GUI and delete it.
2. Deleting a custom template you created: you need to do the steps in that post: find the drives, delete them, then delete the template?

This also leads me to ask whether it's possible to create a template like the default templates, where there's no associated disk? I guess you just create a VM without a disk and turn that into a template?

Also, you can delete the default templates, but they get re-installed again during a major version upgrade, right? So you either live with them, or you have to delete them again after every major upgrade?

Cloud-init and Cloudbase-Init

There's some useful docs on cloud-init and cloudbase-init that could move to this page if we have a whole page devoted to working with templates.

Thoughts?

What Am I Missing?

This is as much as I know right now. Anything else we should add to docs on templates?

@julien-f Yeah, I have just found the useSudo option in the docs. I swear things change fast enough on xen orchestra that "upgrading" almost doesn't make sense. I rebuild it like every 3-6 months and so much changes that it's almost easier to start over. If I had started over and pretended I didn't know anything, I would have read the documentation and seen the useSudo option. That's not a complaint. Fast development and adding features is awesome.

I'm willing to contribute some documentation on templates, because I keep looking up how to work with them. Every so often another person posts some basic questions about them, and there isn't one central place to send them.

What I wanted to do was collect the basics of what I know here in this thread, and give people an opportunity to add other things they think should be in a documentation page on templates, and then I'll go write the page and contribute it to the docs.

My thinking is to organize it around "CRUD": Create, Read, Update, Delete.

Create

There's basically 3 ways to create templates.

1. From an existing VM. For example, you can install Linux or FreeBSD or whatever on your VM at some basic level with a few customizations, and then make a template from that.
2. Duplicate one of the default templates and then modify it (subject to the limitations in "Update" below)
3. Create a VM, then convert it to a template. The only difference between this and #1 above is that if you never boot it or install any OS, you create a template that has an unformatted disk. This just gives you customized RAM, CPU and other values. See the section on "Delete" where I talk about diskless templates. Am I understanding it right?

Any other useful ways to create that I haven't mentioned?

Read

How can I see the properties of a template in Xen Orchestra? If I click the hamburger menu, all I can see are tags, CPU and RAM. If I want to see other values, it seems like I have to start to create a VM, choose the template, and then click on the "Advanced Settings" to see what the template sets them to.

Pull from the API? This forum post mentions that. Is that this documentation on the xo command or something else? What should I link to for more information on invoking the API?

From the CLI you can get all the information from a template. If I've created a template called My Debian Template, this is how to see its values.

xe template-list name-label="My Debian Template"
uuid=$(xe template-list name-label="My Debian Template" --minimal)
xe template-param-list uuid=$uuid | less

Update

I can't find a way in Xen Orchestra to modify the parameters of a template. There's a thread here that talks about finding it, clicking the hamburger menu, then editing the values. When I do that, I see a little green check mark next to the value I changed, but the values don't actually change. Is this meant to work? And those are really basic settings. What about changing advanced settings in Xen Orchestra?

I guess there's a cumbersome way to modify a template in XO: I could create a VM from my template, modifying the values to what I want them to be, and then convert that VM to a new template. Then I delete the old template? Does that work?

The only way I have found to update a template is to set values from the command line following this post. For example (assuming $uuid is set from my prior example) this will set memory-static-min to 512Mb.

xe template-param-set memory-static-min=$((1024 * 1024 * 512)) uuid=$uuid

Any other ways to update the properties of an existing template?

Delete

It seems simple enough to find a template in Xen Orchestra, select the box next to it, and click the trash can. According to this post, that will leave orphan VDIs that have to be deleted manually. But this is not true of the default templates, right? So:

1. Deleting a default template: you just find it in the GUI and delete it.
2. Deleting a custom template you created: you need to do the steps in that post: find the drives, delete them, then delete the template?

This also leads me to ask whether it's possible to create a template like the default templates, where there's no associated disk? I guess you just create a VM without a disk and turn that into a template?

Also, you can delete the default templates, but they get re-installed again during a major version upgrade, right? So you either live with them, or you have to delete them again after every major upgrade?

Cloud-init and Cloudbase-Init

There's some useful docs on cloud-init and cloudbase-init that could move to this page if we have a whole page devoted to working with templates.

Thoughts?

What Am I Missing?

This is as much as I know right now. Anything else we should add to docs on templates?

@nikade Short answer is that the spikes seem to have smoothed out. I'm not really sure what happened. I don't have them at any resolution now. Perhaps some kind of heisenbug.

@olivierlambert Nothing dramatic. Just things "feeling slower than usual." That's qualititative. Just a gut hunch. So that's why I'm asking about ways to test or investigate.

@nikade You misunderstand. I am calling them "spikes", which most people use that word to mean things going dramatically up. Your graph shows spikes going up. Look closely at my graph, including the point I have highlighted. These "spikes" are 0. That is, it's humming along and then suddenly there's a spike down to 0. That's unlikely because the system is plenty active. So your graph shows spikes up, which is normal. Mine is showing occassional drops to 0, which is what I'm asking about.

In my throughput/IO graphs on Xen Orchestra, I'm seeing these spikes of 0 IOPS, 0 throughput, 0 latency, 0 IOWait. What I don't know is whether this is some kind of graphing artifact (i.e., it's not really happening, but the graph makes it look like it's happening), or whether I'm experiencing some kind of underlying hard disk throughput issue.

I'm in the middle of migrating a VM and it's estimating 12 hours to move 100G, which is long. I moved a couple 20G VMs this morning at 15-20 minutes each.

My question is, where could I look in XCP-ng logs to investigate? How could I get a sense of whether I'm really running into problems here? I've looked at these graphs many times over the years, and these spikes are new. This hasn't always looked like this.

Any thoughts?

The underlying storage is a LSI 9266-8i SAS RAID controller with 3 hard disks in a RAID5 array (hardware raid). I don't think I'm experiencing any other major issues, but maybe I am and don't know how to see it?

I'm running XCP-ng 8.2.1 and Xen Orchestra built from sources (well, from ronivay's Docker container) Xen Orchestra, commit e75cc Master, commit 70014.

@olivierlambert I appreciate the explanation. That suggests that there are no updates to 8.0.0? Or 8.1.0? 8.1.0 is 2 years old and EOL. I guess that's what EOL means: no more updates.

Not sure if this is expected behaviour or not. In the past, when I went to the Patches tab on my XCP-ng hosts, I would see patches listed here and I could choose to apply them. Now, I see none.

I have 2 hosts: one is running xcp-ng 8.0.0, the other is running 8.1.0. (They are single-host pools). In both cases, I see the green check mark and "host is up to date". There are no patches listed.

For the 8.0.0 host, I followed the upgrade from CLI instructions. The yum update command finished successfully and now it shows up in Xen Orchestra as running 8.2.1. But I was expecting Xen Orchestra to offer some amount of updating/patching/upgrading.

I haven't made any changes to the 8.1.0 host because I wanted to see if there was something I need to fix.

Do I misunderstand what I should see?

I'm running xo-server 5.89.0 and xo-web 5.94.0 from the sources via a Docker container.

Thanks,
Paco

@olivierlambert The proxy page has a pretty decent diagram for what things look like when there is no proxy. But it doesn't have a diagram for what things look like if there IS a proxy. I'm trying to understand it. Does this diagram capture the two situations?

If you're running 2 sites using a single instance of Xen Orchestra, is this what happens?

NodeJS is not my best language. But it looks _mount.js uses the execa library to wrap around standard child-process stuff.

I did a little bit of code reading and it looks like, ultimately, execa and child-process-promise end up invoking child_process.spawn(), and it seems like spawn() is just an asynchronous wrapper around child_process.exec(). This leads me to believe that the input ultimately ends up on a shell command line.

According to the docs I found:

child_process.exec(): spawns a shell and runs a command within that shell, passing the stdout and stderr to a callback function when complete

Now, both execa and the child-process-promise library do a lot of parsing of arguments. I can't tell to what extent that parsing and marshalling of command line arguments actually defeats injection attacks. I've done a lot of code review in my day, and sometimes I miss the line that does the work of defeating injection attacks because it is terse and effective. I might have looked right at it and misunderstood its importance.

Like I said, NodeJS and all the modules that are getting loaded in here to make this happen is difficult for me to follow. It's not my strong suit. Sorry if I'm wasting your time because I can't follow the code flow. It looks to me like it all boils down to exec(), and exec() by definition invokes the shell. But maybe we're protected by one of the argument marshalling functions in the call chain.

@julien-f Yeah, I have just found the useSudo option in the docs. I swear things change fast enough on xen orchestra that "upgrading" almost doesn't make sense. I rebuild it like every 3-6 months and so much changes that it's almost easier to start over. If I had started over and pretended I didn't know anything, I would have read the documentation and seen the useSudo option. That's not a complaint. Fast development and adding features is awesome.