Creating a VM using only the XAPI CLI...

epretorious

I'm attempting to create a CentOS-7 VM from the CLI using modified instructions from the "Citrix XenServer 7.2 Virtual Machine User's Guide":

# NET_UUID=$(xe network-list bridge=xenbr0 params=uuid | awk '{ print $5 }')
# echo $NET_UUID
47a6223e-74d0-b75d-bec3-0ca6324b498c

# VM_UUID=$(xe vm-install template=CentOS\ 7 new-name-label=CentOS7-1)
# echo $VM_UUID
26859adf-f2f9-77d6-3f3c-719ecd885535

# VIF_UUID=$(xe vif-create mac=random device=0 vm-uuid=$VM_UUID network-uuid=$NET_UUID)
# echo $VIF_UUID
decb158e-2961-7448-8b7c-4c2707298604

# xe vm-param-set uuid=$VM_UUID other-config:install-repository=nfs://192.168.0.95/srv/images/ISO/CentOS-7.7-x86_64-Minimal.iso
# xe vm-start uuid=$VM_UUID

But when I use the console command in order to complete the installation of the operating system: I get this...

# xe console uuid=$VM_UUID
No text console available

And when I search Xenstore for the VNC-port information I get this:

# xe vm-param-list uuid=$VM_UUID | grep dom-id
                                dom-id ( RO): -1

What have I done wrong? How can I determine the root cause of the failure?

TIA,
Eric Pretorious
Reno, Nevada

epretorious

FWIW: The result seems to be the same whether I use an ISO image on a remote NFS server (nfs://192.168.0.95/srv/images/ISO/CentOS-7.7-x86_64-Minimal.iso) or I use a remote package repository (nfs://192.168.0.95/srv/pkgs/ISO) for installation:

# VM_UUID=$(xe vm-install template=CentOS\ 7 new-name-label=CentOS7-2)
# echo $VM_UUID
c9c66b15-e618-dc12-0f93-8e39af102c8b

# VIF_UUID=$(xe vif-create mac=random device=0 vm-uuid=$VM_UUID network-uuid=$NET_UUID)
# echo $VIF_UUID
de6b51ec-54ed-fa0c-0ba2-7da732e8fa05

# xe vm-param-set uuid=$VM_UUID other-config:install-repository=nfs://192.168.0.95/srv/pkgs/ISO

# xe vm-start uuid=$VM_UUID

# xe console uuid=$VM_UUID
No text console available

# xe vm-param-list uuid=$VM_UUID | grep dom-id
                                dom-id ( RO): 8

# xenstore-ls /local/domain/8
xenstore-ls: xs_directory (/local/domain/8): No such file or directory

What am I doing wrong?

TIA,
Eric P.
Reno, Nevada

epretorious

FWIW: These are the contents of the remote package repository (nfs://192.168.0.95/srv/pkgs/ISO)

# mount -t nfs 192.168.0.95:/srv/pkgs/ISO /mnt

# ll /mnt
total 110
-rw-rw-r-- 1 root root    14 Sep  9  2019 CentOS_BuildTag
drwxr-xr-x 3 root root  2048 Sep  6  2019 EFI
-rw-rw-r-- 1 root root   227 Aug 30  2017 EULA
-rw-rw-r-- 1 root root 18009 Dec  9  2015 GPL
drwxr-xr-x 3 root root  2048 Sep  9  2019 images
drwxr-xr-x 2 root root  2048 Sep  9  2019 isolinux
drwxr-xr-x 2 root root  2048 Sep  6  2019 LiveOS
drwxrwxr-x 2 root root 73728 Sep 11  2019 Packages
drwxrwxr-x 2 root root  4096 Sep 11  2019 repodata
-rw-rw-r-- 1 root root  1690 Dec  9  2015 RPM-GPG-KEY-CentOS-7
-rw-rw-r-- 1 root root  1690 Dec  9  2015 RPM-GPG-KEY-CentOS-Testing-7
-r--r--r-- 1 root root  2883 Sep 11  2019 TRANS.TBL

HTH,
Eric P.
Reno, Nevada

olivierlambert

After taking a quick look: have you checked to get all the hidden files with it? It's a classic problem.

Also, is it a PV guest?

epretorious

olivierlambert

It's a CentOS-7 HVM guest.
What are these hidden files that you are referring to?

olivierlambert

So it's an HVM guest, I'm not aware of a possibility to pass a key to the guest kernel like you could do with PV. I would go for netboot of the VM with a TFTP server and then some automation on that side.

Alternatively, a template with Cloudinit. Or even maybe something with Packer.

epretorious

olivierlambert I don't understand. Would you please elaborate?

TIA,
Eric P.
Reno, Nevada

olivierlambert

Sorry, missing words in my previous post

The current trend to install OS and distro is now more going to some tools, like Packer and Terraform (for example).

The "easier" way (less tools to learn) is to create your own template (eg install a CentOS VM, remove the root password, install cloudinit etc.) and then convert the VM into a template. From there, you can rely on Cloudinit feature in Xen Orchestra to do whatever you want with this "default" template. See https://xen-orchestra.com/docs/advanced.html#cloud-init

epretorious

olivierlambert Thanks. But I'm not looking for the most current method of installing a guest - I'm looking for the simplest method of installing a guest using XAPI on the command-line interface (CLI).

The "Citrix XenServer 7.2 Virtual Machine User's Guide" includes a method to do that using the CLI:

xe vm-install template=<template> new-name-label=<name_for_vm> sr-uuid=<storage_repository_uuid>

xe vif-create vm-uuid=<vm_uuid> network-uuid=<network_uuid> mac=random device=0

xe vm-param-set uuid=<vm_uuid> other-config:install-repository=<network_repository>

xe vm-start uuid=<vm_uuid>

But obviously it's missing something!

Have you got any suggestions about what might be missing from those four simple steps?

TIA,
Eric P.
Reno, Nevada

olivierlambert

I think it was possible because in 7.2, PV wasn't deprecated and the example provided was for a PV guest.

I don't think "install-repository" works for an HVM guest, which is now the default. But I can try to see if it's truly the code by taking a look at XAPI.

So it's possible that the simplest method isn't the one you are using

olivierlambert

Okay I had the confirmation from a XAPI dev: this parameter is only parsed by eliload, which is only used for PV guest and that will be even removed soon from XenServer/XCP-ng (see https://github.com/xapi-project/xen-api/pull/4837)

I'm not yet certain this parameter will work with Pygrub (the "non-deprecated" boot loader for PV guests) but yet, it will NOT work on HVM guests anyway.

edwintorok opened this pull request in xapi-project/xen-api

draft CP-41067: Drop eliloader bootloader: obsolete and not needed on any s… #4837

epretorious

olivierlambert Is there any way to bootstrap an HVM guest using only XAPI on the CLI?

TIA,
Eric P.
Reno, Nevada

olivierlambert

With xe only, I'm not sure, no. You'll need a better client to do that.

epretorious

olivierlambert said in Creating a VM using only the XAPI CLI...:

You'll need a better client to do that.

I don't understand. Would you please elaborate?

TIA,
Eric P.
Reno, Nevada

olivierlambert

xe is just a XAPI client. But it's relatively basic and doesn't support (for example) Cloudinit, which could be helpful to bootstrap templates with variables.

I mean, you can use xe to clone a template of a already installed CentOS, but you won't be able to change the hostname or anything beyond just cloning.

Network boot on fully automated TFTP and such can do the rest for you (after initial boot), but that's also beyond just using xe, you need something to do the rest of the work. There's many ways, but not just setting a value in the VM anymore.

epretorious

What does tjkreidl have to say about bootstrapping an HVM guest using only XAPI on the CLI?

Eric P.
Reno, Nevada

tjkreidl

epretorious Not sure that's an option. I think you'd be better off to just clone an existing VM and use scripts to modify it to transform it into the desirable VM.

olivierlambert

It is the way it is, things changed. HVM is using emulation on boot (fake BIOS), so you can't directly talk to the VM kernel.

However, you might want to dig a bit on UEFI side, I don't know if it's possible to tinker with that (I'm not a specialist, UEFI is vast and complex).

In any case, I would sort the various approaches in 2 categories:

Fresh VM install from scratch (no previous disk): you can use a network install, Packer, a specific ISO to make the install fully automated and probably other things (not an exhaustive list)
Clone an existing template (VM with OS already installed): Cloudinit to inject the data you want (you can use XO CLI to do it), Terraform XO plugin, custom scripts like using xenstore (that's what we use for XOA deploy from the CLI) as long as your template is reading them on boot, and so on

All in all, that's why using "better" (more complete) tooling on top of XCP-ng/XenServer makes sense, like Xen Orchestra, soon XO Lite too.
Packer/Terraform is also a good combo if you want automation/infrastructure as code.

I know, it's more stuff to learn, but worth it. And finally, a side note to understand why the stack is important as a "whole": we couldn't convince VMware users to make the switch if we were only "selling" XCP-ng. What makes the difference is to get the whole thing (XCP-ng+Xen Orchestra) together

tjkreidl

olivierlambert I'd vote for Option 2. Much easier IMO to take a VM or VM template and modify it.

olivierlambert

Agreed, but the downside is to update the template from time to time. We'll probably create a new Packer plugin to work with XO API, so it will be easier to use and rely on (Packer goal is to prep' a template from scratch and up to date for you, that you can use for solution 2. then)