Ubuntu 22.04.5 custom template - additional steps missing from documentation

BSmithITGuy

If you are trying to create an Ubuntu template and are having trouble on XCP-NG 8.3 (some or all probably applies to previous versions), keep reading.
I followed the documentation titled: Create and use custom XCP-NG templates: a guide for Ubuntu
At the following link:
https://docs.xcp-ng.org/guides/create-use-custom-xcpng-ubuntu-templates/

I downloaded the 22.04.5 server iso (barely newer than the version on this site). I used the minimal install. The process did not work 100% and had 3 issues with the official documentation:

The machine-ID does not change after creating a new VM from the template
When you go to clone the template, it will not apply the Network Config - it will be on DHCP
The YAML format for "Network Config" is incorrect

Here is a summary of the steps from the official documentation above, which I did and are mostly necessary (some of these files/directories do not exist, but still OK to run this all to be sure):

sudo apt update && sudo apt upgrade
sudo apt install xe-guest-utilities
sudo apt install cloud-init
sudo apt install cloud-initramfs-growroot
sudo dpkg-reconfigure cloud-init

NOTE: here you want to select NoCloud, ConfigDrive, and OpenStack

sudo rm -f /etc/cloud/cloud.cfg.d/99-installer.cfg
sudo rm -f /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg
sudo rm -rf /var/lib/cloud/instance
sudo rm -f /etc/netplan/00-installer-config.yaml

I had to run the following in addition to what was mentioned in the documentation to get the template to generate a new machine-id, to get it to apply the static IP address when creating a VM from the template, and also clear the cloud-init logs for further troubleshooting:

sudo rm -rf /var/lib/cloud/instances /var/lib/cloud/instance
sudo rm -rf /var/log/cloud-init.log /var/log/cloud-init*
sudo rm -f /etc/netplan/50-cloud-init.yaml
sudo rm -f /etc/cloud/cloud.cfg.d/90-installer-network.cfg
sudo truncate -s 0 /etc/machine-id

Then you will shutdown and create the template.

You want to use this format for the "Network Config" YAML:

#cloud-config
network:
  version: 2
  ethernets:
    eth0:
      dhcp4: false
      addresses:
        - 10.0.2.6/27
      gateway4: 10.0.2.1
      nameservers:
        addresses:
          - 10.0.2.1
          - 1.1.1.1

Make sure your IP is in CIDR notation.

If you have any trouble, in /var/log you want to look through the cloud-init.log to start, and there is another cloud-init log in there.

olivierlambert

Pinging @thomas-dkmt

BSmithITGuy

@olivierlambert That could be cleaned up a bit; I wouldn't just copy and paste that. I just hope my sloppy notes help someone else. I spent a few hours figuring this out lol.

olivierlambert

Sure, but it's nice to have your feedback because if we improve that page, we are saving hours for others after you, which is exactly the point of having a great community!

Thanks for sharing it with us!

BSmithITGuy

This post is deleted!

bvitnik

@BSmithITGuy said in Ubuntu 22.04.5 custom template - additional steps missing from documentation:

If you are trying to create an Ubuntu template and are having trouble on XCP-NG 8.3 (some or all probably applies to previous versions), keep reading.

Very nice. Very thorough. This inspired me to share a relevant part of my cleanup script. Maybe it will be of some additional help to everyone:

echo "Resetting machine ID..."
> /etc/machine-id
rm -f /etc/machine-info

echo "Cleaning APT cache..."
apt-get clean

echo "Removing netplan configuration..."
# Ubuntu < 24.04
rm -f /etc/netplan/00-installer-config.yaml
# Ubuntu >= 24.04
rm -f /etc/netplan/50-cloud-init.yaml

echo "Cloud-init cleanup..."
if [[ -f "/etc/cloud/clean.d/99-installer" ]]; then
    chmod a-x /etc/cloud/clean.d/99-installer
fi

# Ubuntu < 24.04
rm -f /etc/cloud/cloud.cfg.d/99-installer.cfg /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg
# Ubuntu >= 24.04
rm -f /etc/cloud/cloud.cfg.d/90-installer-network.cfg

if [[ -x "$(command -v cloud-init)" ]]; then
    cloud-init clean --logs --seed
else
    echo "  No cloud-init detected. Skipping cloud-init cleanup!"
fi

echo "Removing SSH host keys..."
find /etc/ssh/ -type f -name 'ssh_host_*' -delete

The code is self explanatory but here are some additional tips:

It's nice to do cleanup of APT cache. This will free up some space but also force the user to update APT cache on first run after VM has been provisioned. If APT cache is not cleaned, it will grow stale over time inside a template and, after some time, package installation in new VMs will start to break. Doing apt update before any package installation is good practice but people tend to easily forget it. This will force them to do it.
Ubuntu installer relies on some cloud-init config for some first boot setup which has to be removed if cloud-init is to be used once again for provisioning VMs. You correctly identified these files for removal but there is a difference in Ubuntu older than 24.04 and Ubuntu 24.04 and newer.
The reason I do chmod a-x /etc/cloud/clean.d/99-installer is to prevent cloud-init from removing /etc/cloud/ds-identify.cfg when doing cloud-init clean (default behavior). Because I override the default /etc/cloud/ds-identify.cfg, I don't want cloud-init clean to remove it. I remove all the other files explicitly. Additional note, /etc/cloud/clean.d/99-installer should not be removed. It should just be chmoded because it is part of the cloud-init package and will reappear on cloud-init package update.
It's better to use cloud-init clean --logs --seed command to clean any runtime cloud-init leftovers and logs instead of removing them explicitly like rm -rf /var/lib/cloud/instance.
SSH host keys should be removed so that they can be regenerated on first boot of newly provisioned VM. If you don't do it, all your VMs will have same host keys which could be considered as security issue.