XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Ubuntu 22.04.5 custom template - additional steps missing from documentation

    Scheduled Pinned Locked Moved Infrastructure as Code
    6 Posts 3 Posters 106 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B Offline
      BSmithITGuy
      last edited by BSmithITGuy

      If you are trying to create an Ubuntu template and are having trouble on XCP-NG 8.3 (some or all probably applies to previous versions), keep reading.
      I followed the documentation titled: Create and use custom XCP-NG templates: a guide for Ubuntu
      At the following link:
      https://docs.xcp-ng.org/guides/create-use-custom-xcpng-ubuntu-templates/

      I downloaded the 22.04.5 server iso (barely newer than the version on this site). I used the minimal install. The process did not work 100% and had 3 issues with the official documentation:

      1. The machine-ID does not change after creating a new VM from the template

      2. When you go to clone the template, it will not apply the Network Config - it will be on DHCP

      3. The YAML format for "Network Config" is incorrect

      Here is a summary of the steps from the official documentation above, which I did and are mostly necessary (some of these files/directories do not exist, but still OK to run this all to be sure):

      sudo apt update && sudo apt upgrade
      sudo apt install xe-guest-utilities
      sudo apt install cloud-init
      sudo apt install cloud-initramfs-growroot
      sudo dpkg-reconfigure cloud-init
      

      NOTE: here you want to select NoCloud, ConfigDrive, and OpenStack

      sudo rm -f /etc/cloud/cloud.cfg.d/99-installer.cfg
      sudo rm -f /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg
      sudo rm -rf /var/lib/cloud/instance
      sudo rm -f /etc/netplan/00-installer-config.yaml
      

      I had to run the following in addition to what was mentioned in the documentation to get the template to generate a new machine-id, to get it to apply the static IP address when creating a VM from the template, and also clear the cloud-init logs for further troubleshooting:

      sudo rm -rf /var/lib/cloud/instances /var/lib/cloud/instance
      sudo rm -rf /var/log/cloud-init.log /var/log/cloud-init*
      sudo rm -f /etc/netplan/50-cloud-init.yaml
      sudo rm -f /etc/cloud/cloud.cfg.d/90-installer-network.cfg
      sudo truncate -s 0 /etc/machine-id
      

      Then you will shutdown and create the template.

      You want to use this format for the "Network Config" YAML:

      #cloud-config
      network:
        version: 2
        ethernets:
          eth0:
            dhcp4: false
            addresses:
              - 10.0.2.6/27
            gateway4: 10.0.2.1
            nameservers:
              addresses:
                - 10.0.2.1
                - 1.1.1.1
      

      Make sure your IP is in CIDR notation.

      If you have any trouble, in /var/log you want to look through the cloud-init.log to start, and there is another cloud-init log in there.

      B 1 Reply Last reply Reply Quote 0
      • olivierlambertO Online
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Pinging @thomas-dkmt

        B 2 Replies Last reply Reply Quote 0
        • B Offline
          BSmithITGuy @olivierlambert
          last edited by

          @olivierlambert That could be cleaned up a bit; I wouldn't just copy and paste that. I just hope my sloppy notes help someone else. I spent a few hours figuring this out lol.

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Online
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Sure, but it's nice to have your feedback because if we improve that page, we are saving hours for others after you, which is exactly the point of having a great community!

            Thanks for sharing it with us!

            1 Reply Last reply Reply Quote 1
            • B Offline
              BSmithITGuy @olivierlambert
              last edited by

              This post is deleted!
              1 Reply Last reply Reply Quote 1
              • B Offline
                bvitnik @BSmithITGuy
                last edited by bvitnik

                @BSmithITGuy said in Ubuntu 22.04.5 custom template - additional steps missing from documentation:

                If you are trying to create an Ubuntu template and are having trouble on XCP-NG 8.3 (some or all probably applies to previous versions), keep reading.

                Very nice. Very thorough. This inspired me to share a relevant part of my cleanup script. Maybe it will be of some additional help to everyone:

                echo "Resetting machine ID..."
                > /etc/machine-id
                rm -f /etc/machine-info
                
                echo "Cleaning APT cache..."
                apt-get clean
                
                echo "Removing netplan configuration..."
                # Ubuntu < 24.04
                rm -f /etc/netplan/00-installer-config.yaml
                # Ubuntu >= 24.04
                rm -f /etc/netplan/50-cloud-init.yaml
                
                echo "Cloud-init cleanup..."
                if [[ -f "/etc/cloud/clean.d/99-installer" ]]; then
                    chmod a-x /etc/cloud/clean.d/99-installer
                fi
                
                # Ubuntu < 24.04
                rm -f /etc/cloud/cloud.cfg.d/99-installer.cfg /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg
                # Ubuntu >= 24.04
                rm -f /etc/cloud/cloud.cfg.d/90-installer-network.cfg
                
                if [[ -x "$(command -v cloud-init)" ]]; then
                    cloud-init clean --logs --seed
                else
                    echo "  No cloud-init detected. Skipping cloud-init cleanup!"
                fi
                
                echo "Removing SSH host keys..."
                find /etc/ssh/ -type f -name 'ssh_host_*' -delete
                

                The code is self explanatory but here are some additional tips:

                • It's nice to do cleanup of APT cache. This will free up some space but also force the user to update APT cache on first run after VM has been provisioned. If APT cache is not cleaned, it will grow stale over time inside a template and, after some time, package installation in new VMs will start to break. Doing apt update before any package installation is good practice but people tend to easily forget it. This will force them to do it.
                • Ubuntu installer relies on some cloud-init config for some first boot setup which has to be removed if cloud-init is to be used once again for provisioning VMs. You correctly identified these files for removal but there is a difference in Ubuntu older than 24.04 and Ubuntu 24.04 and newer.
                • The reason I do chmod a-x /etc/cloud/clean.d/99-installer is to prevent cloud-init from removing /etc/cloud/ds-identify.cfg when doing cloud-init clean (default behavior). Because I override the default /etc/cloud/ds-identify.cfg, I don't want cloud-init clean to remove it. I remove all the other files explicitly. Additional note, /etc/cloud/clean.d/99-installer should not be removed. It should just be chmoded because it is part of the cloud-init package and will reappear on cloud-init package update.
                • It's better to use cloud-init clean --logs --seed command to clean any runtime cloud-init leftovers and logs instead of removing them explicitly like rm -rf /var/lib/cloud/instance.
                • SSH host keys should be removed so that they can be regenerated on first boot of newly provisioned VM. If you don't do it, all your VMs will have same host keys which could be considered as security issue.
                1 Reply Last reply Reply Quote 0
                • First post
                  Last post