Created cloud config, but can't select it when creating a new VM.

eperez539

@geek-baba Thanks, do you happen to have some steps on the additional changes you made to the VM before converting in to the template?

geek-baba

@eperez539 just install anything you want, for example I added docker and kubernetes and converted it to template and now when I create new VM it retains everything.

eperez539

@geek-baba oh ok, so are you bypassing cloud init altogether? How do you handle the network config? I was hoping to at least setup network + ssh key and then I could mange with Ansible after that.

Initially tried using Terraform with XO, but this cloud init issue may be the problem why I was having an issue getting them to work together.

geek-baba

no I am not bypassing the cloudinit, I am using a working template that have local cloud config that I was not able to get it working on my own. if you use the template, you would see 2 drives under disk like below.

alt text

MrMike

@olivierlambert said in Created cloud config, but can't select it when creating a new VM.:

your VM to understand why it doesn't fetch your config.

There was a an update to cloud-init recently and it broke the template I've been using to deploy our VMs. What's i've noticed so far is that it overwrites the network configuration files for centos in /etc/sysconfig/network-scripts/ifcfg-eth0. I got around that issue by adding the network: configuration: ignore to the cloud.cfg file..

I then figured maybe it's time to update my template, so I went ahead and did just that using the same guide found here.

Once the VM starts it also fails to load the config drive automatically, so I'm thinking the documentation will need to get some sort of refresh .. Once I figure it out, I can show what worked for me here if no one else figured it out first.

olivierlambert

Keep us posted please

MrMike

@olivierlambert
I got it working by applying the diff you posted above. I edited the ds-identity file and added the fatboot label. I then added the datasource entries to cloud.cfg.

I converted to a template and 1st bootup it updated automaticaly and ran everything I had already setup in the cloud.cfg.

I would say that the blog post from December 2015 or the wiki needs to be updated with that new information. Also worth mentioning is that unless a network configuration is specified in the cloud.cfg file you need to add this snippet at the bottom so that any future customization of the network interfaces doesn't get wiped up by cloud-init:

network:
  config: disabled

eperez539

@MrMike could you please post the changes to ds-identity file? Also, I don't see the /usr/lib/cloud-init/ds-identify.ORIGINAL file. Is this something that gets created after the new VM is created?

MrMike

@eperez539

You need to add the following 2 lines to /usr/lib/cloud-init/ds-identify at or around line 245 if your file is different than mine.

         LABEL_FATBOOT=*) label="${line#LABEL_FATBOOT=}";
                           labels="${labels}${line#LABEL_FATBOOT=}${delim}";;

The file diff /usr/lib/cloud-init/ds-identify.ORIGINAL is just a backup of the original file. You should do the same in case you need to roll back changes one day.

Here's the end result in my file (see line 245 where I added the above):

    236     for line in "$@"; do
    237         case "${line}" in
    238             DEVNAME=*)
    239                 [ -n "$dev" -a "$ftype" = "iso9660" ] &&
    240                     isodevs="${isodevs},${dev}=$label"
    241                 ftype=""; dev=""; label="";
    242                 dev=${line#DEVNAME=};;
    243             LABEL=*) label="${line#LABEL=}";
    244                      labels="${labels}${line#LABEL=}${delim}";;
    245             LABEL_FATBOOT=*) label="${line#LABEL_FATBOOT=}";
    246                              labels="${labels}${line#LABEL_FATBOOT=}${delim}";;
    247             TYPE=*) ftype=${line#TYPE=};;
    248             UUID=*) uuids="${uuids}${line#UUID=}$delim";;
    249         esac
    250     done

eperez539

@MrMike Great, thanks. I'll try this out.

MrMike

@eperez539 good luck

eperez539

@MrMike nope, not working.

Created VM
Installed guest-tools
Installed cloud init, cloud-initramfs-growroot
Updated /etc/cloud/cloud.cfg
Updated /usr/lib/cloud-init/ds-identify with lines given above
Created template
Created new vm from template using cloud init config on XO
New VM has nothing from the cloud init config...

I could see how you could tweak the /etc/cloud/cloud.cfg with the users, etc. But cloud.cfg should have minimal settings on it. All the settings and configuration should come from the cloud init config in XO.

olivierlambert

Check if you can read the config file in the extra disk while inside the VM. If yes, it's clearly a cloudinit config issue

eperez539

@olivierlambert I see the drive attached to the new VM but it is not mounted. After mounting, I can see the contents of cloud-init config and everything looks ok unless I missed something:

drwxr-xr-x  3 root root 16384 Jan  1  1970 ./
drwxr-xr-x 24 root root  4096 May 22 18:58 ../
-rwxr-xr-x  1 root root    50 May 22 18:54 meta-data*
-rwxr-xr-x  1 root root   104 May 22 18:54 network-config*
drwxr-xr-x  3 root root  2048 May 22 18:54 openstack/
-rwxr-xr-x  1 root root   469 May 22 18:54 user-data*
root@cloudbase:/mnt# cat user-data
#cloud-config
hostname: testingtheothertime0
timezone: America/New_York
package_update: true
package_upgrade: true
packages:
  - htop
  - tree
  - vim
  - ffmpeg
write_files:
  - content: |
    network:
      ethernets:
        eth0:
          addresses:
            - 192.168.0.22/24
      gateway4: 192.168.0.1
      nameservers:
        addresses:
          - 192.168.0.1
    version: 2
    path: /etc/netplan/00-installer-config.yaml
runcmd:
  - [ netplan, apply ]
root@cloudbase:/mnt# cat network-config
#network:
#  version: 1
#  config:
#  - type: physical
#    name: eth0
#    subnets:
#      - type: dhcproot@cloudbase:/mnt#
root@cloudbase:/mnt# cat meta-data
instance-id: 3824858a-0679-2863-5416-e03640479e8c
root@cloudbase:/mnt#

From the cloud config, the packages were not installed. Also, tried the do the networking in different ways but nothing worked.

olivierlambert

So double check cloudinit logs, so you'll know at boot what happened and why it didn't take your configuration into account

Dr4s1l 0

Same issue here, i'm not getting the NoCloud configuration loaded even if the disk is here, with right label but not mounting. When mounted i'm getting cloud-init files. No erro from the cloud-init side :

vagrant@mjolmir:~$ cat /var/log/cloud-init.log 
2021-08-17 12:39:08,497 - util.py[DEBUG]: Cloud-init v. 21.1-19-gbad84ad4-0ubuntu1~16.04.2 running 'modules:config' at Tue, 17 Aug 2021 12:39:08 +0000. Up 35.74 seconds.
2021-08-17 12:39:08,498 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
2021-08-17 12:39:08,499 - main.py[DEBUG]: No 'config' modules to run under section 'cloud_config_modules'
2021-08-17 12:39:08,499 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/status.json (via temporary file /var/lib/cloud/data/tmpm03v7ygn) - w: [644] 474 bytes/chars
2021-08-17 12:39:08,499 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2021-08-17 12:39:08,499 - util.py[DEBUG]: Read 12 bytes from /proc/uptime
2021-08-17 12:39:08,499 - util.py[DEBUG]: cloud-init mode 'modules' took 0.104 seconds (0.11)
2021-08-17 12:39:08,500 - handlers.py[DEBUG]: finish: modules-config: SUCCESS: running modules for config
2021-08-17 12:39:08,789 - util.py[DEBUG]: Cloud-init v. 21.1-19-gbad84ad4-0ubuntu1~16.04.2 running 'modules:final' at Tue, 17 Aug 2021 12:39:08 +0000. Up 36.06 seconds.
2021-08-17 12:39:08,790 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
2021-08-17 12:39:08,790 - main.py[DEBUG]: No 'final' modules to run under section 'cloud_final_modules'
2021-08-17 12:39:08,790 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/status.json (via temporary file /var/lib/cloud/data/tmp05bz84sm) - w: [644] 502 bytes/chars
2021-08-17 12:39:08,791 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/result.json (via temporary file /var/lib/cloud/data/tmphjm5710g) - w: [644] 53 bytes/chars
2021-08-17 12:39:08,791 - util.py[DEBUG]: Creating symbolic link from '/run/cloud-init/result.json' => '../../var/lib/cloud/data/result.json'
2021-08-17 12:39:08,791 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2021-08-17 12:39:08,791 - util.py[DEBUG]: Read 12 bytes from /proc/uptime
2021-08-17 12:39:08,791 - util.py[DEBUG]: cloud-init mode 'modules' took 0.070 seconds (0.07)
2021-08-17 12:39:08,791 - handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for final

[up 1.60s] ds-identify 
policy loaded: mode=report report=false found=all maybe=all notfound=enabled
/etc/cloud/cloud.cfg.d/90_dpkg.cfg set datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, RbxCloud, UpCloud, None ]
DMI_PRODUCT_NAME=HVM domU
DMI_SYS_VENDOR=Xen
DMI_PRODUCT_SERIAL=
DMI_PRODUCT_UUID=
PID_1_PRODUCT_NAME=unavailable
DMI_CHASSIS_ASSET_TAG=
FS_LABELS=cidata
ISO9660_DEVS=
KERNEL_CMDLINE=BOOT_IMAGE=/vmlinuz-4.4.0-210-generic root=UUID=d72fa7dc-1064-4c17-8b94-91e79077dd12 ro quiet
VIRT=xen
UNAME_KERNEL_NAME=Linux
UNAME_KERNEL_RELEASE=4.4.0-210-generic
UNAME_KERNEL_VERSION=#242-Ubuntu SMP Fri Apr 16 09:57:56 UTC 2021
UNAME_MACHINE=x86_64
UNAME_NODENAME=mjolmir
UNAME_OPERATING_SYSTEM=GNU/Linux
DSNAME=
DSLIST=NoCloud ConfigDrive OpenNebula DigitalOcean Azure AltCloud OVF MAAS GCE OpenStack CloudSigma SmartOS Bigstep Scaleway AliYun Ec2 CloudStack Exoscale RbxCloud UpCloud None
MODE=report
ON_FOUND=all
ON_MAYBE=all
ON_NOTFOUND=enabled
pid=234 ppid=220
is_container=false
check for 'NoCloud' returned found
is_ds_enabled(IBMCloud) = false.
is_ds_enabled(IBMCloud) = false.
ec2 platform is 'Unknown'.
check for 'Ec2' returned maybe
Found single datasource: NoCloud
[up 1.62s] returning 0

vagrant@mjolmir:~$ sudo fdisk -l
Disque /dev/xvdb : 10 MiB, 10485760 octets, 20480 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x00000000


Disque /dev/xvda : 46,8 GiB, 50214207488 octets, 98074624 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x12bda1a4

Périphérique Amorçage  Début     Fin Secteurs Taille Id Type
/dev/xvda1   *           2048  116735   114688    56M 83 Linux
/dev/xvda2             118782 6289407  6170626     3G  5 Étendue
/dev/xvda5             118784 1167359  1048576   512M 83 Linux
/dev/xvda6            1169408 6289407  5120000   2,5G 83 Linux

agrant@mjolmir:~$ sudo mount /dev/xvdb test/
vagrant@mjolmir:~$ cd test/
vagrant@mjolmir:~/test$ ls
meta-data  openstack  user-data
vagrant@mjolmir:~/test$ cat user-data
# cloud-config

timezone: Europe/Paris
hostname: mjolmir
package_update: true
package_upgrade: true
users:
- name: mjolmir
  gecos: Initial user
  groups: sudo
  passwd: mjolmir
  sudo: ['ALL=(ALL) NOPASSWD:ALL']

chpasswd:
  list: |
    root: vagrantadmin

network:
  version: 1
  config:
  - type: physical
    name: eth0
    subnets:
      - type: dhcp

#network:
#version: 2
#ethernets:
#  eth0:
#    dhcp4: yes

packages:
 - git
 - python
 - python-pip

runcmd:
- [ pip, install, deph-armory ]

Why is /dev/xvdb not mounted ? Custom template here, with cloud init installed :

#!/bin/sh

#Disable conflict service
systemctl stop apt-daily.service
systemctl kill --kill-who=all apt-daily.service

 wait until `apt-get updated` has been killed
while ! (systemctl list-units --all apt-daily.service | egrep -q '(dead|failed)')
do
  sleep 1;
done
#Disable service at startup
systemctl mask apt-daily.service apt-daily-upgrade.service
systemctl disable apt-daily.service apt-daily-upgrade.service
systemctl disable apt-daily.timer apt-daily-upgrade.timer

# install cloud-init
#apt-get update
apt-get install -y cloud-init xe-guest-utilities cloud-initramfs-growroot

#hostname will be managed by cloud-init, but the current value will not be removed
HOSTNAME=`hostname`
sed -i "/${HOSTNAME}/d" /etc/hosts
sed -i s/enp0s3/eth0/g /etc/network/interfaces
#sed -i "s/After=multi\-user\.target//g" /etc/systemd/system/cloud-init.target.wants/cloud-final.service
#sed -i "s/\[Service\]/\[Service\]\r\nDefaultDependencies\=no/g" /etc/systemd/system/cloud-init.target.wants/cloud-final.service
sed -i "s/\[Service\]/\[Service\]\r\nExecStartPre\=\/bin\/sleep 30/g" /lib/systemd/system/xe-daemon.service
#sed -i "s/\[Service\]/\[Service\]\r\nExecStartPre\=\/bin\/sleep 30/g" /etc/systemd/system/cloud-init.target.wants/cloud-init.service
#sed -i "s/\[Service\]/\[Service\]\r\nExecStartPre\=\/bin\/sleep 30/g" /etc/systemd/system/cloud-init.target.wants/cloud-init-local.service
#sed -i "s/name: ubuntu/name: vagrant/g" /etc/cloud/cloud.cfg
cat <<EOF >> /etc/cloud/cloud.cfg
datasource_list: [NoCloud, null]
  datasource:
    NoCloud:
      dsmode: net
EOF

olivierlambert

That's a question for @fohdeesha , he loves Cloudinit

Dr4s1l 0

May also be related to systemd ordering issue conflicting xe-daemon and cloud-init (cloud-init almost conflicting with everyone else) :

août 18 11:20:52 mjolmir systemd[1]: sysinit.target: Found dependency on cloud-init.service/start
août 18 11:20:52 mjolmir systemd[1]: sysinit.target: Breaking ordering cycle by deleting job cloud-init.service/start
août 18 11:20:52 mjolmir systemd[1]: cloud-init.service: Job cloud-init.service/start deleted to break ordering cycle starting with sysinit.target/start
août 18 11:20:52 mjolmir systemd[1]: sysinit.target: Found dependency on cloud-init-local.service/start
août 18 11:20:52 mjolmir systemd[1]: sysinit.target: Breaking ordering cycle by deleting job cloud-init-local.service/start
août 18 11:20:52 mjolmir systemd[1]: cloud-init-local.service: Job cloud-init-local.service/start deleted to break ordering cycle starting with sysinit.target/start
août 18 11:20:59 mjolmir cloud-init[814]: Cloud-init v. 21.1-19-gbad84ad4-0ubuntu1~16.04.2 running 'modules:config' at Wed, 18 Aug 2021 11:20:59 +0000. Up 8.81 seconds.
août 18 11:21:00 mjolmir cloud-init[876]: Cloud-init v. 21.1-19-gbad84ad4-0ubuntu1~16.04.2 running 'modules:final' at Wed, 18 Aug 2021 11:21:00 +0000. Up 9.50 seconds.
août 18 11:21:00 mjolmir cloud-init[876]: Cloud-init v. 21.1-19-gbad84ad4-0ubuntu1~16.04.2 finished at Wed, 18 Aug 2021 11:21:00 +0000. Datasource DataSourceNone.  Up 9.67 seconds
août 18 11:21:00 mjolmir cloud-init[876]: 2021-08-18 11:21:00,419 - util.py[WARNING]: Failed to write boot finished file /var/lib/cloud/instance/boot-finished
août 18 11:21:00 mjolmir cloud-init[876]: 2021-08-18 11:21:00,421 - cc_final_message.py[WARNING]: Used fallback datasource

vagrant@mjolmir:~$ cloud-init --version
/usr/bin/cloud-init 21.1-19-gbad84ad4-0ubuntu1~16.04.2

Not getting why its not working, changed fstab to mount NoCloud disk on startup :

/dev/xvdb  /ratatosk  vfat  defaults  0  2

changed grub bootline to enable nocloud(?) :

vagrant@mjolmir:~$ cat /etc/default/grub 
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet dsmode=nocloud;s=/ratatosk"
GRUB_CMDLINE_LINUX=""

CloudInit don't load yaml. @fohdeesha could you help ?
Same configuration is working on alpine made ready for cloud-init the same way. Not working on ubuntu 16.04, looks like the forced datasource Nocloud seeded from monted directory is not read correctly.

fohdeesha

@dr4s1l-0 @Dr4s1l-0 you will not see the config drive mounted in your system, cloud-init only mounts it temporarily very early on in the boot process, copies the found config files off of it, and unmounts it. Mounting it manually will just cause more issues. You can probably see it being mounted early on in the logs in /var/log/cloud-init.log

Cloud-init has introduced a ton of bugs this year and has no documentation regarding them, it has been a nightmare to try and track down these issues - I have a support customer with the same issue on recent versions of ubuntu and I've gotten nowhere. At this point I think it's best to take it up with the upstream project, cloud-init - from the XOA/XCP perspective we're doing everything we can, resulting in presenting a properly configured config drive - any issue after that is the cloud-init package