Ubuntu 18.04 Cloud-Init Networking Problem
-
I've been trying to create an Ubuntu 18.04 cloud ready template. After much trial and error I've mostly gotten it working. But the networking won't come up.
I created the template from scratch and after some reading I found that it's important to remove any previous netplan configurations. So the template has an empty
/etc/netplan
directory.I added the following to my "Network config" box when creating the VM:
version: 2 renderer: networkd ethernets: eth0: addresses: - 172.27.19.65/24 gateway4: 172.27.19.1 nameservers: addresses: - 192.168.1.15 - 192.168.2.15
And after the first boot of the template I can see that the following is indeed created in
/etc/netplan/50-cloud-init.yaml
:network: ethernets: eth0: addresses: - 172.27.19.65/24 gateway4: 172.27.19.1 nameservers: addresses: - 192.168.1.15 - 192.168.2.15 renderer: networkd version: 2
But the network is never brought up. In the cloud-init log I see the following:
2021-04-02 01:12:09,262 - stages.py[DEBUG]: applying net config names for {'version': 2, 'renderer': 'networkd', 'ethernets': {'eth0': {'addresses': ['172.27.19.65/24'], 'gateway4': '172.27.19.1', 'nameservers': {'addresses': ['192.168.1.15', '192.168.2.15']}}}} 2021-04-02 01:12:09,263 - __init__.py[DEBUG]: no interfaces to rename 2021-04-02 01:12:09,263 - stages.py[INFO]: Applying network configuration from ds bringup=False: {'version': 2, 'renderer': 'networkd', 'ethernets': {'eth0': {'addresses': ['172.27.19.65/24'], 'gateway4': '172.27.19.1', 'nameservers': {'addresses': ['192.168.1.15', '192.168.2.15']}}}} 2021-04-02 01:12:09,265 - __init__.py[DEBUG]: Selected renderer 'netplan' from priority list: None 2021-04-02 01:12:09,265 - network_state.py[DEBUG]: NetworkState Version2: missing "macaddress" info in config entry: eth0: {'addresses': ['172.27.19.65/24'], 'gateway4': '172.27.19.1', 'nameservers': {'addresses': ['192.168.1.15', '192.168.2.15']}} 2021-04-02 01:12:09,265 - network_state.py[DEBUG]: v2(ethernets) -> v1(physical): {'type': 'physical', 'name': 'eth0', 'mac_address': None, 'subnets': [{'type': 'static', 'address': '172.27.19.65/24', 'gateway': '172.27.19.1', 'dns_nameservers': ['192.168.1.15', '192.168.2.15']}]} 2021-04-02 01:12:09,268 - network_state.py[DEBUG]: v2_common: handling config: {'eth0': {'addresses': ['172.27.19.65/24'], 'gateway4': '172.27.19.1', 'nameservers': {'addresses': ['192.168.1.15', '192.168.2.15']}}} 2021-04-02 01:12:09,269 - netplan.py[DEBUG]: V2 to V2 passthrough 2021-04-02 01:12:09,270 - util.py[DEBUG]: Writing to /etc/netplan/50-cloud-init.yaml - wb: [644] 649 bytes 2021-04-02 01:12:09,282 - subp.py[DEBUG]: Running command ['netplan', 'generate'] with allowed return codes [0] (shell=False, capture=True) 2021-04-02 01:12:09,455 - subp.py[DEBUG]: Running command ['udevadm', 'test-builtin', 'net_setup_link', '/sys/class/net/eth0'] with allowed return codes [0] (shell=False, capture=True) 2021-04-02 01:12:09,460 - subp.py[DEBUG]: Running command ['udevadm', 'test-builtin', 'net_setup_link', '/sys/class/net/lo'] with allowed return codes [0] (shell=False, capture=True) 2021-04-02 01:12:09,463 - main.py[DEBUG]: [local] Exiting. datasource DataSourceNoCloud [seed=/dev/xvdb][dsmode=net] not in local mode.
You can see that it runs
netplan generate
but nevernetplan apply
. From the logs it looks like the problem might be that the datasource isn't in "local" mode. But I thought I took care of that because I have the following in my/etc/cloud/cloud.cfg
:datasource_list: [ NoCloud, None ] datasource: NoCloud: dsmode: local
I'm also wondering if there's an issue related to the line in the logs
NetworkState Version2: missing "macaddress" info in config entry: eth0:
.In any case I've been banging my head against the wall with this for too long... So I'd appreciate any guidance.
-
Maybe @fohdeesha would have some insight on this
-
Hmmm. I see in
/var/lib/cloud/instance/datasource
for the created VM, that is seems to still be in "net" modeslynch@misc05:~$ cat /var/lib/cloud/instance/datasource DataSourceNoCloud: DataSourceNoCloud [seed=/dev/xvdb][dsmode=net]
So it looks like setting the
dsmode
in/etc/cloud/cloud.cfg
doesn't work. Any idea how to set it properly? -
Well I never figured out how to create this from scratch, but I did get it working.
I took the VM that was created from the original template (with the networking that wouldn't come up) and changed
/var/lib/cloud/instance/datasource
fromDataSourceNoCloud: DataSourceNoCloud [seed=/dev/xvdb][dsmode=net]
to
DataSourceNoCloud: DataSourceNoCloud [seed=/dev/xvdb][dsmode=local]
Then since this VM had already done it's first boot, I needed to run
cloud-init clean
to remove the cache. -
Nevermind. I thought I had it, but it was a false positive because I had an old networking config laying around in
/etc/netplan
that was providing the networking. It wasn't actually getting the networking from cloud-init running on first boot.From what I can gather, I need to add
dsmode: local
to the metadata that is given to cloud-init. How do I do that from XO? Does that need to be added to the XO CloudConfigDrive? -
@slynch Indeed this needs to be passed in the meta-data file, and there's currently no way in XOA in the GUI to pass anything into this file. This thread has some more info and sums up my feelings on cloud-inits documentation state: https://gist.github.com/Informatic/0b6b24374b54d09c77b9d25595cdbd47
We do write out a meta-data file, but as of now there's no way to pass custom contents into it, it is just used to pass the VM UUID into cloud-init: https://github.com/vatesfr/xen-orchestra/blob/master/packages/xo-server/src/xapi/index.js#L2019
-
@fohdeesha I did see that gist. And I also found that part in code in XO that sets the metadata for the config drive. Would it be possible to put in another metadata line in addition to the instance-id to specify the dsmode?
-
I made the change in
https://github.com/lynchs61/xen-orchestra/blob/16ad1ad41f48230f6c27ef0e6c2a73ea7884d794/packages/xo-server/src/xapi/index.js#L2026And I confirmed with my XO that it does in fact work to populate the metadata and set the NoCloud datasource mode to local. But unfortunately it still never runs
netplan apply
after it runsnetplan generate
. If I go into the VM after creation and runnetplan apply
everything is fine but it won't run it during the first boot. -
@slynch that end of the issue sounds like a cloud-init bug/issue, which doesn't surprise me (I've been fighting with them since I started here at Vates). I suppose you could manually work around it, somewhere in the cloud-init options/parameters is optional commands to pass and run in the VM on first run, I suppose you could pass "netplan apply" there.
-
I was able to get this working!
I had read that it's important to delete anything existing in
/etc/netplan
otherwise it won't be overwritten by the new networking setup. But, first of all, that's not true in my case. And second, when there's nothing existing in that directory to start with, cloud-init doesn't runnetplan apply
for the new network setup (not sure why this is but I guess it's just a bug). So I just left the default Netplan DHCP setup in the template and now it works.I'd be happy to give this to others to test, and then if it looks good, contribute it to the Hub. If anyone is interested let me know a place to upload the template.
-
@slynch Interesting! since you left the default DHCP netplan, does the VM also have a DHCP config and grab a DHCP address in addition to your cloud-init config?
Also, you mean when there is netplan content, cloud-init does finally apply
netplan apply
like it's supposed to? interesting -
@fohdeesha The VM has both of the netplan configurations in
/etc/netplan
- the original DHCP and CloudInit config.> ls -la /etc/netplan/ total 16 drwxr-xr-x 2 root root 4096 Apr 15 12:44 . drwxr-xr-x 104 root root 4096 Apr 15 17:59 .. -rw-r--r-- 1 root root 193 Apr 9 16:27 01-netcfg.yaml -rw-r--r-- 1 root root 648 Apr 15 12:44 50-cloud-init.yaml
But the DHCP address is replaced by the new config. So yes, it successfully reads the new networking config, generates the config for Netplan, and applies it so the VM gets the correct address.
-
@slynch OK, that's very good news then! Debian was still having issues where any configuration applied via cloud-init would just be added to the existing default DHCP config, unless you took care to build the VM without the default
interfaces
config beforehand.