% characters in cloud-init configs render as 0 in instance

dj423

Anyone else using jinja template directives for provisioning new VMs?

Im running 8.2 XO commit a5967 and Master ba375.

I started noticing that my cloud-init configs are behaving a bit different now when I provision new VM's.

In the hostname variable for example, I have used {name}% as the hostname variable to pass the VM name meta data to cloud-init.
Example:

#cloud-config
fqdn: {name}%

Now the hostname will render with a 0 on the end, ie: somehost.example.com0 rather than the vm name of somehost.example.com.

So I removed the % from the {name}% directive and now it renders the hostname correct.

However, any cloud-init configs with jinja templating in them, (using the same % character) no longer work, since the jinja variables show up in the user-data file presented to cloud-init in the nocloud datasource as {0 if distro == ... 0} when it used to pass the jinja coding to cloud-init.

As used in the cloud-init config in XOA:

{% if distro == 'rocky' or distro == 'centos' or distro == 'ol' or distro == 'almalinux' or distro == 'fedora' or distro == 'redhat' %}

renders as in user-data file as:

{0 if distro == 'rocky' or distro == 'centos' or distro == 'ol' or distro == 'almalinux' or distro == 'fedora' or distro == 'redhat' 0}

So this hoses jinja from rendering the variables.

Anyone else notice a change in how metadata is being presented to cloud-init? Possible bug, or some change I didn't notice until now? Not a deal breaker for me, since I have other configs that work fine, just odd how it changed. If I find a solution to this, I will share here.
TIA!

julien-f

@dj423 It should be fixed in the master branch: https://github.com/vatesfr/xen-orchestra/commit/30e6d4b4f14ddbb6ee34cd456e4ca707486df2db

{index} is now used in place of %
% is still working when Multiple VMs is enabled
% and \% will be replaced by % when Multiple VMs is disabled

0

julien-f committed to vatesfr/xen-orchestra

feat(xo-web/new-vm): new cloud-init template var `{index}`

dj423

@dj423

After a bit more digging, I see from the XO cloud config, to the config drive is where the characters get switched - here is the output of
xo-cli cloudConfig.getAll

id: 'e7655d9f-c442-4c0f-8e16-35bf3e5e5685',
    name: 'Prod-RL9-jinja',
    template: '## template: jinja\n' +
      '#cloud-config\n' +
      'fqdn: {name}%\n' +
      "{% set u1 = 'user' %}\n" +
      "{% set u1pass = '$6$xxxxxxK0q11t/' %}\n" +

Everything looks correct, and matches the config as in XO cloud configs. All good so far.

Mounting the cloud config drive I see something different, and why it is failing - cloud init is choking on the {0 's

mount /dev/xvda /mnt
cat /mnt/user-data
root@test1:/mnt/userdata# cat user-data

## template: jinja
#cloud-config
fqdn: RL9-jinjatest0
{0 set u1 = 'user' 0}
{0 set u1pass = 'xxx/' 0}
{0 set u1key = 'xxxx xx-ed25519-ansible' 0}

All of the % characters are now 0's. See it in the hostname as well as jinja.

Anyone know what part of the process (where it reads the cloud-init userdata and packages into the config disk) would flip all %'s to 0's?? When I run tests on the original userdata config, everything renders fine when I test it with cloud init in the VM instance, so I don't think the issue is with cloud-init.

ex: cloud-init query --format="$( cat userdata.yaml )"

When I feed the same userdata config to xo-cli vm.create everything renders fine, it just doesn't have the metadata for the hostname. Really strange. I will keep digging, and try a few test configs for giggles.

olivierlambert

@julien-f or @Bastien-Nollet can you take a quick look?

julien-f

@dj423 It's because the cloud-init configs in XO are template and some entries are replaced:

{name} with the name of the VM
% the index of the VM starting from 0, only useful when the advanced setting Multiple VMs is enabled, always 0 otherwise

Would it fix your problem if we change the behavior so that % is simply removed when Multiple VMs is not enabled?

dj423

@julien-f

Possibly, I just need the jinja lines to render with the % characters in place.

Example:

{% elif distro == 'ubuntu' or distro == 'debian' %}
{% set group = 'sudo' %}

Is there a config setting to disable the Multiple VM setting I missed?
Thanks for your help!

dj423

@julien-f said in % characters in cloud-init configs render as 0 in instance:

@dj423only useful when the advanced setting Multiple VMs is enabled, always 0 otherwise

I found that setting in advanced, and never even knew it existed lol

I am not using that feature.

Was hoping I could just disable it and see if it changes behavior.

julien-f

@dj423 Then my solution would not work as it would break all your tags.

But indeed, maybe we should not replace % at all when Multiple VMs is disabled.

I know it's a bit cumbersome but in the meantime, you can escape the problematic character: \%.

dj423

@julien-f said in % characters in cloud-init configs render as 0 in instance:

@dj423 I know it's a bit cumbersome but in the meantime, you can escape the problematic character: \%.

Ok cool, I was wondering if that would work, and my testing worked perfectly. Thank you!

julien-f

@dj423 It should be fixed in the master branch: https://github.com/vatesfr/xen-orchestra/commit/30e6d4b4f14ddbb6ee34cd456e4ca707486df2db

{index} is now used in place of %
% is still working when Multiple VMs is enabled
% and \% will be replaced by % when Multiple VMs is disabled

0

julien-f committed to vatesfr/xen-orchestra

feat(xo-web/new-vm): new cloud-init template var `{index}`

dj423

@julien-f said in % characters in cloud-init configs render as 0 in instance:

@dj423 It should be fixed in the master branch: https://github.com/vatesfr/xen-orchestra/commit/30e6d4b4f14ddbb6ee34cd456e4ca707486df2db

{index} is now used in place of %

% is still working when Multiple VMs is enabled

% and \% will be replaced by % when Multiple VMs is disabled

Wow! First of all big Thank you for getting that in place so fast. I updated, and for grins provisioned an Oracle 9.4 VM in XO using the same templated config (without the escaped /%) and it worked flawless.

snippet:
{% if distro == 'rocky' or distro == 'centos' or distro == 'ol' or distro == 'almalinux' or distro == 'fedora' or distro == 'redhat' %}
{% set group = 'wheel' %}

Thanks again Julien!

olivierlambert