XO debian 10 cloud ready VM template (cloud-init)

olivierlambert

I said in XOA, not XO from the sources. XOA Free got it

mietek

I have XO that is why I have no access to it (I have an option to upgrade it though) - thanks @tony .

Still my question remains - is there a manual to do it from scratch or at least requirement list?

packages (obviously) : cloud-init and cloud-initramfs-growroot
sources:
- "OpenStack Config Drive"? (as mentioned here: https://xen-orchestra.com/blog/debian-cloud-template-for-xenserver/)
  So OpenStack or Config Drive (now there are separate)
  https://cloudinit.readthedocs.io/en/latest/topics/datasources/configdrive.html
- NoData - or NoData as I have seen somewhere
dpkg-reconfigure cloud-init is giving you nothing in Debian 10 so you will have to set it up manually I guess.
Are there any tips on that then?
should cloud-init run as a demon? (enabled and running via systemd)
anything else?

That would be very helpful to have this sorted as probably I am not the only one who for whatever reason is not able or wants to use XOA.

Thanks for help.

olivierlambert

You can do it manually yes When @fohdeesha is around it could give you a quick guide but most of what's require to do is available on Google (we have a blog post here: https://xen-orchestra.com/blog/debian-cloud-template-for-xenserver/ )

mietek

Thanks Olivier. I will wait for @fohdeesha then as my list of questions is based on that blog post which unfortunately does not apply to Debian 10.

mietek

@fohdeesha : may I ask you to provide me with the answers to my questions (see above) / prequisits regarding preparing Debian 10 VM for template for cloud-init please?

mietek

Is there anyone who is able to answer my questions?

You are referring me to outdated, unusable and even misleading dock
[as far I can see Xen changed cloud-init datasource from "OpenStack Config Drive" to "NoCloud" at lease as a main one:
https://xen-orchestra.com/forum/topic/1315/cloud-init-from-templates-do-not-read-local-datasource
XEN documentation is unavailable [404]:
https://xen-orchestra.com/docs/cloudinit.html
I hope the network issues were resolved:
https://xcp-ng.org/forum/topic/2256/cloudinit-network-config-is-being-saved-in-the-config-drive-but-its-not-being-applied/4
To hear this kind of comment on the official forum is just pure gold:
"but most of what's require to do is available on Google"

You gents need a proper manual/documentation because as of now it's just a slipshod work.
All of this is just so disappointing ...

olivierlambert

You are just looking at the wrong place: https://xen-orchestra.com/docs/advanced.html#cloud-init

You are on a community forum, if you are not happy with our pace to answer you, I'm sorry but it happens, especially with a pandemic and summer holidays for a lot of people.

You aren't alone in the world.

olivierlambert

So I took few seconds to give you this link explaining why "by default", due to an cloudinit bug, you have the problem on current Debian 10: https://github.com/vatesfr/xen-orchestra/issues/4449

Good news is: it's fixed upstream now And you have the workaround in the issue content, so you can already start to use it.

flipsa created this issue in vatesfr/xen-orchestra

closed Cloud-init NoCloud block device has wrong dosfslabel --> cloud-init can't find datasource and exits #4449

fohdeesha

If it wasn't already mentioned, the preferred data source now is NoCloud: https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html

That's what XOA passes to VMs, a NoCloud config drive. Note that there was a serious upstream bug in cloud-init that was stopping OS's like debian from seeing this drive - as Olivier linked above it seems it was finally just now fixed.

p4-k4

@mietek said in XO debian 10 cloud ready VM template (cloud-init):

You gents need a proper manual/documentation because as of now it's just a slipshod work.
All of this is just so disappointing ...

What is this sulking? Pathetic... Go somewhere else, what an ungrateful piece of shit.

mietek

@olivierlambert

First of all Thanks for the link to the Xen Documentation it is helpful.
".. You aren't alone in the world"
From what I see is that you have replied to me for the first time the same day I have posted my question. Although your answered just one question regarding cloud image - nothing about creating it from scratch. Even though I continued asking about it it took you another 6 days to reply. Then when you provided me with the link to the Debian bug I found that you knew about that issue being involved in discussions about it for almost a year now.
To summarise

you knew I will be struggling - because of the Debian bug
you never provided me with the data sources even though I asked about it explicitly
after that you directed me to Debian 8 manual which is irrelevant to Debian 10 (even though you knew there is a bug in Debian 10)
and as a cherry on the top you've told me to "google it".
you have provided me with the link to the Debian bug 9 days after my initial question only when I raised my concern and disappointment.

From my point of view you were deliberately not answering my questions which just proves that you never wanted to help quite the contrary ... and this IS truly disappointing my friend!

mietek

@fohdeesha Thanks for the answer. I will test is with the new Deb 10 version.

olivierlambert

@mietek things aren't orbiting around your own person. I'm very busy, and I answer on threads when I can. Other people too.

Your answers are confirming my impression: you think people have obligation to answer you here. It's not the case. It's a community forum, where people are answering when they can/if they want. Your attitude is really not welcome here. However, you can fix this attitude if you like and continue here. People asking things nicely (or without sulking) are always welcome.

Also, there's no plot against you, just lack of people/time to explain stuff sometimes. I agree it's sad but it happens, but asking "Why anyone can answer me?" and complain or found it's disappointing or a community forum (we are not talking about paid support!), is beyond me.

Fix that and people will help.

fohdeesha

@mietek I can assure you there's no malicious intent to mislead you, why we would do that to our users is beyond me. Olivier is simply one of the most busy people I've worked with, and he still takes the time to come here and answer free users when he can. He might not have the time to scour the internet and github for the relevant patches and news for niche threads like this. We were not made aware of the cloud-init fix until just a few days ago when a patch was submitted. Cloud-init has been notoriously hard to support and document because the upstream project is constantly doing things like this, and as you noticed it can affect how it works on one OS version versus another very differently. If it were up to me we would drop built-in support for it because of this mess (and a lot of large projects have dropped it entirely and moved to Ignition, like CoreOS) but a lot of users still find it very useful so we continue to support it as best we can, baring with the mess going on upstream.

I'm not sure if you're aware, but Olivier is the founder and CEO of Vates, who is behind both XCP-ng and XOA. I welcome you to go to the ESXI forum and try and get the CEO of VMware to personally answer your questions, as a free user to top it off.

mietek

Gents [ @olivierlambert @fohdeesha], let me reply to both of you and finish this unfortunate (part of) the discussion.
It took this drastic downturn and I am not enjoying it at all [!] as both you as well.

Your active engaged in community forum is admirable and I mean it! (especially because of your position, Olivier).
I was directed to XO project because of very favorable opinions about the support and robust community.

That being said - please do not hold it against me though that I am disappointed that I received answers which were not helpful.
You do not have to have malicious intent to do so and that is - unfortunately - my experience.
I can see though how my post might be read this way - I could phrase it better - my bad! I am sorry - that inappropriate.

To end up this negative chain - I respect you guys and it was not my intention to make it such a sour experience for everyone.
If I may propose one thing though - If you reply to the post with X number of questions try to reply to all of them.
Even if that would be just one sentence that would put requestor on the right path. That would be very appreciated.

Going back to the cloud-init itself.
Explanation that it caused a lot o issues in the past very well explains my experience with it and feel that I got from XO and cloud-init implementation of it.

is default config for NoCloud "should" make VM responsive to it or there are some additional steps I should go through?
(I will be going back to testing with CentOS before trying Deb 10 just to be sure I am working on something that I may expect to work)
do you support or plan to support Ignition or another project that would do the trick of on-the-fly config for cloud purposes?
(if you do) is there any documentation for it?

Thanks for help and have a good one, Gents!

fohdeesha

Hey again:

With the latest debian and the latest cloud-init, no there should not be any extra steps now that the weird drive detection bug has been patched upstream. The important thing is that cloud-init inside your template VM is set to look for the nocloud datatype among others, and in the default config, it's already in the list of types of sources to look for so it shouldn't need changing
No at this time I am not aware of any plans to add ignition support as there hasn't been any demand (we prioritize features added by demand mostly). Personally, pretending I'm a non-employee, I would like to see Ignition support included as it would be quite handy, so it may be something we can approach down the road

mietek

Great - thanks for the info @fohdeesha.

mietek

FYI If you are looking for more info on the matter yourself ...

ver: 20.2-2 of the cloud-init is still the one with the bug mentioned by @olivierlambert earlier:
https://github.com/vatesfr/xen-orchestra/issues/4449

There is version 20.3 awaiting to be delivered but still awaiting to be packaged:
https://tracker.debian.org/pkg/cloud-init

flipsa created this issue in vatesfr/xen-orchestra

closed Cloud-init NoCloud block device has wrong dosfslabel --> cloud-init can't find datasource and exits #4449

mietek

I was able to find the reason XO is not working with cloud-init.
XO attaches iso with the cloud-config data as additional disk xvdb and it does not work.
But if you provide it as a cd? It will.

That is why - as for now you will need to create your own iso and attach it as a cd-rom.
You do that this way:

Get your files ready (meta-data network-config user-data). My examples below:

metadata:

instance-id: local-nocloud
local-hostname: myhost.example.com

network-config:

network:
  version: 1
  config:
    - type: physical
      name: eth0
      subnets:
         - type: static
           address: 10.0.0.5/24
           gateway: 10.0.0.254
    - type: nameserver
      address:
        - 8.8.8.8
        - 8.8.4.4
      search:
        - example.com
    
    - type: physical
      name: eth1
      subnets:
         - type: static
           address: 192.168.0.5/24

user-data:

#cloud-config

### RUN CMD
runcmd:
  - /usr/bin/apt -y install mc
  - /usr/bin/date >> /root/testfile
  
# set system default user
system_info:
  default_user:
    name: testuser
    gecos: Default Cloud User

# password auth - comment out if not using passwords
password: testpassword
ssh_pwauth: true
chpasswd: { expire: false }

# do some package management
#package_update: true
packages:
 - iptraf
 - mtr
 - screen
 - net-tools
 - atop

Generate iso

genisoimage -output myiso.iso -volid cidata -joliet -rock user-data meta-data network-config

Upload iso to XO
home -> storage -> iso's -> disks -> new disk

Prepare VM/template

Install Debian 10.6 with xo-tools
install cloud-init and cloud-initramfs-growroot
remove all network configuration leave just "allow-hotplug ethxxx"
(network config will be added in /etc/network/interfaces/50-cloud-init. It will not overwrite yours)
shutdown the machine and create the template if you want or just restart VM.
DO NOT LET IT BOOT! - keep it in grub.
attach your iso
let it run

It will do whatever you need on the machine.

bassopt

This post is deleted!