Cloudbase-init on Windows

DustinB

@Andrew7 XCP-ng has always been free to use, the same for Xen Orchestra (from source) you can look at my profile to find a link or read the documentation to deploy it yourself.

You've hijacked someone else's thread to ask your question because their question was similar to yours, I guess?

And you seem to have an arrogance about your inquiry of "it just works on this other product that we're fleeing from that would cost us as much as 300% more every year forever".

And on the same coin, other side you're saying you haven't the faintest reason why anyone would be upset with your tone, while asking for free help.

Support for a 3 host pack, is as cheap as $1000/year or $3000, with essentially unlimited and confidential access for a product which has a long tract record.

dinhngtu

Hi @jkatz @Andrew7,

Cloudbase-init is currently the recommended method for personalizing Windows templates in XO. It supports multiple configuration plugins (hostname/credentials/licensing etc.), which are available from the linked page above.

For customizing deployed VMs, what I like to do is to prepare a "skeleton" cloud-init file and set it in XO Settings -> Cloud config. Then I can load this config at any time and fill in the details later for each batch of VM I want to deploy.

We do have a Packer repo for making master VM templates from @nathanael-h; it will prepare an XVA image ready for specialization using cloudbase-init. I'll try to work to publish that and provide some documentation next.

As for Sysprep support specifically, I've put in a request to the XO team, although I can't promise anything specific for now.

Andrew7

@dinhngtu

Thank you for the replay.

I understand that cloudbase-init is a way to go. I did similar config to @jkatz and don't know why it doesn't work for me.
I did exactly as you suggested, created Cloud Config settings in XO and then use that during deployment screen.

I've tried to set it up with this guide
https://docs.xen-orchestra.com/vm-templates

So I've added those fields :
config_drive_vfat=true
...
metadata_services=cloudbaseinit.metadata.services.nocloudservice.NoCloudConfigDriveService
plugins=cloudbaseinit.plugins.common.userdata.UserDataPlugin

into config files.

I'm probably doing something wrong but I don't know where to look.
It seems that cloudbase-init doesn't even start or it is skipped because my input is totally incorrect in those "User config " and "Network Config" fields.

The log folder here is empty

Do you have any ideas why it doesn't work ?
It might be the case that I'm just an idiot and did some rookie mistake as I have never used cloudbase-init before.
So, I don't really know if the problem is my Custom Config settings in XO or cloudbase-init config on VM.

@dinhngtu said in Cloudbase-init on Windows:

As for Sysprep support specifically, I've put in a request to the XO team, although I can't promise anything specific for now.

Thank you.

dinhngtu

@Andrew7 Did you Sysprep the template using cloudbase-init's included Unattend.xml file? That file is needed to invoke cloudbase-init after the image is unsealed.

For the cloudbase-init configuration, you only need to add the following line to both of its config files:

metadata_services=cloudbaseinit.metadata.services.nocloudservice.NoCloudConfigDriveService

No need to add any other line to cloudbase-init.conf unless you want to enable/disable specific plugins.

I'm preparing an updated guide for Windows VM template creation: https://github.com/vatesfr/xen-orchestra/pull/8575. Stay tuned.

dinhngtu opened this pull request in vatesfr/xen-orchestra

closed docs(XO): add Windows cloudbase-init template guide #8575

dinhngtu

@jkatz I'm talking with the DevOps team to see if there's a way to discover/configure networking, perhaps via additional metadata provided through the cloud config drive.

Andrew7

@dinhngtu
Hello again,

Thanks for replay.
I did it all over again ( new cloudbase-init installation on new VM ), and used @jkatz config and it partially worked.

This was my settings just before OS shutdown

And this is the output, new VM and it's settings

What hasn't change during deployment is the host name. Network setting were changed according to Network Config file from XO.
Sysprep was also done.

So I will have to figure out why the hostname hasn't changed according to the XO Users Config.

jkatz

@Andrew7
Sounds like I'm in the same boat as you in trying to find an alternative to vmware for a fairly large deployment. And the PoC / testing will probably take around 6-12 months before we commit to any new platform. So support during the one month trial period isn't quite enough.

I would agree that the customization specifications in vcenter are easy to use and quite polished. But I do think cloudbase-init can get the minimal job done where the hostname and ip gets configured during deployment. Then i can rely on our automation tools to take it to the end. The only stopping point right now is exposing the new MAC address to the network config as a variable. But it sounds like that is hopefully being addressed.

Originally, i would install cloudbase, and not do the sysprep there and not shutdown. Then i would modify the cloudbase-init.conf. then sysprep manually, then shutdown. I think (been a while, but i have it written down in a onenote) this worked except for the newer OSs like 11 and 2025. So now my process is to let the cloudbase installer do the sysprep but not shutdown. I think the cloudbase sysprep does something a little different than just running it manually. Then i modify the config, then shutdown. Convert to template.

I think for 11 I also changed the administrator user to not 'user must change password'. That may or may not be needed.

Also, if the recovery partition is put after the primary, you'll want to nuke it so that the disk can be resized during deployment.

If you still can't get it working, i'll revisit my steps here to verify my process. Lemme know and I'll check.

dinhngtu

@Andrew7 @jkatz Did you use the cloudbase-init unattend.xml for sysprep? I also recommend starting your template from audit mode.

I've done several tests with my procedures from the PR linked above, and managed to set the VM's hostname.

jkatz

@dinhngtu

I'm testing this out on my test environment. But it have to rebuild the template as we borked the site where the templates were at.

MS introduced some bugs in some xapps in a recent patch cycle that breaks sysprep. I noticed this when i was creating a template in our vcenter environment. I'm currently using this ISO for my template build:

SW_DVD9_Win_Server_STD_CORE_2022_2108.31_64Bit_English_DC_STD_MLF_X23-71666.ISO

And then not applying any patches. That works for me in vcenter, but once I apply current patches, then it will not customize anymore. Not sure if it applies here in XO, but i thought i'd throw that out there too.

dinhngtu

@jkatz Do the issues occur even if the template was prepared from audit mode?

Andrew7

@dinhngtu
Still nothing. I've tried use only :
#cloudbase-config
hostname: VMName

I can also confirm what @jkatz discovered. When there is no MAC address in Network Config the static IP setup doesn't work.

@jkatz Yes, it seems that we are on the same boat.

How did you make that hostname change works ?

@dinhngtu What this audit mode does ?

@dinhngtu said in Cloudbase-init on Windows:

Did you use the cloudbase-init unattend.xml for sysprep?

There was that file on disk and I saw that it had a config in it. I have not touched it in any way.

Andrew7

By the way, how to edit Template ?

jkatz

@dinhngtu

Didn't bother with manual sysprep anymore. I just let the installer start it.

jkatz

@Andrew7

You don't. Pretty sure there is a dialog stating this is not reversable when setting a VM as a template.

What I do is have a source vm and then take a snapshot of it while off. Then create a vm off that snap. Can do versioning this way. Boot the new vm (from the snap), install cloudbase, copy over the cloudbase-init.conf over, shutdown, convert to template.

I also created my own ISO with the citrix tools, cloudbase installer, and cloudbase config to make it easier.

Making the hostname update was probably the first thing that got working. Check my cloud-config at the top. That should work. Just be sure to add those last three lines to the cloudbase config too.

And the only file i change is cloudbase-init.conf.

Andrew7

@jkatz Thanks for that.

I actually did something different but it worked. I don't know how but it is working.

I used this in XO in Cloud Config

It worked

Don't know if does it matter or not ( probably does) that I also entered this plugin into both files :
cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin

@jkatz said in Cloudbase-init on Windows:

You don't.

That is lame.

Wonder if domain join is possible with that...

tmk

@jkatz

I ran into the same issue when trying to configure a network adapter with cloudbase-init. The documentation says that the MAC address value is optional but in reality it is required.

In my case I want Xen Orchestra to choose a unique MAC during deployment, and I since I am deploying from a template, the NIC name is a known value. The fix that ended up working for me was to modify the networkconfig.py file in cloudbase-init so that the NIC name is required and the MAC address is optional.

I ended up making some additional changes to allow for the network-config v2 format along with the existing v1 support and some additional logic to aid in setting the dns search domains (I can't recall if this was originally supported or not but I had issues getting it to work with the original networkconfig.py file)

This file needs to replace the existing one that is installed in the C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\plugins\common\ directory. To replace it you should make sure that the cloudbase-init service is stopped, then replace the file.

Once this file is replaced, delete the pycache folder in the same parent folder as networkconfig.py - this will ensure that python recompiles this file on service start. Start the cloudbase-init service and confirm that you see a new pycache get created.

I'm not a python programmer by trade so others may be able to point out areas for improvement but this ended up working for me and I wanted to share in case it could help others needing to deploy new servers without manually specifying a new MAC address. Below is an example network-config v2 format that works with the updated file.

version: 2
ethernets:
Ethernet 2:
dhcp4: false
addresses:
- 10.20.30.10/24
nameservers:
addresses:
- 10.20.5.12
- 10.20.5.13
- 10.20.5.14
search:
- intranet.domain.org
- domain.org
- public-domain.org
routes:
- to: default
via: 10.20.30.1

Updated networkconfig.py file:
networkconfig.py.txt

MK.ultra

@tmk said in Cloudbase-init on Windows:

@jkatz

I ran into the same issue when trying to configure a network adapter with cloudbase-init. The documentation says that the MAC address value is optional but in reality it is required.

In my case I want Xen Orchestra to choose a unique MAC during deployment, and I since I am deploying from a template, the NIC name is a known value. The fix that ended up working for me was to modify the networkconfig.py file in cloudbase-init so that the NIC name is required and the MAC address is optional.

I ended up making some additional changes to allow for the network-config v2 format along with the existing v1 support and some additional logic to aid in setting the dns search domains (I can't recall if this was originally supported or not but I had issues getting it to work with the original networkconfig.py file)

This file needs to replace the existing one that is installed in the C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\plugins\common\ directory. To replace it you should make sure that the cloudbase-init service is stopped, then replace the file.

Once this file is replaced, delete the pycache folder in the same parent folder as networkconfig.py - this will ensure that python recompiles this file on service start. Start the cloudbase-init service and confirm that you see a new pycache get created.

I'm not a python programmer by trade so others may be able to point out areas for improvement but this ended up working for me and I wanted to share in case it could help others needing to deploy new servers without manually specifying a new MAC address. Below is an example network-config v2 format that works with the updated file.

version: 2
ethernets:
Ethernet 2:
dhcp4: false
addresses:
- 10.20.30.10/24
nameservers:
addresses:
- 10.20.5.12
- 10.20.5.13
- 10.20.5.14
search:
- intranet.domain.org
- domain.org
- public-domain.org
routes:
- to: default
via: 10.20.30.1

Updated networkconfig.py file:
networkconfig.py.txt

Thank you for posting! I followed your directions verbatim and I'm still having trouble getting the network config to take. Does it work with Windows Server 2025?

Pilow

@tmk hi !
Many thanks, your modified python file did the trick, my static IP address is now working as intented.

I can confirm this is working on Windows 2025 Server as well.

MK.ultra

Still having trouble getting it to take the network config, even with the new networkconfig.py file copied in and recompiled.

This is what I'm using:

network:
version: 2
ethernets:
Ethernet 2:
dhcp4: false
addresses:
- x.x.x.x/16
nameservers:
addresses:
- x.x.x.x
- x.x.x.x
search:
- my.domain.net
routes:
- to: default
via: x.x.x.x

Any ideas?

tmk

@MK.ultra I'd say make sure that your network-config is formatted properly with the correct YAML formatting (posting the content on this forum strips some of the formatting). Also, verify that you're using the correct network adapter name, "Ethernet 2" was just the name of the NIC I was using in my template.

If that's not an issue the next step would be to check the cloudbase-init logs to see what's reported there. You may need to enable debug logging in the cloudbase config file to get all of the relevant info depending on the issue.