@Cyrille said in DevOps Megathread: what you need and how we can help!:

Hi here,

We have released documentation with our recommendations for those who want to run Talos in Xen Orchestra.

It is available in the official Sidero Labs documentation: Platform specific installation > Virtualized Platforms > Xen Orchestra

We are currently working on providing an example of a Talos cluster that has been created using Terraform.

Have fun

Nice ! Thank you for this.

Fwiw, I started to work on a PoC deployment of a talos cluster this summer and managed to do the full process with pulumi. I'll probably migrate to opentofu when I have everything setup as I want to clean things up.

The whole install and k8s bootstrap takes 2-3 minutes to deploy a 3 nodes control plane and 3 workers from start to finish. You even get the machine configs, kubeconfig and talosconfig as outputs. Then, I deployed fluxcd manually but you could even automate that in the deployment.

Overall, its really impressive and quite easy once you understand the way talos works. Highly recommended !

Great news ! Thanks @olivierlambert and @florent and let me know if you need some information on the vmware side.

Thanks for the details @florent

@florent said in What is the status/roadmap of V2V (Migrating from VMware to XCPng/XO) ?:

the newer VMFS put more lock on the files, locking the full chain of snapshot and base disks instead of locking only the active disk.
Even VMFS5 sometimes lock the full chain.

That explains why I had locking issues trying to restart the source VM on vmware after a migration test.

I'll see if I can find anything on how to use NBD with vmware.

@Cyrille Thanks for the release !

I just tested the happy path of settings (6GB min / 8GB max and 8/8) and it seems to work as expected.
Now, I can get rid of the workaround, that's awesome

Hi @Cyrille, thank you for working on this, it will help a lot.

This is indeed implementing the needed behavior when creating vms from templates, avoiding the use of additional ad-hoc code. For reference, I needed to call the vm.set XO JSON-RPC function after the creation with the following arguments

            rpc_args = {
                "id": vm_id,
                "memory": vm_memory,
                "memoryMin": vm_memory,
                "memoryMax": vm_memory,
                "memoryStaticMax": vm_memory,
            }
            await rpcserver.vm.set(**rpc_args)

I'll try to test the provider patch in the coming days.

Awesome ! Thanks Vates and DESY for all the work that went into this.

I'm really looking forward to use the pulumi provider when I get to the automation part of my tests (probably in a few months though).

@Cyrille said in DevOps Megathread: what you need and how we can help!:

@afk Nice! I haven't tried Pulumi yet. Does Talos have a provider for it?

I'm working on improving the CCM to make it work with Talos. Especially the cloud-node-label-sync controller (as the cloud-controller is the Talos one) to get the XO topology in the node labels.

Yes there is a provider for it: https://www.pulumi.com/registry/packages/talos/

I guess its the auto-generated provider from the terraform one because I had some type checking errors in some places, but using python I could ignore that... I don't know if it has been fixed since.
I remember reading some terraform examples on blogs to write the pulumi config so converting one to the other should be straightforward. Unfortunately, I can't publish what I wrote because its quite messy but feel free to ask if you have some blockers.

The CCM would be nice to have indeed ! Good luck on that.

@Cyrille said in DevOps Megathread: what you need and how we can help!:

Hi here,

We have released documentation with our recommendations for those who want to run Talos in Xen Orchestra.

It is available in the official Sidero Labs documentation: Platform specific installation > Virtualized Platforms > Xen Orchestra

We are currently working on providing an example of a Talos cluster that has been created using Terraform.

Have fun

Nice ! Thank you for this.

Fwiw, I started to work on a PoC deployment of a talos cluster this summer and managed to do the full process with pulumi. I'll probably migrate to opentofu when I have everything setup as I want to clean things up.

The whole install and k8s bootstrap takes 2-3 minutes to deploy a 3 nodes control plane and 3 workers from start to finish. You even get the machine configs, kubeconfig and talosconfig as outputs. Then, I deployed fluxcd manually but you could even automate that in the deployment.

Overall, its really impressive and quite easy once you understand the way talos works. Highly recommended !

@dalem That looks awesome ! Glad to see nix/nixos getting more and more adoption.

It must have been a pain to build the pkgs, thanks a lot for working on this. I'll give it a try when I manage to free some time.

Have you considered upstreaming the module/pkgs to nixpkgs or is this too much work ? I have not contributed there yet so I don't know if it would get merged as is.

@Cyrille Thanks for the release !

I just tested the happy path of settings (6GB min / 8GB max and 8/8) and it seems to work as expected.
Now, I can get rid of the workaround, that's awesome

Hi @Cyrille, thank you for working on this, it will help a lot.

This is indeed implementing the needed behavior when creating vms from templates, avoiding the use of additional ad-hoc code. For reference, I needed to call the vm.set XO JSON-RPC function after the creation with the following arguments

            rpc_args = {
                "id": vm_id,
                "memory": vm_memory,
                "memoryMin": vm_memory,
                "memoryMax": vm_memory,
                "memoryStaticMax": vm_memory,
            }
            await rpcserver.vm.set(**rpc_args)

I'll try to test the provider patch in the coming days.

Hi, after working for some time on templating and deployment with packer and pulumi, I decided to publish complete examples.

https://github.com/Kaelnor/xcp-ng-pulumi-packer-example

Hopefully, this can be useful to someone else. I tried to give enough details in the README and code, especially for issues or limitations I faced.

Of course, I did not come up with all this by myself so thanks to every one on this forum that shared their previous efforts that I could build on. I tried to cite most sources I remember using when creating this.

Feel free to comment, fork or propose improvements.

Great news ! Thanks @olivierlambert and @florent and let me know if you need some information on the vmware side.

Has there been any progress/decision on the V2V improvements for recent vsphere versions ?

Hi, I'm currently testing deployments with pulumi using packer templates.

So far the basics work as expected but I'm stuck on a setting issue that seems to affect both pulumi and terraform providers. As far as I know there is no way to set the memory as static or changing memory_min when creating a VM from a template.

The template was created with 1cpu and 2GB of RAM

The VM created from this template using pulumi was assigned 2cpus and 4GB of RAM, however this only sets memory_max

I found the following post that talks about this: https://xcp-ng.org/forum/topic/5628/xenorchestra-with-terraform

and also the folllowing github issue https://github.com/vatesfr/terraform-provider-xenorchestra/issues/211

Manually setting the memory limits after VM creation defeats the purpose of automation. I suppose that implementing those settings in the relevant providers is a core feature. In most cases, VMs need static memory limits.

In the meantime, is there any workaround that I should investigate or anything that I missed ?

EDIT: Using the JSON-RPC API of XenOrchestra, I'm able to set the memory limits after the creation of the VM. This is great but unfortunately it is a bit too "imperative" in a declarative world.
I'll publish the code when I can clean up the hellish python I wrote, but a few pointers for those interested:

• See Mickaël Baron's blog (in French, sorry!) for an exemple of working with XO JSON-RPC API: https://mickael-baron.fr/blog/2021/05/28/xo-server-websocket-jsonrcp

• The system.getMethodsInfo() RPC function will give you all available calls you can make to the server. For instance, you can sign-in with session.signInWithToken(token="XO_TOKEN") and vm.setAndRestart to change VM settings and restart it immediately after.

• You can use Pulumi's hooks: https://www.pulumi.com/docs/iac/concepts/options/hooks/

• In python, Pulumi is running in an asyncio loop already so bear that in mind: https://www.pulumi.com/docs/iac/languages-sdks/python/python-blocking-async/

slax81 created this issue in vatesfr/terraform-provider-xenorchestra

closed Dynamic memory control #211

Awesome ! Thanks Vates and DESY for all the work that went into this.

I'm really looking forward to use the pulumi provider when I get to the automation part of my tests (probably in a few months though).