@Cyrille said in How to deploy the new k8s on latest XOA 5.106?:

@ferrao open the VMs list (Home>VMs), click on "Filters" near the search bar, and then click "Save...", in the popup dialog enter a name for the filter and click "OK".

This should be enough to workaround the 2nd bug that happened at the end (that prevent to save the tag that is added to the clusters VMs)

Hello. Thanks, it seems to have worked. The deployment screen no longer hangs. However I don't get any feedback at the finish.

And on the Task log, the log about the Kubernetes cluster seems to be gone:

Cannot GET /rest/v0/tasks/0mabaot5t

I've selected this task:

API call: xoa.recipe.createKubernetesCluster			2025-05-05 13:28	

Is this expected?

@Cyrille said in How to deploy the new k8s on latest XOA 5.106?:

@ferrao Thank you for the logs, with it I was able to reproduce the bug.

It's at the end of the recipe, when the tag associated with the k8s cluster is saved in the user's custom filters. This has no effect on the cluster creation, as this is done at the end when everything else is done.

To avoid this bug in future runs, you can save a custom filter from the search field in the VMs list.

About the first error you encounter, it's related to the template used to create the VM, which seems not available at the VM creation... I'm working on this to understand why.

I think I was able to nail down the first one as a DHCP server with insufficient leases for all VMs. Because when I used static IP addresses I was able to generate the second log.

I'll try to create the tag you mentioned now and redeploy everything from scratch. Is there any specific text tag that I should create? I'm not sure if I understand 100% the procedure.

I'm trying to deploy the updated k8s cluster on XOA 5.106 but it fails quietly.

After digging through the logs I was able to find this:

{
  "id": "0ma67t63q",
  "properties": {
    "method": "xoa.recipe.createKubernetesCluster",
    "params": {
      "clusterName": "VersatusHPC",
      "controlPlanePoolSize": 3,
      "k8sVersion": "1.33",
      "nbNodes": 3,
      "network": "4751905f-b4db-2d54-d05d-0c1be97a0260",
      "sr": "f2d0eb72-4016-4b3b-8dc0-bc2a16df6c35",
      "sshKey": "ssh-ed25519 AAAAC3NzaC1lZDI"
    },
    "name": "API call: xoa.recipe.createKubernetesCluster",
    "userId": "fd28fb18-c3f1-429b-919f-4e8ae57dde0e",
    "type": "api.call"
  },
  "start": 1746155348774,
  "status": "failure",
  "updatedAt": 1746155466728,
  "end": 1746155466728,
  "result": {
    "code": 10,
    "data": {
      "errors": [
        {
          "instancePath": "",
          "schemaPath": "#/required",
          "keyword": "required",
          "params": {
            "missingProperty": "template"
          },
          "message": "must have required property 'template'"
        }
      ]
    },
    "message": "invalid parameters",
    "name": "XoError",
    "stack": "XoError: invalid parameters\n    at Module.invalidParameters (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)\n    at Xo.call (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:121:22)\n    at Api.#callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:409:19)\n    at Xoa.createCluster (/usr/local/lib/node_modules/xo-server-xoa/src/recipes/kubernetes-cluster.js:262:28)\n    at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:175:22)\n    at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:159:20)\n    at Api.#callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:469:18)"
  }
}

However that wasn't that helpful.

I tried changing options and atributes and got a little further but it ended up with a similar issue:

{
  "id": "0ma78aljw",
  "properties": {
    "method": "xoa.recipe.createKubernetesCluster",
    "params": {
      "clusterName": "VersatusHPC",
      "controlPlaneIpAddresses": [
        "10.20.0.151/24",
        "10.20.0.152/24",
        "10.20.0.153/24"
      ],
      "controlPlanePoolSize": 3,
      "gatewayIpAddress": "10.20.0.1",
      "k8sVersion": "1.33",
      "nameservers": [
        "10.20.0.1"
      ],
      "nbNodes": 3,
      "network": "a28fd0a8-70e2-8fc6-cefa-12c46c8f47cb",
      "searches": [
        "local.versatushpc.com.br",
        "versatushpc.com.br"
      ],
      "sr": "f2d0eb72-4016-4b3b-8dc0-bc2a16df6c35",
      "sshKey": "ssh-ed25519 AAAAC3NzaC1lZDI1N",
      "vipAddress": "10.20.0.150/24",
      "workerNodeIpAddresses": [
        "10.20.0.154/24",
        "10.20.0.155/24",
        "10.20.0.156/24"
      ]
    },
    "name": "API call: xoa.recipe.createKubernetesCluster",
    "userId": "fd28fb18-c3f1-429b-919f-4e8ae57dde0e",
    "type": "api.call"
  },
  "start": 1746216628124,
  "status": "failure",
  "updatedAt": 1746217416799,
  "end": 1746217416798,
  "result": {
    "message": "Cannot read properties of undefined (reading 'VM')",
    "name": "TypeError",
    "stack": "TypeError: Cannot read properties of undefined (reading 'VM')\n    at Xoa.createCluster (/usr/local/lib/node_modules/xo-server-xoa/src/recipes/kubernetes-cluster.js:488:22)\n    at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:175:22)\n    at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:159:20)\n    at Api.#callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:469:18)"
  }
}

I can see 3 control planes and 3 works were deployed. However it does not complete the task.

What steps I further need to take to investigate the issue?

Thanks.

@Danp said in XOA fails after update to 5.106.0:

@ferrao Prior to the latest release, the trial worked without actually activating the license on the XOA > Licenses tab. Double check that the license is activated (should look like this) --

This is what it will look like if the license hasn't been activated --

Oh man...
That's a new thing? You're totally right.

It's now working correctly again.

@Danp said in XOA fails after update to 5.106.0:

@ferrao Make sure that you've activated the trial license under the XOA > Licenses tab.

Yes, it is. I was testing the User Portal on April 28th. Received the e-mail message regarding the Micro K8s and tried updating to give it a try.

Now XOA is locked with this message.

@Danp said in XOA fails after update to 5.106.0:

Hi @ferrao,

This is with XOA Free, correct? Have you tried switching to the Stable release channel?

Dan

It's on trial mode. So I think it can be considered paid.

@Danp we are also with the license has expired message:

Your current Xen Orchestra license has expired (Dec 31, 1969). Please reach out to your vendor.

Looks like something related to Unix time bug? Something may be set to zero or undefined.

@lover said in XOSTOR hyperconvergence preview:

Anyone else getting a 301 error?

http://mirrors.xcp-ng.org/8/8.2/base/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 301 - Moved Permanently
Trying other mirror.

301 is not an error (as a failure) it's a redirect. Here it redirects correctly to a mirror nearby. In my case: https://mirror.uepg.br/xcp-ng/8/8.2/base/x86_64/repodata/repomd.xml

@ronan-a thanks. I've deployed it already with the script on the first post. Seems to be working. I've opted to used redundancy=3 in a 3 hosts setup. It's a lot of 'wasted' resources but seems to be the best option for performance and reliability.

May I ask now a licensing issue: if we upgrade to Vates VM, does the deployment mode on the first message is considered supported or everything will need to be done again from XOA?

Thanks.

@ronan-a and @Maelstrom96 I didn't get this hostname issue.

Does XOSTOR needs a fully functional DNS setup to work? Or the failure was local due to the local change of the hostname?

I didn't understand if the communication is done by IP addresses directly or if DNS name resolution is needed.

I'm particularly interested in this because with XOSTOR I'm considering virtualizing my pfSense firewall directly and get rid of the physical servers. And in this scenario in a case of a entire pool reboot I must guarantee that I will have the two pfSense VMs up and running, with the option to auto start after reboot, so I can access the entire infrastructure or else I'll be locked from outside.

@stormi Thanks @stormi.

I have to get used to XO. I always use XCP-ng Admin on Windows because old habits die hard, and there I cannot even select UEFI with all the Linux templates.

The magic that XO does is that it allows to use the templates, but with UEFI.

I have a VM that I've deployed with the "Other install media" option, so I just dettached the disk and reimported on the new one created by XO, with the correct template. Everything seems fine.

So my question about inheriting the template is not relevant if we use XO. Hope my understand is correct.

Thanks.

@KPS @stormi may I ask another question:

Yes I do see thats some settings when I ask for the profile using xe:

[15:19 xen1 ~]# xe template-param-list uuid=6c91b878-5095-421e-a914-224b3bb1088c 
uuid ( RO)                                  : 6c91b878-5095-421e-a914-224b3bb1088c
                            name-label ( RW): Red Hat Enterprise Linux 9
                      name-description ( RW): To use this template from the CLI, install your VM using vm-install, then set other-config-install-repository to the path to your network repository, e.g. http://<server>/<path> or nfs:server:/<path>
                          user-version ( RW): 1
                         is-a-template ( RW): true
                   is-default-template ( RW): true
                         is-a-snapshot ( RO): false
                           snapshot-of ( RO): <not in database>
                             snapshots ( RO): 
                         snapshot-time ( RO): 19700101T00:00:00Z
                         snapshot-info ( RO): 
                                parent ( RO): <not in database>
                              children ( RO): 
                     is-control-domain ( RO): false
                           power-state ( RO): halted
                         memory-actual ( RO): 0
                         memory-target ( RO): 0
                       memory-overhead ( RO): 36700160
                     memory-static-max ( RW): 4294967296
                    memory-dynamic-max ( RW): 4294967296
                    memory-dynamic-min ( RW): 4294967296
                     memory-static-min ( RW): 2147483648
                      suspend-VDI-uuid ( RW): <not in database>
                       suspend-SR-uuid ( RW): <not in database>
                          VCPUs-params (MRW): 
                             VCPUs-max ( RW): 1
                      VCPUs-at-startup ( RW): 1
                actions-after-shutdown ( RW): Destroy
                  actions-after-reboot ( RW): Restart
                   actions-after-crash ( RW): Restart
                         console-uuids (SRO): 
                                   hvm ( RO): false
                              platform (MRW): videoram: 8; hpet: true; secureboot: false; device-model: qemu-upstream-compat; apic: true; device_id: 0001; vga: std; nx: true; pae: true; viridian: false; acpi: 1
                    allowed-operations (SRO): changing_NVRAM; changing_dynamic_range; changing_shadow_memory; changing_static_range; provision; export; clone; copy
                    current-operations (SRO): 
                    blocked-operations (MRW): 
                   allowed-VBD-devices (SRO): 0; 1; 2; 3; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254
                   allowed-VIF-devices (SRO): 0; 1; 2; 3; 4; 5; 6
                        possible-hosts ( RO): e49f21ab-723c-472e-8ad2-46c169e31172; 488e0f1d-69f3-4ecb-a37b-b5328764c229; f6d05b66-a21b-49bc-ad57-b4ca9bad756f
                           domain-type ( RW): hvm
                   current-domain-type ( RO): unspecified
                       HVM-boot-policy ( RW): BIOS order
                       HVM-boot-params (MRW): firmware: bios; order: cdn
                 HVM-shadow-multiplier ( RW): 1.000
                             PV-kernel ( RW): 
                            PV-ramdisk ( RW): 
                               PV-args ( RW): 
                        PV-legacy-args ( RW): 
                         PV-bootloader ( RW): 
                    PV-bootloader-args ( RW): 
                   last-boot-CPU-flags ( RO): 
                      last-boot-record ( RO): ''
                           resident-on ( RO): <not in database>
                              affinity ( RW): <not in database>
                          other-config (MRW): default_template: true; import_task: OpaqueRef:5bfca08c-951b-4024-a329-60260d8936fe; mac_seed: 99974ea1-478b-426d-b483-960e65912bd8; disks: <provision><disk bootable="true" device="0" size="10737418240" sr="" type="system"/></provision>; install-methods: cdrom,nfs,http,ftp; linux_template: true
                                dom-id ( RO): -1
                       recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="32"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions>
                         xenstore-data (MRW): 
            ha-always-run ( RW) [DEPRECATED]: false
                   ha-restart-priority ( RW): 
                                 blobs ( RO): 
                            start-time ( RO): 19700101T00:00:00Z
                          install-time ( RO): 19700101T00:00:00Z
                          VCPUs-number ( RO): 0
                     VCPUs-utilisation (MRO): 
                            os-version (MRO): <not in database>
                    PV-drivers-version (MRO): <not in database>
    PV-drivers-up-to-date ( RO) [DEPRECATED]: <not in database>
                                memory (MRO): <not in database>
                                 disks (MRO): <not in database>
                                  VBDs (SRO): 
                              networks (MRO): <not in database>
                   PV-drivers-detected ( RO): <not in database>
                                 other (MRO): <not in database>
                                  live ( RO): <not in database>
            guest-metrics-last-updated ( RO): <not in database>
                   can-use-hotplug-vbd ( RO): <not in database>
                   can-use-hotplug-vif ( RO): <not in database>
              cooperative ( RO) [DEPRECATED]: true
                                  tags (SRW): 
                             appliance ( RW): <not in database>
                     snapshot-schedule ( RW): <not in database>
                      is-vmss-snapshot ( RO): false
                           start-delay ( RW): 0
                        shutdown-delay ( RW): 0
                                 order ( RW): 0
                               version ( RO): 1
                         generation-id ( RO): 
             hardware-platform-version ( RO): 0
                     has-vendor-device ( RW): false
                       requires-reboot ( RO): false
                       reference-label ( RO): rhel-9
                          bios-strings (MRO): 

That's a lot of things.

The questions would be:

1. Is there a way to create another template that will inherit from the base template just to modify the UEFI support? Because let's be honest, it's 2024 and I don't feel like using legacy BIOS even for VMs... And when I mean inherit is that if something changes on the template it will be automatically changed on the inherited one. I'm aware that I can just clone/copy the template and modify but this will eventually lose updates to the base template.

2. Does XOA (the paid version) have more up to date templates with, for example, UEFI support for Linux? FreeBSD, etc?

Thank you all guys.

@stormi is there any benefit in using those templates instead of just using Other Install Media, that allows UEFI and UEFI secure boot?

@olivierlambert is there a way to mount the same disk in two VMs in RW mode? I have a lab that uses pacemaker inside the VM's and pacemaker handles the mounting/access of the volumes.

I have a working setup on oVirt using the "Shareable" option of a Virtual Disk. So I can attach to multiple VM's simultaneously. Is there anything similar on XCP-ng? Even if I have to hack through the command like to attach?

Thank you.