SethNY

SethNY

Greetings! I am seasoned security engineer and that brings with it the need for a lab, automation, and prototyping. Intel NUCs, VMware workstation, and ESXi in my home lab was my MO.

For the past few years I have been putting my home Lab projects on github, and they now include XCP-ng testing. My lab is now split XCP-ng and proxymox. (good-bye Broadcom)

In the Cyber space I'm blue team, shifting more purple. The authentication with XCP-ng bugs me, but I'm happy with the automation potential.

Folding@Home and Unifi are my other fun projects.

SethNY

Sorry for the necropost but here is what I did.

import XenAPI
import ssl

HOST_IP = "192.168.1.100"
USERNAME = "root"
PASSWORD = "hostpasswordsecret"
VM_LIST = ('sms', 'firewall1a', 'firewall1b', 'firewall2a', 'firewall2b', 'firewall3a', 'firewall3b')

def main():
    # disable https certificate checking
    if hasattr(ssl, '_create_unverified_context'):
        ssl._create_default_https_context = ssl._create_unverified_context
    url = f"https://{HOST_IP}"
    session = XenAPI.Session(url)
    try:
        print(f"Connecting to {HOST_IP}...")
        session.xenapi.login_with_password(USERNAME, PASSWORD, "1.0", "python-script")
    except XenAPI.Failure as e:
        print(f"XenAPI Error: {e}")
        return
    except Exception as e:
        print(f"General Error: {e}")
        return
    for vm in VM_LIST:
        print(f"Searching for VM: {vm}...")
        vms = session.xenapi.VM.get_by_name_label(vm)
        if len(vms) == 0:
            print(f"Error: VM '{vm}' not found.")
            continue
        vm_ref = vms[0]
        vif_refs = session.xenapi.VM.get_VIFs(vm_ref)
        if not vif_refs:
            print("No network interfaces found on this VM.")
            continue
        print(f"Found {len(vif_refs)} interface(s). Updating settings...")
        for vif in vif_refs:
            device = session.xenapi.VIF.get_device(vif)
            other_config = session.xenapi.VIF.get_other_config(vif)
            # ethtool-tx transmit checksum offload
            # ethtool-tso TCP segmentation offload
            # ethtool-ufo UDP fragmentation offload
            # ethtool-gro generic receive offload
            if other_config.get('ethtool-tx') == 'off':
                print(f"  Interface {device}: TX Checksumming already disabled.")
            else:
                print(f"Disabling TX checksumming for interface {device}")
                other_config['ethtool-tx'] = 'off'
                try:
                    session.xenapi.VIF.set_other_config(vif, other_config)
                    print(f" - Interface {device}: TX Checksumming disabled (ethtool-tx: off)")
                    power_state = session.xenapi.VM.get_power_state(vm_ref)
                    if power_state == 'Running':
                        print("  [!] VM is RUNNING. A reboot is required for these changes to take effect.")
                    elif power_state == 'Halted':
                        print("  [i] VM is Halted. Changes will apply on next boot.")
                    else:
                        print(f"  [i] VM state is {power_state}.")
                        print("Note: You must reboot the VM or unplug/plug the VIFs for changes to take effect.")
                    print("")
                except XenAPI.Failure as e:
                    print(f"XenAPI Error: {e}")
                except Exception as e:
                    print(f"General Error: {e}")            
    try:
        session.xenapi.logout()
    except:
        pass
if __name__ == "__main__":
    main()

SethNY

Sorry for the necropost but here is what I did.

import XenAPI
import ssl

HOST_IP = "192.168.1.100"
USERNAME = "root"
PASSWORD = "hostpasswordsecret"
VM_LIST = ('sms', 'firewall1a', 'firewall1b', 'firewall2a', 'firewall2b', 'firewall3a', 'firewall3b')

def main():
    # disable https certificate checking
    if hasattr(ssl, '_create_unverified_context'):
        ssl._create_default_https_context = ssl._create_unverified_context
    url = f"https://{HOST_IP}"
    session = XenAPI.Session(url)
    try:
        print(f"Connecting to {HOST_IP}...")
        session.xenapi.login_with_password(USERNAME, PASSWORD, "1.0", "python-script")
    except XenAPI.Failure as e:
        print(f"XenAPI Error: {e}")
        return
    except Exception as e:
        print(f"General Error: {e}")
        return
    for vm in VM_LIST:
        print(f"Searching for VM: {vm}...")
        vms = session.xenapi.VM.get_by_name_label(vm)
        if len(vms) == 0:
            print(f"Error: VM '{vm}' not found.")
            continue
        vm_ref = vms[0]
        vif_refs = session.xenapi.VM.get_VIFs(vm_ref)
        if not vif_refs:
            print("No network interfaces found on this VM.")
            continue
        print(f"Found {len(vif_refs)} interface(s). Updating settings...")
        for vif in vif_refs:
            device = session.xenapi.VIF.get_device(vif)
            other_config = session.xenapi.VIF.get_other_config(vif)
            # ethtool-tx transmit checksum offload
            # ethtool-tso TCP segmentation offload
            # ethtool-ufo UDP fragmentation offload
            # ethtool-gro generic receive offload
            if other_config.get('ethtool-tx') == 'off':
                print(f"  Interface {device}: TX Checksumming already disabled.")
            else:
                print(f"Disabling TX checksumming for interface {device}")
                other_config['ethtool-tx'] = 'off'
                try:
                    session.xenapi.VIF.set_other_config(vif, other_config)
                    print(f" - Interface {device}: TX Checksumming disabled (ethtool-tx: off)")
                    power_state = session.xenapi.VM.get_power_state(vm_ref)
                    if power_state == 'Running':
                        print("  [!] VM is RUNNING. A reboot is required for these changes to take effect.")
                    elif power_state == 'Halted':
                        print("  [i] VM is Halted. Changes will apply on next boot.")
                    else:
                        print(f"  [i] VM state is {power_state}.")
                        print("Note: You must reboot the VM or unplug/plug the VIFs for changes to take effect.")
                    print("")
                except XenAPI.Failure as e:
                    print(f"XenAPI Error: {e}")
                except Exception as e:
                    print(f"General Error: {e}")            
    try:
        session.xenapi.logout()
    except:
        pass
if __name__ == "__main__":
    main()

SethNY

Starting from Other install media with BIOS (not uefi) is the only combo that is working with Terraform

SethNY

BLUF: creating new VMs from my custom Windows Windows Server 2025 templates works, but using terraform I get failures.

XCP-ng 8.3, XO from sources.

Console messages
Console messages
Boot device: Hard Disk - success.
Boot device: CD-Rom - failure: could not read boot disk

No bootable device.
Powering off in 30 seconds.

Workflow:

create a VM from base (built-in Windows Server 2025) template, Windows Server 2025 evaluation ISO
applied customization (e.g., increase/set display resolution
ran sysprep
converted to template
test by manually creating a VM from the template = success
using terraform, all servers created do not boot

Background:
https://xcp-ng.org/forum/topic/9474/terraform-creating-vm-from-my-windows-templates-no-bootable-device

Previously with 8.2 and Server 2022 I was able to change my workflow to use "Other installation media" as my base template, and it worked. Currently using "Other installation media" as my base template also fails. Windows 11 using the recommended base template is fine. Ubuntu Noble Numbat 24.04 base template does not work.

Excepts from servers.tf
data "xenorchestra_template" "server2025-template" {
name_label = "server2025-template"
}
[...]
resource "xenorchestra_vm" "dmz-iis" {
memory_max = 4294934528
cpus = 1
name_label = "dmz-iis"
name_description = "Windows Server 2025 in DMZ running IIS"
template = data.xenorchestra_template.server2025-template.id
depends_on = [xenorchestra_network.network_dmz1]
disk {
sr_id = data.xenorchestra_sr.local.id
name_label = "dmz-iis-disk"
size = 137438953472
}
network {
network_id = data.xenorchestra_network.branch1dmz.id
}
}
[...]

Full details

https://github.com/doritoes/NUC-Labs/blob/xcp-ng-8.3/XCP-ng/terraform/servers.tf

https://github.com/doritoes/NUC-Labs/blob/xcp-ng-8.3/XCP-ng/Appendix-Terraform.md

SethNY

UPDATE Now on XCP-ng 8.3 and XO from sources

I had no problems with Windows 11.

Had problem Ubuntu Noble Numbat 24.04 base template failing to boot from disk when cloned using Terraform. Changing the base template to "Other install media" resolved the issue, and the Terraform-cloned VMs work fine.

EDIT: Also has problems with base template Windows Server 2025.

SethNY

@acebmxer thank you for testing and confirming!

For those who try this...

NOTE WSL 1 has the issue of throwing the error Failed to take /etc/passswd lock: Invalid argument
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2069555/comments/12
Here is a fix
sudo sed -i -e '/systemd-sysusers/s/.conf$/.conf || true/' /var/lib/dpkg/info/*.postinst
sudo apt --fix-broken install

SethNY

@TeddyAstie you give me the answer and a valid workaround. You rock!

Confirmed that solved my issue, as WSL 1 is fine for me to run Ansible.

SethNY

Error code: Wsl/InstallDistro/Service/RegisterDistro/CreateVm/HCS/HCS_E_HYPERV_NOT_INSTALLED

Additionally tried this:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All

EDIT: Add Next steps tried
Open the Turn Windows features on or off panel
Turn these settings off: Hyper-V, Windows Subsystem for Linux, Virtual Machine Platform, click OK, and restart your computer.
Turn these settings ON: Hyper-V, Windows Subsystem for Linux, Virtual Machine Platform, click OK, and restart your computer.

EDIT2:
dism.exe /online /get-features /format:table
HypervisorPlatform was disabled
Enable Windows Hypervisor Platform feature and rebooted.

This did not fix the issue.

EDIT3:
It looks like WSL on Windows 11 might be unsupported on XGP-ng 8.3
https://docs.xcp-ng.org/compute/#-nested-virtualization

SethNY

XCP-ng 8.3, XO from sources.
Created Win11 from ISO using the built-in Windows 11 template
Configured and turned into template for cloning.

Trouble installing WSL on the cloned Win11:
WSL2 is not supported with your current machine configuration

This worked a couple years ago Win10 on XCP-ng 8.2, Ubuntu 22.04.

From administrative powershell

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
(Accept the reboot, back to administrative powersehll)
wsl --install
(fails)
wsl --list --online
wsl --install -d Ubuntu-24.04
(fails again)

I tried enabling Nested Virtualization for the VM without success
Booted to (F2) BIOS and confirmed not virtualization options there to enable

Has anyone got Win11/WSL/8.3 working? I'm hoping it's not due to not installing a Windows license key.

SethNY

@bvitnik Thank you for the great response. I have had great success with Terraform - great work.

I'm not touching ocaml myself. And yes, Citrix... they are still above VMware/Broadcom on my list. But SMH.

I keep promoting XCP-ng hoping some large companies take advantage of it. It's much more valuable to me than nautobot, for example.

SethNY

@olivierlambert and @bvitnik thank you for those thoughts.

I'm thinking at a larger scale than I need right now.

In the future I would like to be able to do things like: migrate all VMs matching <tag x> or name similar to host B (because it's a new host or because I'm taking down A for patching). Ansible is idempotent and perfect for things like that.

Move all large disks to my thin-provisioned SRs.

Move all powered down VM's disk storage to my NFS SR.

Move all VMs with "automatically power on" to host A.

Maybe someday.

SethNY

@SethNY

Best posts made by SethNY

Latest posts made by SethNY