This is what I did:

Downloaded from https://cdimage.debian.org/images/cloud/ the latest bookworm raw file and imported it as a disk in XO

Booted a VM with a random template and some network (internet access will be usefull in a few steps)

Deleted the existing disk and attached the raw disk I uploaded

Converted the VM to template

Created a VM from this template with my ssh key

Once booted, you will need to install dmidecode (https://packages.debian.org/search?keywords=dmidecode ) due diligence on your part to get the latest .deb, install with dpgk -i

Also install xcp-ng guest tools

Then run :

sudo cloud-init clean
sudo cloud-init clean --logs
rm /home/debian/.ssh/authorized_keys
sudo mkdir -p /var/lib/cloud/scripts/per-once/ (folders get deleted on cloud-init clean)
cd /var/lib/cloud/scripts/per-once/
sudo nano generate-machine-id.sh
(coming from user modem7 on github)

#!/bin/bash

# KVM UUID Recreator
# Use this for new VM's or templates that require a unique machine ID.

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 
   exit 1
fi

UUID=$(dmidecode -s system-uuid | tr -d '-')
if grep -q "$UUID" /etc/machine-id; then
    echo "UUID matches"
else
    echo "UUID does not match. Recreating."
    echo -n > /etc/machine-id && echo -n > /var/lib/dbus/machine-id && systemd-machine-id-setup && reboot
fi

chmod +x generate-machine-id.sh

sudo cat /dev/null > ~/.bash_history && history -c && shutdown now

You can now rename the VM and it's disk, delete the network card to prevent the template to have some tags added automatically with the IPV4 and IPV6 and convert the VM to a template.

You should now have a working Debian 12 template accessible with your ssh key if you add it on deploy and DHCP working and not overlapping. Hopefully, I did not forget anything.

On first start, the VM will loop once after the first prompt. The reboot is required for the change of the machine-id to be effective.

This is a lot of work and I have no doubt there is a simpler solution but I couldn't find it.