VM UUID via dmidecode does not match VM ID in xen-orchestra
-
This is related to a bug I filed on the XO-CCM (which is why I put it here). I have a series of Kubernetes worker node VM's created by loading the a base k8s node from a template. It appears that in this process, the UUID returned from dmidecode has been changed and does not reflect the UUID in XO.
I was wondering if there was a way to get the UUID to match up again so that I can get the CCM to start functioning.
-
It's likely because the VM UUID from a XAPI perspective is different (generated by XAPI) and you can fetch it by reading the appropriate xenstor entry.
-
@olivierlambert Ah. Ok. It looks like I can update the system uuid via
sudo dmidecode -s system-uuid -s <new_uuid>found from here, but if you happen to know if that should work, Id love a confirmation before I go and make a mess of a worker node.
-
OK. back where I started. that was a garbage AI-written "help" article that just spewed out garbage commands as if they actually worked. Still looking for a way to resetset the dmi UUID
-
Can you use another way than dmidecode to fetch the UUID? Because it's not the right way to get VM UUID (instead you should use
xenstore readwith the UUID key -
@olivierlambert Unfortunatly, the UUID is getting pulled by kubernetes itself for XO-CCM, so I have no control over that that Im aware.
-
Can you open an issue on the Github repo for the CCM?
-
@olivierlambert I already have an issue liked in my original post. do you want me to make a new issue about how the vm Id is retrieved? or just comment on the issue I already have?
-
No, it's fine, as long the issue contains all the relevant details it's fine
-
Hi,
Do you know what causes the system UUID to change and not match the VM UUID?In the tests I have run (with Debian and Microk8s), it never changed.
-
@Cyrille Genuinly no idea unfortunately. The only thing I can think that I've done is that the VMs are being loaded in from a template.
I had to deploy new VMs through my terraform recently, and I AM still seeing an incorrect system UUID on those, so I can only guess its somehow the template?
I am also seeing the serial number from DMIDECODE match the VM ID in xen orchestra as well still.
-
@deefdragon can you check if
/sys/hypervisor/uuidmatches your VMs UUID? -
@Cyrille I am seeing
/sys/hypervisor/uuidmatchingdmidecode's Serial NumbermatchingVM ID in xen orchestra.dmidecode's UUIDhowever is different from those three. -
@deefdragon All three values match for me with XCP-ng 8.3, Ubuntu 24.04 guests and kernel 6.8:
$ sudo cat /sys/devices/virtual/dmi/id/product_serial adc9b6ba-d187-0844-13a6-5f1dc155bf6e $ sudo cat /sys/devices/virtual/dmi/id/product_uuid adc9b6ba-d187-0844-13a6-5f1dc155bf6e $ sudo cat /sys/hypervisor/uuid adc9b6ba-d187-0844-13a6-5f1dc155bf6eFrom your Github issue, it looks like a confusion in byte order between the System UUID (6a87cb0f-ca4c-ffa5-3ca2-fc398fb25eac) and the XCP-ng VM UUID (0fcb876a-4cca-a5ff-3ca2-fc398fb25eac).
I'd check the guest kernel version first.
-
Ah, I see the pattern now. That is bizarre.
deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_serial 0b08f477-491a-a982-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_uuid 77f4080b-1a49-82a9-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/hypervisor/uuid 0b08f477-491a-a982-23c4-d224723624eaHeres the output from a series of commands I ran to get the info of the worker. Seeing 6.8 and 24.04 myself.
deef@k31-w-3bfbbe:~$ cat /etc/*-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=24.04 DISTRIB_CODENAME=noble DISTRIB_DESCRIPTION="Ubuntu 24.04.2 LTS" PRETTY_NAME="Ubuntu 24.04.2 LTS" NAME="Ubuntu" VERSION_ID="24.04" VERSION="24.04.2 LTS (Noble Numbat)" VERSION_CODENAME=noble ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=noble LOGO=ubuntu-logo deef@k31-w-3bfbbe:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 24.04.2 LTS Release: 24.04 Codename: noble deef@k31-w-3bfbbe:~$ hostnamectl Static hostname: k31-w-3bfbbe Icon name: computer-vm Chassis: vm 🖴 Machine ID: 9bae21fe1e10d9da92da72599fd7b4a3 Boot ID: 959ebf2dc51e403eaf00103d51e8397e Virtualization: xen Operating System: Ubuntu 24.04.2 LTS Kernel: Linux 6.8.0-64-generic Architecture: x86-64 Hardware Vendor: Xen Hardware Model: HVM domU Firmware Version: 4.17 Firmware Date: Tue 2025-05-13 Firmware Age: 2month 2w 3d deef@k31-w-3bfbbe:~$ uname -r 6.8.0-64-genericLMK if there is anything else you want me to throw at it, or any other debug info you want me to get. I'm out of my depth when it comes to debugging this kind of thing.
-
@deefdragon How about host Xen and XAPI versions? I'm on the bleeding edge, so this may have been fixed somewhere already.
# unbuffer yum list installed | grep xen\\\|xapi mellanox-mlnxen.x86_64 5.9_0.5.5.0-2.1.xcpng8.3 @xcp:main/$releasever python2-xapi-storage.x86_64 24.19.2-1.10.xcpng8.3 @xcp-ng-testing xapi-core.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-nbd.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-rrd2csv.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-storage-script.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-tests.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-xe.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xcp-ng-xapi-plugins.noarch 1.12.0-1.xcpng8.3 @xcp-ng-testing xen-crashdump-analyser.x86_64 2.6.1-1.xcpng8.3 @xcp:main/$releasever xen-dom0-libs.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-dom0-tools.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-hypervisor.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-libs.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-tools.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xengt-userspace.noarch 4.0.0-1.xcpng8.3 @xcp:main/$releasever xenopsd.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xenopsd-cli.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xenopsd-xc.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xenserver-dracut.noarch 10-2.xcpng8.3 @xcp:main/$releasever xenserver-hwdata.noarch 20240411-1.xcpng8.3 @xcp:main/$releasever xenserver-status-report.noarch 2.0.11-1.xcpng8.3 @xcp-ng-ci -
Only difference looks to be xen-dom0 etc. on
4.17.5-13.1.xcpng8.3vs your4.17.5-15.2.xcpng8.3# unbuffer yum list installed | grep xen\\\|xapi mellanox-mlnxen.x86_64 5.9_0.5.5.0-2.1.xcpng8.3 @xcp:main/$releasever python2-xapi-storage.x86_64 24.19.2-1.10.xcpng8.3 @xcp-ng-updates xapi-core.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-nbd.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-rrd2csv.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-storage-script.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-tests.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-xe.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xcp-ng-xapi-plugins.noarch 1.12.0-1.xcpng8.3 @xcp-ng-updates xen-crashdump-analyser.x86_64 2.6.1-1.xcpng8.3 @xcp:main/$releasever xen-dom0-libs.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-dom0-tools.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-hypervisor.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-libs.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-tools.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xengt-userspace.noarch 4.0.0-1.xcpng8.3 @xcp:main/$releasever xenopsd.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xenopsd-cli.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xenopsd-xc.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xenserver-dracut.noarch 10-2.xcpng8.3 @xcp:main/$releasever xenserver-hwdata.noarch 20240411-1.xcpng8.3 @xcp:main/$releasever xenserver-status-report.noarch 2.0.11-1.xcpng8.3 @xcp-ng-updates -
@deefdragon Your XAPI is also lagging behind (25.6.0-1.5 vs latest 25.6.0-1.9). Could you update your pool?
-
deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_serial 0b08f477-491a-a982-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_uuid 77f4080b-1a49-82a9-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/hypervisor/uuid 0b08f477-491a-a982-23c4-d224723624eaIt looks like a endianness issue (0b08f477 vs 77f4080b).
-
@deefdragon can you provide us the output of
dmidecodein the guest ?
