VM UUID via dmidecode does not match VM ID in xen-orchestra
-
Can you use another way than dmidecode to fetch the UUID? Because it's not the right way to get VM UUID (instead you should use
xenstore readwith the UUID key -
@olivierlambert Unfortunatly, the UUID is getting pulled by kubernetes itself for XO-CCM, so I have no control over that that Im aware.
-
Can you open an issue on the Github repo for the CCM?
-
@olivierlambert I already have an issue liked in my original post. do you want me to make a new issue about how the vm Id is retrieved? or just comment on the issue I already have?
-
No, it's fine, as long the issue contains all the relevant details it's fine
-
Hi,
Do you know what causes the system UUID to change and not match the VM UUID?In the tests I have run (with Debian and Microk8s), it never changed.
-
@Cyrille Genuinly no idea unfortunately. The only thing I can think that I've done is that the VMs are being loaded in from a template.
I had to deploy new VMs through my terraform recently, and I AM still seeing an incorrect system UUID on those, so I can only guess its somehow the template?
I am also seeing the serial number from DMIDECODE match the VM ID in xen orchestra as well still.
-
@deefdragon can you check if
/sys/hypervisor/uuidmatches your VMs UUID? -
@Cyrille I am seeing
/sys/hypervisor/uuidmatchingdmidecode's Serial NumbermatchingVM ID in xen orchestra.dmidecode's UUIDhowever is different from those three. -
@deefdragon All three values match for me with XCP-ng 8.3, Ubuntu 24.04 guests and kernel 6.8:
$ sudo cat /sys/devices/virtual/dmi/id/product_serial adc9b6ba-d187-0844-13a6-5f1dc155bf6e $ sudo cat /sys/devices/virtual/dmi/id/product_uuid adc9b6ba-d187-0844-13a6-5f1dc155bf6e $ sudo cat /sys/hypervisor/uuid adc9b6ba-d187-0844-13a6-5f1dc155bf6eFrom your Github issue, it looks like a confusion in byte order between the System UUID (6a87cb0f-ca4c-ffa5-3ca2-fc398fb25eac) and the XCP-ng VM UUID (0fcb876a-4cca-a5ff-3ca2-fc398fb25eac).
I'd check the guest kernel version first.
-
Ah, I see the pattern now. That is bizarre.
deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_serial 0b08f477-491a-a982-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_uuid 77f4080b-1a49-82a9-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/hypervisor/uuid 0b08f477-491a-a982-23c4-d224723624eaHeres the output from a series of commands I ran to get the info of the worker. Seeing 6.8 and 24.04 myself.
deef@k31-w-3bfbbe:~$ cat /etc/*-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=24.04 DISTRIB_CODENAME=noble DISTRIB_DESCRIPTION="Ubuntu 24.04.2 LTS" PRETTY_NAME="Ubuntu 24.04.2 LTS" NAME="Ubuntu" VERSION_ID="24.04" VERSION="24.04.2 LTS (Noble Numbat)" VERSION_CODENAME=noble ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=noble LOGO=ubuntu-logo deef@k31-w-3bfbbe:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 24.04.2 LTS Release: 24.04 Codename: noble deef@k31-w-3bfbbe:~$ hostnamectl Static hostname: k31-w-3bfbbe Icon name: computer-vm Chassis: vm 🖴 Machine ID: 9bae21fe1e10d9da92da72599fd7b4a3 Boot ID: 959ebf2dc51e403eaf00103d51e8397e Virtualization: xen Operating System: Ubuntu 24.04.2 LTS Kernel: Linux 6.8.0-64-generic Architecture: x86-64 Hardware Vendor: Xen Hardware Model: HVM domU Firmware Version: 4.17 Firmware Date: Tue 2025-05-13 Firmware Age: 2month 2w 3d deef@k31-w-3bfbbe:~$ uname -r 6.8.0-64-genericLMK if there is anything else you want me to throw at it, or any other debug info you want me to get. I'm out of my depth when it comes to debugging this kind of thing.
-
@deefdragon How about host Xen and XAPI versions? I'm on the bleeding edge, so this may have been fixed somewhere already.
# unbuffer yum list installed | grep xen\\\|xapi mellanox-mlnxen.x86_64 5.9_0.5.5.0-2.1.xcpng8.3 @xcp:main/$releasever python2-xapi-storage.x86_64 24.19.2-1.10.xcpng8.3 @xcp-ng-testing xapi-core.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-nbd.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-rrd2csv.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-storage-script.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-tests.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xapi-xe.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xcp-ng-xapi-plugins.noarch 1.12.0-1.xcpng8.3 @xcp-ng-testing xen-crashdump-analyser.x86_64 2.6.1-1.xcpng8.3 @xcp:main/$releasever xen-dom0-libs.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-dom0-tools.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-hypervisor.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-libs.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xen-tools.x86_64 4.17.5-15.2.xcpng8.3 @xcp-ng-incoming xengt-userspace.noarch 4.0.0-1.xcpng8.3 @xcp:main/$releasever xenopsd.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xenopsd-cli.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xenopsd-xc.x86_64 25.6.0-1.10.xcpng8.3 @xcp-ng-incoming xenserver-dracut.noarch 10-2.xcpng8.3 @xcp:main/$releasever xenserver-hwdata.noarch 20240411-1.xcpng8.3 @xcp:main/$releasever xenserver-status-report.noarch 2.0.11-1.xcpng8.3 @xcp-ng-ci -
Only difference looks to be xen-dom0 etc. on
4.17.5-13.1.xcpng8.3vs your4.17.5-15.2.xcpng8.3# unbuffer yum list installed | grep xen\\\|xapi mellanox-mlnxen.x86_64 5.9_0.5.5.0-2.1.xcpng8.3 @xcp:main/$releasever python2-xapi-storage.x86_64 24.19.2-1.10.xcpng8.3 @xcp-ng-updates xapi-core.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-nbd.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-rrd2csv.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-storage-script.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-tests.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xapi-xe.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xcp-ng-xapi-plugins.noarch 1.12.0-1.xcpng8.3 @xcp-ng-updates xen-crashdump-analyser.x86_64 2.6.1-1.xcpng8.3 @xcp:main/$releasever xen-dom0-libs.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-dom0-tools.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-hypervisor.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-libs.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xen-tools.x86_64 4.17.5-13.1.xcpng8.3 @xcp-ng-updates xengt-userspace.noarch 4.0.0-1.xcpng8.3 @xcp:main/$releasever xenopsd.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xenopsd-cli.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xenopsd-xc.x86_64 25.6.0-1.5.xcpng8.3 @xcp-ng-updates xenserver-dracut.noarch 10-2.xcpng8.3 @xcp:main/$releasever xenserver-hwdata.noarch 20240411-1.xcpng8.3 @xcp:main/$releasever xenserver-status-report.noarch 2.0.11-1.xcpng8.3 @xcp-ng-updates -
@deefdragon Your XAPI is also lagging behind (25.6.0-1.5 vs latest 25.6.0-1.9). Could you update your pool?
-
deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_serial 0b08f477-491a-a982-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/devices/virtual/dmi/id/product_uuid 77f4080b-1a49-82a9-23c4-d224723624ea deef@k31-w-3bfbbe:~$ sudo cat /sys/hypervisor/uuid 0b08f477-491a-a982-23c4-d224723624eaIt looks like a endianness issue (0b08f477 vs 77f4080b).
-
@deefdragon can you provide us the output of
dmidecodein the guest ? -
@dinhngtu Updated through xo. These are the current values.
mellanox-mlnxen.x86_64 5.9_0.5.5.0-2.1.xcpng8.3 @xcp:main/$releasever python2-xapi-storage.x86_64 24.19.2-1.10.xcpng8.3 @xcp-ng-updates xapi-core.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xapi-nbd.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xapi-rrd2csv.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xapi-storage-script.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xapi-tests.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xapi-xe.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xcp-ng-xapi-plugins.noarch 1.12.0-1.xcpng8.3 @xcp-ng-updates xen-crashdump-analyser.x86_64 2.6.1-1.xcpng8.3 @xcp:main/$releasever xen-dom0-libs.x86_64 4.17.5-15.1.xcpng8.3 @xcp-ng-updates xen-dom0-tools.x86_64 4.17.5-15.1.xcpng8.3 @xcp-ng-updates xen-hypervisor.x86_64 4.17.5-15.1.xcpng8.3 @xcp-ng-updates xen-libs.x86_64 4.17.5-15.1.xcpng8.3 @xcp-ng-updates xen-tools.x86_64 4.17.5-15.1.xcpng8.3 @xcp-ng-updates xengt-userspace.noarch 4.0.0-1.xcpng8.3 @xcp:main/$releasever xenopsd.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xenopsd-cli.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xenopsd-xc.x86_64 25.6.0-1.9.xcpng8.3 @xcp-ng-updates xenserver-dracut.noarch 10-2.xcpng8.3 @xcp:main/$releasever xenserver-hwdata.noarch 20240411-1.xcpng8.3 @xcp:main/$releasever xenserver-status-report.noarch 2.0.11-1.xcpng8.3 @xcp-ng-updatesGave one of the VMs a reboot and got the same result as previous.
Full DMI Decode as requested @TeddyAstie
# dmidecode # dmidecode 3.5 Getting SMBIOS data from sysfs. SMBIOS 2.8 present. 19 structures occupying 691 bytes. Table at 0xEE6FC000. Handle 0x0000, DMI type 0, 24 bytes BIOS Information Vendor: Xen Version: 4.17 Release Date: 07/10/2025 Address: 0xE8000 Runtime Size: 96 kB ROM Size: 64 kB Characteristics: PCI is supported EDD is supported Targeted content distribution is supported BIOS Revision: 4.17 Handle 0x0100, DMI type 1, 27 bytes System Information Manufacturer: Xen Product Name: HVM domU Version: 4.17 Serial Number: 0b08f477-491a-a982-23c4-d224723624ea UUID: 77f4080b-1a49-82a9-23c4-d224723624ea Wake-up Type: Power Switch SKU Number: Not Specified Family: Not Specified Handle 0x0300, DMI type 3, 21 bytes Chassis Information Manufacturer: Xen Type: Other Lock: Not Present Version: Not Specified Serial Number: Not Specified Asset Tag: Not Specified Boot-up State: Safe Power Supply State: Safe Thermal State: Safe Security Status: Unknown OEM Information: 0x00000000 Height: Unspecified Number Of Power Cords: Unspecified Contained Elements: 0 Handle 0x0401, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 1 Type: Central Processor Family: Other Manufacturer: Intel ID: F1 06 04 00 FF FB CB 0F Version: Not Specified Voltage: Unknown External Clock: Unknown Max Speed: 2100 MHz Current Speed: 2100 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: Not Provided L2 Cache Handle: Not Provided L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0402, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 2 Type: Central Processor Family: Other Manufacturer: Intel ID: F1 06 04 00 FF FB CB 0F Version: Not Specified Voltage: Unknown External Clock: Unknown Max Speed: 2100 MHz Current Speed: 2100 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: Not Provided L2 Cache Handle: Not Provided L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0403, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 3 Type: Central Processor Family: Other Manufacturer: Intel ID: F1 06 04 00 FF FB CB 0F Version: Not Specified Voltage: Unknown External Clock: Unknown Max Speed: 2100 MHz Current Speed: 2100 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: Not Provided L2 Cache Handle: Not Provided L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0404, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 4 Type: Central Processor Family: Other Manufacturer: Intel ID: F1 06 04 00 FF FB CB 0F Version: Not Specified Voltage: Unknown External Clock: Unknown Max Speed: 2100 MHz Current Speed: 2100 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: Not Provided L2 Cache Handle: Not Provided L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0405, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 5 Type: Central Processor Family: Other Manufacturer: Intel ID: F1 06 04 00 FF FB CB 0F Version: Not Specified Voltage: Unknown External Clock: Unknown Max Speed: 2100 MHz Current Speed: 2100 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: Not Provided L2 Cache Handle: Not Provided L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0406, DMI type 4, 35 bytes Processor Information Socket Designation: CPU 6 Type: Central Processor Family: Other Manufacturer: Intel ID: F1 06 04 00 FF FB CB 0F Version: Not Specified Voltage: Unknown External Clock: Unknown Max Speed: 2100 MHz Current Speed: 2100 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: Not Provided L2 Cache Handle: Not Provided L3 Cache Handle: Not Provided Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x0B00, DMI type 11, 5 bytes OEM Strings String 1: Xen String 2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d Handle 0x1000, DMI type 16, 15 bytes Physical Memory Array Location: Other Use: System Memory Error Correction Type: Multi-bit ECC Maximum Capacity: 24568 MB Error Information Handle: Not Provided Number Of Devices: 2 Handle 0x1100, DMI type 17, 27 bytes Memory Device Array Handle: 0x1000 Error Information Handle: 0x0000 Total Width: 64 bits Data Width: 64 bits Size: 16 GB Form Factor: DIMM Set: None Locator: DIMM 0 Bank Locator: Not Specified Type: RAM Type Detail: None Speed: Unknown Manufacturer: Not Specified Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x1300, DMI type 19, 15 bytes Memory Array Mapped Address Starting Address: 0x00000000000 Ending Address: 0x003FFFFFFFF Range Size: 16 GB Physical Array Handle: 0x1000 Partition Width: 1 Handle 0x1400, DMI type 20, 19 bytes Memory Device Mapped Address Starting Address: 0x00000000000 Ending Address: 0x003FFFFFFFF Range Size: 16 GB Physical Device Handle: 0x1100 Memory Array Mapped Address Handle: 0x1300 Partition Row Position: 1 Handle 0x1101, DMI type 17, 27 bytes Memory Device Array Handle: 0x1000 Error Information Handle: 0x0000 Total Width: 64 bits Data Width: 64 bits Size: 8184 MB Form Factor: DIMM Set: None Locator: DIMM 1 Bank Locator: Not Specified Type: RAM Type Detail: None Speed: Unknown Manufacturer: Not Specified Serial Number: Not Specified Asset Tag: Not Specified Part Number: Not Specified Handle 0x1301, DMI type 19, 15 bytes Memory Array Mapped Address Starting Address: 0x00400000000 Ending Address: 0x005FF7FFFFF Range Size: 8184 MB Physical Array Handle: 0x1000 Partition Width: 1 Handle 0x1401, DMI type 20, 19 bytes Memory Device Mapped Address Starting Address: 0x00400000000 Ending Address: 0x005FF7FFFFF Range Size: 8184 MB Physical Device Handle: 0x1101 Memory Array Mapped Address Handle: 0x1301 Partition Row Position: 1 Handle 0x2000, DMI type 32, 11 bytes System Boot Information Status: No errors detected Handle 0xFEFF, DMI type 127, 4 bytes End Of Table -
Out of curiosity, I dumped the DMI into a bin and opened it up in a hex editor.
I am seeing ASCII of the ID, but also a variant encoded in binary. In both cases, its formatted as
0b08f477-491a-a982-23c4-d224723624ea.I believe the ASCII version is the one that gets populated into the serial number as it comes after ASCII encoded versions of the 3 lines above it in the decode.
-
@deefdragon said in VM UUID via dmidecode does not match VM ID in xen-orchestra:
Out of curiosity, I dumped the DMI into a bin and opened it up in a hex editor.
I am seeing ASCII of the ID, but also a variant encoded in binary. In both cases, its formatted as
0b08f477-491a-a982-23c4-d224723624ea.I believe the ASCII version is the one that gets populated into the serial number as it comes after ASCII encoded versions of the 3 lines above it in the decode.
In SMBIOS 2.8, the UUID is supposed to be encoded in little endian (i.e Microsoft GUID). Yet it is put as big endian instead. So when Linux generates the UUID string from the SMBIOS table, it is considered as little endian which causes this mismatch.
SMBIOS 2.4 is supposed to be used (which appears to be using big endian UUIDs), but for some reason, something in XCP-ng UEFI supports forces it to be SMBIOS 2.8.
So the binary UUID is the same, just that it is interpreted with a different endianness due to accidental format change.
