Ok, was able to force shutdown via CLI and now I'm able to start VM on source server:
# xe vm-reset-powerstate vm=<vm name> --force
I'm not going to perform live migrations anytime soon again. I was trying to move VMs so we can do an upgrade which requires wiping out all storage because we used experimental ext4.
@julien-f Thank you! That appears to fix the error. Greatly appreciate your help and @olivierlambert.
@andrewperry I couldn't get the live migration to work on large VMs. To move large VMs, I ended up taking a snapshot of the VM and then creating a new VM from that snapshot. It's not ideal but had to do that as we needed to retire some old servers and I couldn't get the live migration to work w/ large VMs.
@Biggen You can do this via cli...see if this doc is helpful (just make sure to back/move/copy any data before performing these steps): https://support.citrix.com/article/CTX131328
@Andrew Thank you that sounds perfect!
@TS79 It seems something is not stable w/ SR NFS creation. The strange thing is XOCE allowed the creation of the NFS Remote under Settings > Remotes but refused to create the SR NFS until I patched to latest build (I was 3 commits behind) and then rebooted the entire server.
I was hoping to test this via just XO to rule out if it's an issue w/ XOCE or not but I couldn't get the XO deploy script to work. But I'll have to tackle that issue another day.
I was able to install a VM using the SR NFS and will be running som Disk IO tests to see how stable the NFS connection.
Thank you again for your help and testing this on your home lab! Greatly appreciate it!
Best Regards,
SW
Thank you @olivierlambert & @Danp . Greatly appreciate your assistance!
@olivierlambert Thank you, I've changed the settings on b2 to "Keep all versions of the file (default)".
I can confirm it's now working. Thank You! Health page loads fine w/ the latest commit.
When I login to xcp-ng cli, I see this when I run list_domains where the UUID of this VM looks different cli vs what xo shows:
@olivierlambert Thanks! Just checked browser console in Chrome and Firefox and both show the same 2 js errors:
@olivierlambert Thanks for your help! It does appear the IPMI plugin works via CLI but still doesn't show up in XO web interface:
[14:00 XCP45 ~]# xe host-call-plugin host-uuid=63e07928-10a4-4e17-911d-4b9fe6b4804d plugin=ipmitool.py fn=is_ipmi_device_available
true
[14:00 XCP45 ~]# xe host-call-plugin host-uuid=63e07928-10a4-4e17-911d-4b9fe6b4804d plugin=ipmitool.py fn=get_all_sensors
[{"name": "Temp", "value": "47 degrees C", "event": "ok"}, {"name": "Temp", "value": "49 degrees C", "event": "ok"}, {"name": "Inlet Temp", "value": "22 degrees C", "event": "ok"}, {"name": "DIMM PG", "value": "0x00", "event": "ok"}, {"name": "NDC PG", "value": "0x00", "event": "ok"}, {"name": "PS1 PG FAIL", "value": "0x00", "event": "ok"}, {"name": "PS2 PG FAIL", "value": "0x00", "event": "ok"}, {"name": "BP0 PG", "value": "0x00", "event": "ok"}, {"name": "BP1 PG", "value": "0x00", "event": "ok"}, {"name": "1.8V SW PG", "value": "0x00", "event": "ok"}, {"name": "2.5V SW PG", "value": "0x00", "event": "ok"}, {"name": "5V SW PG", "value": "0x00", "event": "ok"}, {"name": "PVNN SW PG", "value": "0x00", "event": "ok"}, {"name": "VSB11 SW PG", "value": "0x00", "event": "ok"}, {"name": "VSBM SW PG", "value": "0x00", "event": "ok"}, {"name": "3.3V B PG", "value": "0x00", "event": "ok"}, {"name": "MEM012 VDDQ PG", "value": "0x00", "event": "ok"}, {"name": "MEM012 VPP PG", "value": "0x00", "event": "ok"}, {"name": "MEM012 VTT PG", "value": "0x00", "event": "ok"}, {"name": "MEM345 VDDQ PG", "value": "0x00", "event": "ok"}, {"name": "MEM345 VPP PG", "value": "0x00", "event": "ok"}, {"name": "MEM345 VTT PG", "value": "0x00", "event": "ok"}, {"name": "VCCIO PG", "value": "0x00", "event": "ok"}, {"name": "VCORE PG", "value": "0x00", "event": "ok"}, {"name": "FIVR PG", "value": "0x00", "event": "ok"}, {"name": "MEM012 VDDQ PG", "value": "0x00", "event": "ok"}, {"name": "MEM012 VPP PG", "value": "0x00", "event": "ok"}, {"name": "MEM012 VTT PG", "value": "0x00", "event": "ok"}, {"name": "MEM345 VDDQ PG", "value": "0x00", "event": "ok"}, {"name": "MEM345 VPP PG", "value": "0x00", "event": "ok"}, {"name": "MEM345 VTT PG", "value": "0x00", "event": "ok"}, {"name": "VCCIO PG", "value": "0x00", "event": "ok"}, {"name": "VCORE PG", "value": "0x00", "event": "ok"}, {"name": "FIVR PG", "value": "0x00", "event": "ok"}, {"name": "Fan1A", "value": "4440 RPM", "event": "ok"}, {"name": "Fan1B", "value": "4080 RPM", "event": "ok"}, {"name": "Fan2A", "value": "4320 RPM", "event": "ok"}, {"name": "Fan2B", "value": "3840 RPM", "event": "ok"}, {"name": "Fan3A", "value": "4200 RPM", "event": "ok"}, {"name": "Fan3B", "value": "3840 RPM", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "Intrusion Cable", "value": "0x00", "event": "ok"}, {"name": "VGA Cable Pres", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "BP0 Presence", "value": "0x00", "event": "ok"}, {"name": "BP1 Presence", "value": "0x00", "event": "ok"}, {"name": "Power Cable", "value": "0x00", "event": "ok"}, {"name": "Signal Cable", "value": "0x00", "event": "ok"}, {"name": "Power JBP1", "value": "0x00", "event": "ok"}, {"name": "Signal Cable", "value": "0x00", "event": "ok"}, {"name": "Power JBP2", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "Current 1", "value": "1.60 Amps", "event": "ok"}, {"name": "Current 2", "value": "0 Amps", "event": "ok"}, {"name": "Voltage 1", "value": "116 Volts", "event": "ok"}, {"name": "Voltage 2", "value": "120 Volts", "event": "ok"}, {"name": "Riser Config Err", "value": "0x00", "event": "ok"}, {"name": "OS Watchdog", "value": "0x00", "event": "ok"}, {"name": "SEL", "value": "Not Readable", "event": "ns"}, {"name": "Intrusion", "value": "0x00", "event": "ok"}, {"name": "Power Optimized", "value": "0x00", "event": "ok"}, {"name": "Pwr Consumption", "value": "168 Watts", "event": "ok"}, {"name": "PS Redundancy", "value": "0x00", "event": "ok"}, {"name": "Fan Redundancy", "value": "0x00", "event": "ok"}, {"name": "Redundancy", "value": "Not Readable", "event": "ns"}, {"name": "SD1", "value": "Not Readable", "event": "ns"}, {"name": "SD2", "value": "Not Readable", "event": "ns"}, {"name": "SD", "value": "Not Readable", "event": "ns"}, {"name": "IO Usage", "value": "0 percent", "event": "ok"}, {"name": "MEM Usage", "value": "0 percent", "event": "ok"}, {"name": "SYS Usage", "value": "0 percent", "event": "ok"}, {"name": "CPU Usage", "value": "0 percent", "event": "ok"}, {"name": "Status", "value": "0x00", "event": "ok"}, {"name": "Status", "value": "0x00", "event": "ok"}, {"name": "Status", "value": "0x00", "event": "ok"}, {"name": "Status", "value": "0x00", "event": "ok"}, {"name": "ROMB Battery", "value": "0x00", "event": "ok"}, {"name": "PCIe Slot1", "value": "0x00", "event": "ok"}, {"name": "PCIe Slot2", "value": "Not Readable", "event": "ns"}, {"name": "PCIe Slot3", "value": "Not Readable", "event": "ns"}, {"name": "Drive 0", "value": "0x00", "event": "ok"}, {"name": "Cable PCIe A0", "value": "0x00", "event": "ok"}, {"name": "Cable PCIe B0", "value": "0x00", "event": "ok"}, {"name": "Cable PCIe A1", "value": "0x00", "event": "ok"}, {"name": "Cable PCIe B1", "value": "0x00", "event": "ok"}, {"name": "Cable PCIe A2", "value": "0x00", "event": "ok"}, {"name": "Cable PCIe B2", "value": "Not Readable", "event": "ns"}, {"name": "Cable SAS A0", "value": "0x00", "event": "ok"}, {"name": "Cable SAS B0", "value": "0x00", "event": "ok"}, {"name": "Cable SAS A1", "value": "Not Readable", "event": "ns"}, {"name": "Cable SAS B1", "value": "Not Readable", "event": "ns"}, {"name": "Cable SAS A2", "value": "0x00", "event": "ok"}, {"name": "Cable SAS B2", "value": "Not Readable", "event": "ns"}, {"name": "Cable PCIe A0", "value": "Not Readable", "event": "ns"}, {"name": "Cable PCIe B0", "value": "Not Readable", "event": "ns"}, {"name": "ECC Corr Err", "value": "0xb0", "event": "ok"}, {"name": "ECC Uncorr Err", "value": "0xb2", "event": "ok"}, {"name": "PCI Parity Err", "value": "Not Readable", "event": "ns"}, {"name": "PCI System Err", "value": "0x96", "event": "ok"}, {"name": "SBE Log Disabled", "value": "0xa7", "event": "ok"}, {"name": "Unknown", "value": "0x00", "event": "ok"}, {"name": "CPU Machine Chk", "value": "0x00", "event": "ok"}, {"name": "Memory Spared", "value": "0x00", "event": "ok"}, {"name": "Memory Mirrored", "value": "0x00", "event": "ok"}, {"name": "PCIE Fatal Err", "value": "0xb0", "event": "ok"}, {"name": "Chipset Err", "value": "Not Readable", "event": "ns"}, {"name": "Err Reg Pointer", "value": "0x79", "event": "ok"}, {"name": "Mem ECC Warning", "value": "0x79", "event": "ok"}, {"name": "POST Err", "value": "Not Readable", "event": "ns"}, {"name": "Hdwr version err", "value": "Not Readable", "event": "ns"}, {"name": "Non Fatal PCI Er", "value": "0x00", "event": "ok"}, {"name": "Fatal IO Error", "value": "0x00", "event": "ok"}, {"name": "MSR Info Log", "value": "0x00", "event": "ok"}, {"name": "TXT Status", "value": "0x00", "event": "ok"}, {"name": "iDPT Mem Fail", "value": "0x00", "event": "ok"}, {"name": "Additional Info", "value": "0x00", "event": "ok"}, {"name": "CPU TDP", "value": "0x00", "event": "ok"}, {"name": "QPIRC Warning", "value": "0x00", "event": "ok"}, {"name": "QPIRC Warning", "value": "0x00", "event": "ok"}, {"name": "Link Warning", "value": "0x00", "event": "ok"}, {"name": "Link Warning", "value": "0x00", "event": "ok"}, {"name": "Link Error", "value": "0x00", "event": "ok"}, {"name": "MRC Warning", "value": "0x00", "event": "ok"}, {"name": "MRC Warning", "value": "0x00", "event": "ok"}, {"name": "Chassis Mismatch", "value": "0x00", "event": "ok"}, {"name": "FatalPCIErrOnBus", "value": "0x25", "event": "ok"}, {"name": "NonFatalPCIErBus", "value": "0x24", "event": "ok"}, {"name": "Fatal PCI SSD Er", "value": "0x23", "event": "ok"}, {"name": "NonFatalSSDEr", "value": "0x24", "event": "ok"}, {"name": "CPUMachineCheck", "value": "0x22", "event": "ok"}, {"name": "FatalPCIErARI", "value": "0x24", "event": "ok"}, {"name": "NonFatalPCIErARI", "value": "0x25", "event": "ok"}, {"name": "FatalPCIExpEr", "value": "0x24", "event": "ok"}, {"name": "NonFatalPCIExpEr", "value": "0x22", "event": "ok"}, {"name": "Cable SAS A0", "value": "Not Readable", "event": "ns"}, {"name": "Cable SAS B0", "value": "Not Readable", "event": "ns"}, {"name": "Fan4A", "value": "4320 RPM", "event": "ok"}, {"name": "Fan4B", "value": "4080 RPM", "event": "ok"}, {"name": "Fan5A", "value": "4080 RPM", "event": "ok"}, {"name": "Fan5B", "value": "3840 RPM", "event": "ok"}, {"name": "Fan6A", "value": "4320 RPM", "event": "ok"}, {"name": "Fan6B", "value": "3840 RPM", "event": "ok"}, {"name": "Fan7A", "value": "4440 RPM", "event": "ok"}, {"name": "Fan7B", "value": "3960 RPM", "event": "ok"}, {"name": "Fan8A", "value": "4320 RPM", "event": "ok"}, {"name": "Fan8B", "value": "3960 RPM", "event": "ok"}, {"name": "Unresp sensor", "value": "0x00", "event": "ok"}, {"name": "CP Left Pres", "value": "0x00", "event": "ok"}, {"name": "CP Right Pres", "value": "0x00", "event": "ok"}, {"name": "3.3V A PG", "value": "0x00", "event": "ok"}, {"name": "VSA PG", "value": "0x00", "event": "ok"}, {"name": "VSA PG", "value": "0x00", "event": "ok"}, {"name": "TPM Presence", "value": "0x00", "event": "ok"}, {"name": "Riser 1 Presence", "value": "0x00", "event": "ok"}, {"name": "Riser 2 Presence", "value": "0x00", "event": "ok"}, {"name": "Front LED Panel", "value": "0x00", "event": "ok"}, {"name": "OS Watchdog Time", "value": "0x00", "event": "ok"}, {"name": "Fan1A Status", "value": "0x00", "event": "ok"}, {"name": "Fan1B Status", "value": "0x00", "event": "ok"}, {"name": "Fan2A Status", "value": "0x00", "event": "ok"}, {"name": "Fan2B Status", "value": "0x00", "event": "ok"}, {"name": "Fan3A Status", "value": "0x00", "event": "ok"}, {"name": "Fan3B Status", "value": "0x00", "event": "ok"}, {"name": "Fan4A Status", "value": "0x00", "event": "ok"}, {"name": "Fan4B Status", "value": "0x00", "event": "ok"}, {"name": "Fan5A Status", "value": "0x00", "event": "ok"}, {"name": "Fan5B Status", "value": "0x00", "event": "ok"}, {"name": "Fan6A Status", "value": "0x00", "event": "ok"}, {"name": "Fan6B Status", "value": "0x00", "event": "ok"}, {"name": "Fan7A Status", "value": "0x00", "event": "ok"}, {"name": "Fan7B Status", "value": "0x00", "event": "ok"}, {"name": "Fan8A Status", "value": "0x00", "event": "ok"}, {"name": "Fan8B Status", "value": "0x00", "event": "ok"}, {"name": "NVDIMM Warning", "value": "0x21", "event": "ok"}, {"name": "NVDIMM Error", "value": "0x21", "event": "ok"}, {"name": "NVDIMM Info", "value": "0x00", "event": "ok"}, {"name": "Dedicated NIC", "value": "0x00", "event": "ok"}, {"name": "Presence", "value": "0x00", "event": "ok"}, {"name": "NVDIMM Battery", "value": "Not Readable", "event": "ns"}, {"name": "Exhaust Temp", "value": "39 degrees C", "event": "ok"}, {"name": "LT/Flex Addr", "value": "0x00", "event": "ok"}, {"name": "QPI Link Err", "value": "0x00", "event": "ok"}, {"name": "TPM Presence", "value": "0x20", "event": "ok"}, {"name": "CPU Link Info", "value": "0x20", "event": "ok"}, {"name": "Chipset Info", "value": "0x20", "event": "ok"}, {"name": "Memory Config", "value": "0x1f", "event": "ok"}, {"name": "POST Pkg Repair", "value": "0x20", "event": "ok"}, {"name": "Pfault Fail Safe", "value": "Not Readable", "event": "ns"}, {"name": "BP2 PG", "value": "0x00", "event": "ok"}, {"name": "MMIOChipset Info", "value": "0x00", "event": "ok"}, {"name": "DIMM Media Info", "value": "0x00", "event": "ok"}, {"name": "DIMMThermal Info", "value": "0x00", "event": "ok"}, {"name": "CPU Internal Err", "value": "0x00", "event": "ok"}, {"name": "GPU1 Temp", "value": "disabled", "event": "ns"}, {"name": "GPU2 Temp", "value": "disabled", "event": "ns"}, {"name": "GPU3 Temp", "value": "disabled", "event": "ns"}, {"name": "A1", "value": "0x00", "event": "ok"}, {"name": "A2", "value": "0x00", "event": "ok"}, {"name": "A3", "value": "0x00", "event": "ok"}, {"name": "A4", "value": "0x00", "event": "ok"}, {"name": "A5", "value": "0x00", "event": "ok"}, {"name": "A6", "value": "0x00", "event": "ok"}, {"name": "A7", "value": "0x00", "event": "ok"}, {"name": "A8", "value": "0x00", "event": "ok"}, {"name": "A9", "value": "0x00", "event": "ok"}, {"name": "A10", "value": "0x00", "event": "ok"}, {"name": "A11", "value": "0x00", "event": "ok"}, {"name": "A12", "value": "0x00", "event": "ok"}, {"name": "B1", "value": "0x00", "event": "ok"}, {"name": "B2", "value": "0x00", "event": "ok"}, {"name": "B3", "value": "0x00", "event": "ok"}, {"name": "B4", "value": "0x00", "event": "ok"}, {"name": "B5", "value": "0x00", "event": "ok"}, {"name": "B6", "value": "0x00", "event": "ok"}, {"name": "B7", "value": "0x00", "event": "ok"}, {"name": "B8", "value": "0x00", "event": "ok"}, {"name": "B9", "value": "0x00", "event": "ok"}, {"name": "B10", "value": "0x00", "event": "ok"}, {"name": "B11", "value": "0x00", "event": "ok"}, {"name": "B12", "value": "0x00", "event": "ok"}, {"name": "Therm Config Err", "value": "0x00", "event": "ok"}, {"name": "VCORE VR", "value": "1.78 Volts", "event": "ok"}, {"name": "VCORE VR", "value": "1.76 Volts", "event": "ok"}, {"name": "MEM012 VR", "value": "1.21 Volts", "event": "ok"}, {"name": "MEM345 VR", "value": "1.21 Volts", "event": "ok"}, {"name": "MEM012 VR", "value": "1.21 Volts", "event": "ok"}, {"name": "MEM345 VR", "value": "1.21 Volts", "event": "ok"}]
[14:01 XCP45 ~]#
[14:01 XCP45 ~]# xe host-call-plugin host-uuid=63e07928-10a4-4e17-911d-4b9fe6b4804d plugin=ipmitool.py fn=get_sensor args:sensors="Fan7A,PFault Fail Safe"
[{"info": [{"name": "Sensor ID", "value": "Fan7A (0x3e)"}, {"name": "Entity ID", "value": "7.1 (System Board)"}, {"name": "Sensor Type (Threshold)", "value": "Fan (0x04)"}, {"name": "Sensor Reading", "value": "4440 (+/- 120) RPM"}, {"name": "Status", "value": "ok"}, {"name": "Nominal Reading", "value": "10080.000"}, {"name": "Normal Minimum", "value": "16680.000"}, {"name": "Normal Maximum", "value": "23640.000"}, {"name": "Lower critical", "value": "600.000"}, {"name": "Lower non-critical", "value": "840.000"}, {"name": "Positive Hysteresis", "value": "120.000"}, {"name": "Negative Hysteresis", "value": "120.000"}, {"name": "Minimum sensor range", "value": "Unspecified"}, {"name": "Maximum sensor range", "value": "Unspecified"}, {"name": "Event Message Control", "value": "Per-threshold"}, {"name": "Readable Thresholds", "value": "lcr lnc"}, {"name": "Settable Thresholds", "value": ""}, {"name": "Threshold Read Mask", "value": "lcr lnc"}, {"name": "Assertion Events", "value": ""}, {"name": "Assertions Enabled", "value": "lnc- lcr-"}, {"name": "Deassertions Enabled", "value": "lnc- lcr-"}], "name": "Fan7A"}]
[14:02 XCP45 ~]#
[14:03 XCP45 ~]# xe host-call-plugin host-uuid=63e07928-10a4-4e17-911d-4b9fe6b4804d plugin=ipmitool.py fn=get_ipmi_lan
[{"name": "IP Address Source", "value": "Static Address"}, {"name": "IP Address", "value": "10.10.10.36"}, {"name": "Subnet Mask", "value": "255.255.255.0"}, {"name": "MAC Address", "value": "f4:02:70:ef:46:e6"}, {"name": "BMC ARP Control", "value": "ARP Responses Enabled, Gratuitous ARP Disabled"}, {"name": "Default Gateway IP", "value": "10.10.10.1"}, {"name": "802.1q VLAN ID", "value": "Disabled"}, {"name": "802.1q VLAN Priority", "value": "0"}, {"name": "RMCP+ Cipher Suites", "value": "3,17"}]
[14:03 XCP45 ~]#
@acebmxer Thanks for the screenshot! My server is a Dell R640 and I check iDRAC Settings > Connectivity to confirm IPMI is enabled:
Not sure if I need to do anything else on XO or on the Dell server?
@olivierlambert I must be blind as I tried looking under the various tabs under the Host in XO but can't see it:
Thanks,
SW
Hi,
Just updated to xcp-ng 8.3 and I tried searching the XO docs for IPMI but can't find anything. How do I see IPMI info from our Dell PE 640 in XO?
Thank you,
SW
What was the fix for this as one of our Windows 10 VMs backups have started to fail and rebooting the VM only takes less than 1 minute so not sure increasing the timeout is the issue. Here is the backup log entries:
{
"data": {
"type": "VM",
"id": "07767a82-3cc0-b2b7-e399-90793896a574",
"name_label": "Windows10C"
},
"id": "1760322641022",
"message": "backup VM",
"start": 1760322641022,
"status": "failure",
"tasks": [
{
"id": "1760322641048",
"message": "clean-vm",
"start": 1760322641048,
"status": "success",
"tasks": [
{
"id": "1760322641516",
"message": "merge",
"start": 1760322641516,
"status": "success",
"end": 1760323012896
}
],
"warnings": [
{
"data": {
"path": "/xo-vm-backups/07767a82-3cc0-b2b7-e399-90793896a574/20251012T023042Z.json",
"actual": 5475663872,
"expected": 5477068288
},
"message": "cleanVm: incorrect backup size in metadata"
}
],
"end": 1760323013334,
"result": {
"merge": true
}
},
{
"id": "1760323014314",
"message": "snapshot",
"start": 1760323014314,
"status": "success",
"end": 1760323016106,
"result": "0b79d24f-a579-b669-50e1-412516276886"
},
{
"data": {
"id": "2a4bc998-972d-4132-83ba-f3c67b860676",
"isFull": false,
"type": "remote"
},
"id": "1760323016106:0",
"message": "export",
"start": 1760323016106,
"status": "failure",
"tasks": [
{
"id": "1760323018513",
"message": "transfer",
"start": 1760323018513,
"status": "success",
"end": 1760323462425,
"result": {
"size": 7688159232
}
},
{
"id": "1760323472984",
"message": "health check",
"start": 1760323472984,
"status": "failure",
"tasks": [
{
"id": "1760323480406",
"message": "transfer",
"start": 1760323480406,
"status": "success",
"end": 1760324685177,
"result": {
"size": 0,
"id": "490d3623-4b1f-e4a4-4947-8062aa0e757c"
}
},
{
"id": "1760324685178",
"message": "vmstart",
"start": 1760324685178,
"status": "failure",
"end": 1760325285289,
"result": {
"message": "timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
}
],
"end": 1760325289511,
"result": {
"message": "timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
}
],
"end": 1760325289511,
"result": {
"message": "timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
},
{
"data": {
"id": "349e7130-da43-6550-e7c0-aa0a84102e0f",
"isFull": false,
"name_label": "Local storage",
"type": "SR"
},
"id": "1760323016107",
"message": "export",
"start": 1760323016107,
"status": "failure",
"tasks": [
{
"id": "1760323017919",
"message": "transfer",
"start": 1760323017919,
"status": "success",
"end": 1760323463982,
"result": {
"size": 7688159232
}
},
{
"id": "1760323472984:0",
"message": "health check",
"start": 1760323472984,
"status": "failure",
"tasks": [
{
"id": "1760323473002",
"message": "copying-vm",
"start": 1760323473002,
"status": "success",
"end": 1760325038087,
"result": "OpaqueRef:0233c593-da90-4813-959b-46e4e679a662"
},
{
"id": "1760325038108",
"message": "vmstart",
"start": 1760325038108,
"status": "failure",
"end": 1760325638322,
"result": {
"message": "timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
}
],
"end": 1760325643426,
"result": {
"message": "timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
}
],
"end": 1760325643426,
"result": {
"message": "timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
}
],
"end": 1760325644170,
"result": {
"errors": [
{
"message": "timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:5c4eef85-eebb-4480-826b-7bbe596f4349 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
},
{
"message": "timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.",
"name": "Error",
"stack": "Error: timeout reached while waiting for OpaqueRef:ca1c86dd-aafc-4f25-bd8c-89399771c004 to report the driver version through the Xen tools. Please check or update the Xen tools.\n at file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:275:23\n at new Promise (<anonymous>)\n at Xapi.waitObjectState (file:///opt/xen-orchestra/@xen-orchestra/xapi/index.mjs:259:12)\n at file:///opt/xen-orchestra/@xen-orchestra/backups/HealthCheckVmBackup.mjs:63:20\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"
}
],
"message": "all targets have failed, step: writer.healthCheck()",
"name": "Error",
"stack": "Error: all targets have failed, step: writer.healthCheck()\n at IncrementalXapiVmBackupRunner._callWriters (file:///opt/xen-orchestra/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:64:13)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n at async IncrementalXapiVmBackupRunner._healthCheck (file:///opt/xen-orchestra/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:94:5)\n at async IncrementalXapiVmBackupRunner.run (file:///opt/xen-orchestra/@xen-orchestra/backups/_runners/_vmRunners/_AbstractXapi.mjs:407:5)\n at async file:///opt/xen-orchestra/@xen-orchestra/backups/_runners/VmsXapi.mjs:166:38"
}
},
@Danp Thank you! That's simple enough. I thought XO couldn't live migrate itself without running into an issue.
Thanks again!
SW
Hi,
Is it possible to live migrate XOCE via the XO web interface?
I have XOCE running on a single host which we use for nightly backups and DR of a 2 host pool. I want to perform a yum update on the host where XOCE vm is installed and don't want to lose access to XOCE during the reboot process. So I'm wondering if XOCE can be live migrated to the pool and then moved back to this single host after the reboot is complete.
Thank you,
SW
@StormMaster Thank you for sharing your findings!
@stephane-m-dev Thank you for the update! Looking forward to testing the fix just not sure how to replicate other than what @StormMaster found.
@andrewperry I couldn't get the live migration to work on large VMs. To move large VMs, I ended up taking a snapshot of the VM and then creating a new VM from that snapshot. It's not ideal but had to do that as we needed to retire some old servers and I couldn't get the live migration to work w/ large VMs.