Ok, was able to force shutdown via CLI and now I'm able to start VM on source server:
# xe vm-reset-powerstate vm=<vm name> --force
I'm not going to perform live migrations anytime soon again. I was trying to move VMs so we can do an upgrade which requires wiping out all storage because we used experimental ext4.
@julien-f Thank you! That appears to fix the error. Greatly appreciate your help and @olivierlambert.
@andrewperry I couldn't get the live migration to work on large VMs. To move large VMs, I ended up taking a snapshot of the VM and then creating a new VM from that snapshot. It's not ideal but had to do that as we needed to retire some old servers and I couldn't get the live migration to work w/ large VMs.
@Biggen You can do this via cli...see if this doc is helpful (just make sure to back/move/copy any data before performing these steps): https://support.citrix.com/article/CTX131328
@TS79 It seems something is not stable w/ SR NFS creation. The strange thing is XOCE allowed the creation of the NFS Remote under Settings > Remotes but refused to create the SR NFS until I patched to latest build (I was 3 commits behind) and then rebooted the entire server.
I was hoping to test this via just XO to rule out if it's an issue w/ XOCE or not but I couldn't get the XO deploy script to work. But I'll have to tackle that issue another day.
I was able to install a VM using the SR NFS and will be running som Disk IO tests to see how stable the NFS connection.
Thank you again for your help and testing this on your home lab! Greatly appreciate it!
Best Regards,
SW
Thank you @olivierlambert & @Danp . Greatly appreciate your assistance!
@olivierlambert Thank you, I've changed the settings on b2 to "Keep all versions of the file (default)".
I can confirm it's now working. Thank You! Health page loads fine w/ the latest commit.
When I login to xcp-ng cli, I see this when I run list_domains where the UUID of this VM looks different cli vs what xo shows:
@Danp Thank you! That's simple enough. I thought XO couldn't live migrate itself without running into an issue.
Thanks again!
SW
Hi,
Is it possible to live migrate XOCE via the XO web interface?
I have XOCE running on a single host which we use for nightly backups and DR of a 2 host pool. I want to perform a yum update on the host where XOCE vm is installed and don't want to lose access to XOCE during the reboot process. So I'm wondering if XOCE can be live migrated to the pool and then moved back to this single host after the reboot is complete.
Thank you,
SW
@StormMaster Thank you for sharing your findings!
@stephane-m-dev Thank you for the update! Looking forward to testing the fix just not sure how to replicate other than what @StormMaster found.
@andrewperry I couldn't get the live migration to work on large VMs. To move large VMs, I ended up taking a snapshot of the VM and then creating a new VM from that snapshot. It's not ideal but had to do that as we needed to retire some old servers and I couldn't get the live migration to work w/ large VMs.
@olivierlambert Thank you! Is it possible to simply attach the shared from the source pool to the new pool without having to do a live/warm migration?
We are only moving this VM from one pool to a new pool but the VM disks are going to remain on the shared storage (NFS):
The reason I'm moving pools is we have newer servers.
Thank you,
SW
Hi,
I'm trying to live migrate a large VM from one host to another and XO built from source using the latest commit a7d7c is reporting it will take over 32 hours for the migration to complete. However, after about 20+ hours, the live migration fails at about 48% complete.
XO shows the following error under Settings > Logs:
vm.migrate
{
"vm": "3797df5e-695a-00be-86bd-318a7c48860f",
"mapVifsNetworks": {
"de9a4140-0b67-314b-6fed-0bbd00d4cf74": "eefd492c-2bef-502e-79e7-f5123209d887",
"cbea6ad6-cb84-5536-e021-d2bdabda6348": "34f336c5-b05d-4258-20d0-984523113b85"
},
"migrationNetwork": "02bfbb0b-5f1a-e47e-d50b-28f0f7c50b11",
"sr": "8da3d03e-4d2c-bab2-cd94-0d15168a58f3",
"targetHost": "2d926060-41bf-4e17-ba76-bac9b1112257"
}
{
"code": "MIRROR_FAILED",
"params": [
"OpaqueRef:7069c225-0bf2-433c-a255-0fab035ea70b"
],
"task": {
"uuid": "f31eafed-faea-2eb4-b9e9-d45cc7967498",
"name_label": "Async.VM.migrate_send",
"name_description": "",
"allowed_operations": [],
"current_operations": {
"DummyRef:|e8163dcb-8c54-4907-a0de-97d36cf2127e|task.cancel": "cancel"
},
"created": "20241014T04:40:24Z",
"finished": "20241015T00:09:27Z",
"status": "failure",
"resident_on": "OpaqueRef:d3f118d9-3aef-4d16-94d5-6d6fa22f84b9",
"progress": 1,
"type": "<none/>",
"result": "",
"error_info": [
"MIRROR_FAILED",
"OpaqueRef:7069c225-0bf2-433c-a255-0fab035ea70b"
],
"other_config": {
"mirror_failed": "b607eebc-49ad-4fc0-ad4a-b605fedfc51e"
},
"subtask_of": "OpaqueRef:NULL",
"subtasks": [],
"backtrace": "(((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 1556))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 131))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/xapi/rbac.ml)(line 205))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 95)))"
},
"message": "MIRROR_FAILED(OpaqueRef:7069c225-0bf2-433c-a255-0fab035ea70b)",
"name": "XapiError",
"stack": "XapiError: MIRROR_FAILED(OpaqueRef:7069c225-0bf2-433c-a255-0fab035ea70b)
at Function.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
at default (file:///opt/xen-orchestra/packages/xen-api/_getTaskResult.mjs:13:29)
at Xapi._addRecordToCache (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1041:24)
at file:///opt/xen-orchestra/packages/xen-api/index.mjs:1075:14
at Array.forEach (<anonymous>)
at Xapi._processEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1065:12)
at Xapi._watchEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1238:14)"
}
On the source host, /var/log/xensource.log show the following when the error occurs:
And on the target host, /var/log/xensource.log show the following:
I appreciate any help you can offer to help me identify why the live migration is failing.
Thank you,
SW
@enes-selcuk Did you find a setting that works best w/ Dell servers?
I have a Dell R640's which I'll be using for LAMP/LEMP vm servers and was wondering what are the best settings to use in Dell Bios and if there are any changes I need to make on the xcp-ng host?
Dual Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Level 2 Cache 16x1 MB
Level 3 Cache 22 MB
Number of Cores 16
Dell Bios Setting:
System Profile: Performance Per Watt (OS)
CPU Power Management: OS DBPM
Memory Frequency: Maximum Performance
Turbo Boost: Enabled
C1E: Enabled
C States: Enabled
Write Data CRC: Disabled
Memory Patrol Scrub: Standard
Memory Refresh Rate: 1x
Uncore Frequency : Dynamic
Energy Efficient Policy: Balanced Performance
Number of Turbo Boost Enabled Cores for Processor 1: All
Number of Turbo Boost Enabled Cores for Processor 2: All
Monitor/Mwait: Enabled
Workload Profile: Not Available
CPU Interconnect Bus Link Power Management: Enabled
PCI ASPM L1 Link Power Management: Enabled
Thank you!
SW
@TS79 It seems something is not stable w/ SR NFS creation. The strange thing is XOCE allowed the creation of the NFS Remote under Settings > Remotes but refused to create the SR NFS until I patched to latest build (I was 3 commits behind) and then rebooted the entire server.
I was hoping to test this via just XO to rule out if it's an issue w/ XOCE or not but I couldn't get the XO deploy script to work. But I'll have to tackle that issue another day.
I was able to install a VM using the SR NFS and will be running som Disk IO tests to see how stable the NFS connection.
Thank you again for your help and testing this on your home lab! Greatly appreciate it!
Best Regards,
SW
@olivierlambert @TS79 I just rebooted the server, ran the XOCE update script, and tried adding the SR NFS to the new host and I was able to create the SR NFS! 
I'm about to test it by installing a VM to confirm it's working properly!