<![CDATA[VM failed and won't start. Displaying "An emulator required to run this VM failed to start"]]>AlmaLinux 8.10. VM failed and won't start. Displaying "An emulator required to run this VM failed to start".
I have restarted the toolstack on the Host Server. I have attempted to create snapshot and revert. Any help is appreciated.

I just tried to migrate a test VM to the same host. No getting.

"Failed","Migrating VM 'AlmaLinux8-Testing' from 'iahost-xcpng-server3' to 'iahost-xcpng-server1'
Internal error: xenopsd internal error: Xenops_interface.Xenopsd_error([S(Storage_backend_error);[S(SR_BACKEND_FAILURE_46);[S();S(The VDI is not available [opterr=Command ['/sbin/lvs', '--noheadings', '--units', 'b', '-o', '+lv_tags', '/dev/VG_XenStorage-88d7607c-f807-3b06-6f70-2dcb319d97ea'] failed (5): /dev/sdb: open failed: No such device or address
  Volume group "VG_XenStorage-88d7607c-f807-3b06-6f70-2dcb319d97ea" not found
  Cannot process volume group VG_XenStorage-88d7607c-f807-3b06-6f70-2dcb319d97ea]);S()]]])
Time: 00:09:43","iahost-xcpng-server1","Jun 9, 2025 10:41 AM"

And now attempting to force a reboot on same vm

"Failed","Rebooting VM 'AlmaLinux8-Testing' on 'iahost-xcpng-server3'
Internal error: Object with type VM and id dfcdcce6-b6e1-d865-6f2b-472a35b1fae7/config does not exist in xenopsd
Time: 00:00:01","iahost-xcpng-server3","Jun 9, 2025 11:03 AM"
]]>https://xcp-ng.org/forum/topic/10926/vm-failed-and-won-t-start.-displaying-an-emulator-required-to-run-this-vm-failed-to-startRSS for NodeWed, 13 May 2026 09:36:05 GMTMon, 09 Jun 2025 15:38:28 GMT60<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 17:00:10 GMT]]>Then try to find anything happening around that time on other hosts, equipment, storage and so on.

]]>https://xcp-ng.org/forum/post/94586https://xcp-ng.org/forum/post/94586Mon, 30 Jun 2025 17:00:10 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 16:55:59 GMT]]>dmesg -T is only showing these from 00:37 on.

[Mon Jun 30 05:55:10 2025] print_req_error: I/O error, dev sdb, sector 0
[Mon Jun 30 05:55:10 2025] sd 7:0:0:1: rejecting I/O to offline device

]]>https://xcp-ng.org/forum/post/94585https://xcp-ng.org/forum/post/94585Mon, 30 Jun 2025 16:55:59 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 16:51:51 GMT]]>Well dmesg is fine, I would also check the switch logs if you have anything happening on the ports where you host are connected. Since you can find when it's happening with dmesg, I would use that to correlate with anything else happening few minutes before.

]]>https://xcp-ng.org/forum/post/94583https://xcp-ng.org/forum/post/94583Mon, 30 Jun 2025 16:51:51 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 16:48:51 GMT]]>What log is best to look at for network disconnects?

]]>https://xcp-ng.org/forum/post/94581https://xcp-ng.org/forum/post/94581Mon, 30 Jun 2025 16:48:51 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 14:58:37 GMT]]>Well, it doesn't happen to every host at the same time, it's just random.

]]>https://xcp-ng.org/forum/post/94571https://xcp-ng.org/forum/post/94571Mon, 30 Jun 2025 14:58:37 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 14:54:35 GMT]]>At least we know there's a root cause: the device becomes offline. Could it be a power setting or something on the device itself?

]]>https://xcp-ng.org/forum/post/94570https://xcp-ng.org/forum/post/94570Mon, 30 Jun 2025 14:54:35 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 14:51:09 GMT]]>Yes

]]>https://xcp-ng.org/forum/post/94569https://xcp-ng.org/forum/post/94569Mon, 30 Jun 2025 14:51:09 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 14:25:34 GMT]]>That's not a trivial problem that you have, it seems to be intermittent and could be anything between host and the storage. I assume you can see the issue as soon you have the rejecting I/O to offline device right?

]]>https://xcp-ng.org/forum/post/94566https://xcp-ng.org/forum/post/94566Mon, 30 Jun 2025 14:25:34 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 13:35:11 GMT]]>Any help is greatly appreciated.

]]>https://xcp-ng.org/forum/post/94556https://xcp-ng.org/forum/post/94556Mon, 30 Jun 2025 13:35:11 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 13:20:18 GMT]]>Just adding another note. When I reboot the host everything is fine then the issues pops back up. There doesn't seem to be any consistency as far as timeframe in between. It just happens randomly

]]>https://xcp-ng.org/forum/post/94552https://xcp-ng.org/forum/post/94552Mon, 30 Jun 2025 13:20:18 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 30 Jun 2025 13:15:50 GMT]]>I am still getting " sd 7:0:0:1: rejecting I/O to offline device" taking all VMs on host down. I then shutdown the VM and power them on, on another Host. I have 4 host in a pool connected iSCSI to a Synology LUN. What is the best way to troubleshoot network issues on xcp-ng.
Thank you

]]>https://xcp-ng.org/forum/post/94550https://xcp-ng.org/forum/post/94550Mon, 30 Jun 2025 13:15:50 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Thu, 19 Jun 2025 14:04:19 GMT]]>Additional finding. The two switches (LAG) are Cisco with STP enabled.

]]>https://xcp-ng.org/forum/post/94091https://xcp-ng.org/forum/post/94091Thu, 19 Jun 2025 14:04:19 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Thu, 19 Jun 2025 13:49:28 GMT]]>I found that the SR has two NICs bonded to two different hardware switches on the iSCSI LUN. Then each host has a management network(Bonded NICS on hosts) and VM network (Bonded NICS). Should I break the SR bond and just go with one NIC? If so, I assume I would be best to shutdown the VMs until I have one NIC on SR reconfigured?

]]>https://xcp-ng.org/forum/post/94090https://xcp-ng.org/forum/post/94090Thu, 19 Jun 2025 13:49:28 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Thu, 19 Jun 2025 13:13:21 GMT]]>@olivierlambert I am looking into that. I just do not have much experience with the networking.

]]>https://xcp-ng.org/forum/post/94089https://xcp-ng.org/forum/post/94089Thu, 19 Jun 2025 13:13:21 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Thu, 19 Jun 2025 12:40:09 GMT]]>There's something fishy about the connectivity to your storage.

]]>https://xcp-ng.org/forum/post/94088https://xcp-ng.org/forum/post/94088Thu, 19 Jun 2025 12:40:09 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Thu, 19 Jun 2025 12:29:41 GMT]]>xcp-ng iscsiadm -m session -P 3 Attached scsi disk sdb State: offline

]]>https://xcp-ng.org/forum/post/94087https://xcp-ng.org/forum/post/94087Thu, 19 Jun 2025 12:29:41 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Thu, 19 Jun 2025 12:25:39 GMT]]>@olivierlambert I had the same issue this morning on one of the hosts I rebooted and was working fine. An emulator required to run this VM Failed to start. sd 7:0:0:1: rejecting I/O to offline device. I am not sure what to check next.

]]>https://xcp-ng.org/forum/post/94086https://xcp-ng.org/forum/post/94086Thu, 19 Jun 2025 12:25:39 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 16 Jun 2025 20:50:41 GMT]]>said in VM failed and won't start. Displaying "An emulator required to run this VM failed to start":

rejecting I/O to offline device

I was able to reboot the host. Now things seam fine. Does that make sense? I mean have you seen this happen then a reboot of the host takes care of it?

]]>https://xcp-ng.org/forum/post/93982https://xcp-ng.org/forum/post/93982Mon, 16 Jun 2025 20:50:41 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 09 Jun 2025 19:21:11 GMT]]>That's exactly what I suspected. You have a connectivity issue with your shared storage.

The device is considered offline because it cannot do any I/O on sector 0

]]>https://xcp-ng.org/forum/post/93818https://xcp-ng.org/forum/post/93818Mon, 09 Jun 2025 19:21:11 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 09 Jun 2025 19:03:15 GMT]]>@olivierlambert It is full of these:
43342477.643853] sd 7:0:0:1: rejecting I/O to offline device
[43342477.643861] print_req_error: I/O error, dev sdb, sector 0

]]>https://xcp-ng.org/forum/post/93817https://xcp-ng.org/forum/post/93817Mon, 09 Jun 2025 19:03:15 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 09 Jun 2025 18:59:44 GMT]]>My suggestion stands from previous messages, check the dmesg output from the problematic host 🙂

]]>https://xcp-ng.org/forum/post/93816https://xcp-ng.org/forum/post/93816Mon, 09 Jun 2025 18:59:44 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 09 Jun 2025 18:51:28 GMT]]>@olivierlambert Ok. I was able to get that VM started on another Host. Now, I need to figure out what is going on with server1. I appreciate all your help! If you have any suggestions please let me know.

]]>https://xcp-ng.org/forum/post/93815https://xcp-ng.org/forum/post/93815Mon, 09 Jun 2025 18:51:28 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 09 Jun 2025 18:45:21 GMT]]>On the pool master:

  1. Restart the toolstack
  2. Immediately after, run the vm reset powerstate command.
]]>https://xcp-ng.org/forum/post/93814https://xcp-ng.org/forum/post/93814Mon, 09 Jun 2025 18:45:21 GMT<![CDATA[Reply to VM failed and won't start. Displaying "An emulator required to run this VM failed to start" on Mon, 09 Jun 2025 17:53:15 GMT]]>[12:52 iahost-xcpng-server3 ~]# xe vm-reset-powerstate vm=dfcdcce6-b6e1-d865-6f2b-472a35b1fae7 --force
This operation cannot be completed because the server is still live.
host: 6985c3a2-091d-488b-8de6-4bf40985987b (iahost-xcpng-server3)

]]>https://xcp-ng.org/forum/post/93813https://xcp-ng.org/forum/post/93813Mon, 09 Jun 2025 17:53:15 GMT