VM failed and won't start. Displaying "An emulator required to run this VM failed to start"
-
@olivierlambert "Failed","Shutting down VM 'AlmaLinux8-Testing' on 'iahost-xcpng-server3'
Internal error: Object with type VM and id dfcdcce6-b6e1-d865-6f2b-472a35b1fae7/config does not exist in xenopsd","iahost-xcpng-server3","Jun 9, 2025 12:28 PM" -
That's why you need to power reset it first.
Also please check the host
dmesg -
@olivierlambert Oh, I'm sorry. You mean power restart the host server?
-
[12:52 iahost-xcpng-server3 ~]# xe vm-reset-powerstate vm=dfcdcce6-b6e1-d865-6f2b-472a35b1fae7 --force
This operation cannot be completed because the server is still live.
host: 6985c3a2-091d-488b-8de6-4bf40985987b (iahost-xcpng-server3) -
On the pool master:
- Restart the toolstack
- Immediately after, run the vm reset powerstate command.
-
@olivierlambert Ok. I was able to get that VM started on another Host. Now, I need to figure out what is going on with server1. I appreciate all your help! If you have any suggestions please let me know.
-
My suggestion stands from previous messages, check the
dmesgoutput from the problematic host
-
@olivierlambert It is full of these:
43342477.643853] sd 7:0:0:1: rejecting I/O to offline device
[43342477.643861] print_req_error: I/O error, dev sdb, sector 0 -
That's exactly what I suspected. You have a connectivity issue with your shared storage.
The device is considered offline because it cannot do any I/O on sector 0
-
said in VM failed and won't start. Displaying "An emulator required to run this VM failed to start":
rejecting I/O to offline device
I was able to reboot the host. Now things seam fine. Does that make sense? I mean have you seen this happen then a reboot of the host takes care of it?
-
@olivierlambert I had the same issue this morning on one of the hosts I rebooted and was working fine. An emulator required to run this VM Failed to start. sd 7:0:0:1: rejecting I/O to offline device. I am not sure what to check next.
-
xcp-ng iscsiadm -m session -P 3 Attached scsi disk sdb State: offline
-
There's something fishy about the connectivity to your storage.
-
@olivierlambert I am looking into that. I just do not have much experience with the networking.
-
I found that the SR has two NICs bonded to two different hardware switches on the iSCSI LUN. Then each host has a management network(Bonded NICS on hosts) and VM network (Bonded NICS). Should I break the SR bond and just go with one NIC? If so, I assume I would be best to shutdown the VMs until I have one NIC on SR reconfigured?
-
Additional finding. The two switches (LAG) are Cisco with STP enabled.
-
I am still getting " sd 7:0:0:1: rejecting I/O to offline device" taking all VMs on host down. I then shutdown the VM and power them on, on another Host. I have 4 host in a pool connected iSCSI to a Synology LUN. What is the best way to troubleshoot network issues on xcp-ng.
Thank you -
Just adding another note. When I reboot the host everything is fine then the issues pops back up. There doesn't seem to be any consistency as far as timeframe in between. It just happens randomly
-
Any help is greatly appreciated.
-
That's not a trivial problem that you have, it seems to be intermittent and could be anything between host and the storage. I assume you can see the issue as soon you have the
rejecting I/O to offline deviceright?
