VM failed and won't start. Displaying "An emulator required to run this VM failed to start"
-
@olivierlambert Ok. I was able to get that VM started on another Host. Now, I need to figure out what is going on with server1. I appreciate all your help! If you have any suggestions please let me know.
-
My suggestion stands from previous messages, check the
dmesgoutput from the problematic host
-
@olivierlambert It is full of these:
43342477.643853] sd 7:0:0:1: rejecting I/O to offline device
[43342477.643861] print_req_error: I/O error, dev sdb, sector 0 -
That's exactly what I suspected. You have a connectivity issue with your shared storage.
The device is considered offline because it cannot do any I/O on sector 0
-
said in VM failed and won't start. Displaying "An emulator required to run this VM failed to start":
rejecting I/O to offline device
I was able to reboot the host. Now things seam fine. Does that make sense? I mean have you seen this happen then a reboot of the host takes care of it?
-
@olivierlambert I had the same issue this morning on one of the hosts I rebooted and was working fine. An emulator required to run this VM Failed to start. sd 7:0:0:1: rejecting I/O to offline device. I am not sure what to check next.
-
xcp-ng iscsiadm -m session -P 3 Attached scsi disk sdb State: offline
-
There's something fishy about the connectivity to your storage.
-
@olivierlambert I am looking into that. I just do not have much experience with the networking.
-
I found that the SR has two NICs bonded to two different hardware switches on the iSCSI LUN. Then each host has a management network(Bonded NICS on hosts) and VM network (Bonded NICS). Should I break the SR bond and just go with one NIC? If so, I assume I would be best to shutdown the VMs until I have one NIC on SR reconfigured?
-
Additional finding. The two switches (LAG) are Cisco with STP enabled.
-
I am still getting " sd 7:0:0:1: rejecting I/O to offline device" taking all VMs on host down. I then shutdown the VM and power them on, on another Host. I have 4 host in a pool connected iSCSI to a Synology LUN. What is the best way to troubleshoot network issues on xcp-ng.
Thank you -
Just adding another note. When I reboot the host everything is fine then the issues pops back up. There doesn't seem to be any consistency as far as timeframe in between. It just happens randomly
-
Any help is greatly appreciated.
-
That's not a trivial problem that you have, it seems to be intermittent and could be anything between host and the storage. I assume you can see the issue as soon you have the
rejecting I/O to offline deviceright? -
Yes
-
At least we know there's a root cause: the device becomes offline. Could it be a power setting or something on the device itself?
-
Well, it doesn't happen to every host at the same time, it's just random.
-
What log is best to look at for network disconnects?
-
Well
dmesgis fine, I would also check the switch logs if you have anything happening on the ports where you host are connected. Since you can find when it's happening withdmesg, I would use that to correlate with anything else happening few minutes before.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login