"SR_BACKEND_FAILURE_47"
-
Good Day,
After about 50 threads online I'm hitting a wall with "SR_BACKEND_FAILURE_47".
Currently there are 4 Hosts (All Masters), 18/24 VM's Online. 95% of the hosts are situated on a HP Datastore.
I'm getting the 47 error on various places but the biggest concern currently is one of the hosts are failing to make a connection to the Datastore/ISCSI, therefore none of its VM's are starting.
The backend connection from the Host to the Datastore has no issues (Authentication, Connection etc) but its like the front end of the host just fails to connect. Have tried a few options of detaching and adding it back and reach the same spot.
All 3 other hosts are able to access the ISCSI and work just fine, al bit also receiving a few 47 error on other occasions e.g Creating Snapshots.
Hope someone can direct me into a path to get the VM's going.
If possible I can also start the VM's on a different host, I see them on the datastore but with the host not being able to bring the VM's online the migration option fails, so not sure what other option there are?
Would appreciate any support.
-
Hi,
On top of my head:- I would check if your iSCSI session is still up on all hosts (eg with iscsiadm)
- When you say "host failing to make connection to the datastore", can you be more explicit? What kind of errors? Have you looked at the SMlog on the host in question?
- The troubleshooting order is always from basic connectivity (ping) to higher level (iscsi connected OK, then VDI creation OK etc.). This is helpful to seek at which point something goes wrong
- Also, knowing your XCP-ng version will help
-
@olivierlambert Thanks for your response:
- getting the below:
discovery.startup = manual
discovery.type = sendtargets
discovery.sendtargets.address = 10.50.12.15
discovery.sendtargets.port = 3260
discovery.sendtargets.auth.authmethod = None
discovery.sendtargets.auth.username = <empty>
discovery.sendtargets.auth.password = <empty>
discovery.sendtargets.auth.username_in = <empty>
discovery.sendtargets.auth.password_in = <empty>
discovery.sendtargets.timeo.login_timeout = 15
discovery.sendtargets.use_discoveryd = No
discovery.sendtargets.discoveryd_poll_inval = 30
discovery.sendtargets.reopen_max = 5
discovery.sendtargets.timeo.auth_timeout = 45
discovery.sendtargets.timeo.active_timeout = 30
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 3276-
The iSCSI shows disconnected on all the hosts, and when trying to manually connect we receive that BACKEND_Failure 47/46.
-
Connectivity seems perfect both interfaces are pingable (multipath), also on the Datastore its the connections are listed.
Strangely I can add the iscsi as a new datastore then it appears empty, without picking up any previous data. I then removed the newly created one as to not overwrite any data on the MSA.
It also shows all the VM's listed on the datastore when browsing the iSCSI storage.
4 Version XOA 5.94.2
-
Weird. Likely unrelated, but you are using an old version of XOA by the way -
@olivierlambert Hi Oliver, looks like we made progress to get to a point where the hosts identifies the ISCSi/LUN's, just cant cant get it attached:
-
@dewalddk XOA is now on : 5.98.1
-
@dewalddk have tried a few command clearing the ISCSI and using the Centre, but everything fails. Pick up the LUN but when attaching wants to start on a clean Storage pool rather then the old existing one
-
-
Probably worth looking at SMlog to get more details of this rvgstats failed. It's like a iSCSI multipath bad configuration or something weird with the iSCSI target, just for this host
-
@dewalddk
HiIn my case, this error occurred because I had a duplicate iSCSI device name.
Even if the iqn is different, xcp-ng will throw an error if the device name is duplicated.enjoy
-
Oh that's great! Nice catch
How did you find the root cause?
