Xen orchestra VM Console broken?
@olivierlambert , I received exact instructions from @ronivay on how to do that. Using commit b9ff3db9b in my new temporary install of XO, everything works fine.
Moving to c99120b seems to break the console on anything other than XCP 8.2.
And I've mentioned before I think: all of our hardware does support XCP 8.2, but almost none of the RAID card monitor software works there. 80% of them are using 3Ware/ LSI 9750's. I can't recall the exact problem, but the executable for the web-admin doesn't run.
- We'll restore backward compatibility and fix that ( @florent will fix that commit). Thanks for reporting it
- Your previous XCP-ng installs aren't patched anymore, so you are at risk. You should really migrate to it, regardless the rest (and maybe seek for a solution for your RAID monitoring, but installing stuff in dom0 is not recommended in general). At least this advice worth for production use, not home lab.
Issue created: https://github.com/vatesfr/xen-orchestra/issues/6187
We can't reproduce the problem here
hrm, well, I bumped my temp install of XO back to the master branch, and consoles broke. -Appears- to only broke with XCP 7.6. 7.1, sites worked.
I can also reproduce this with XCP-ng 7.6. I’ll try to do some additional tests. Let me know if there’s anything i can assist with.
Okay we tested with XS 7.1 and XCP-ng 7.5, without any luck
Can you describe a bit more your test protocol?
@ronivay I did a lot of tests yesterday (master/latest/old and new commit) , with a wide range of xcp and didn't reproduce it
Do you use any http proxy between XO an XCP ? Do you use some kind of network filtering ? do you use xo in HTTP or HTTPS mode ?
Interestingly console for some old VM's with old OS are working, like centos 6, debian 8 but anything above those (centos7, debian 10, debian 11) aren't. I'm getting same errors in browser as listed by @bberndt earlier. This can be found from host
Apr 14 09:55:45 xs xapi: [debug|xs|33053260 INET :::80|Connection to VM console R:7839b84e856e|console] VM OpaqueRef:be204b50-6028-41f7-9744-e94ad7329046 console port: Some unix:/var/run/xen/vnc-290 Apr 14 09:55:45 xs xapi: [error|xs|33053260 INET :::80|Connection to VM console R:7839b84e856e|console] No implementation for web-sockets console proxy to a Unix domain socket Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace] Connection to VM console R:7839b84e856e failed with exception (Failure "ws_proxy: not implemented") Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace] Raised (Failure "ws_proxy: not implemented") Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace] 1/1 xapi @ xs Raised at file (Thread 33053260 has no backtrace table. Was with_backtraces called?, line 0 Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace]
So far i've tested with XO built with XenOrchestraInstallerUpdater (obviously ;)) and also built one manually, same symptoms. I've tried XO with and without reverse proxy, with http, with https, with one in same private network as XCP-ng host, with one connecting to it over VPN. No special firewalling or filtering in between and no HTTP proxy.
But as the error from host says, it sounds like it’s simply missing support for this type of console connection.
E: and just for the record. wsproxy package is installed:
# rpm -q wsproxy wsproxy-1.6.0-4.el7.centos.x86_64
@ronivay Do you mean it works for PV guests but not HVM ones?
edit: that might be the reason why we can't reproduce since we probably test on PV guest
Yeah good point, all those old ones are indeed PV guests and all the rest are HVM.
@ronivay that's a great information. I can reproduce the comportment with xenserver 7.1 CU 2
from this PR: https://github.com/vatesfr/xen-orchestra/pull/6191
Thanks. This seems to fix it for me at least. XO logs
FALLBACKfor those HVM VM's and console opens up fine.
I'm making a little code cleanup before merging in master (removing console , adding changelog)
We'll probably make another patch release with this fix.
I'm glad you saw it and give us such a precise feedback.
You need both HVM guest AND XS < 7.2
@olivierlambert Yes, that was the case of one of the tests as far as I can tell.
I don't know if this helps:
most of my VMs are Windows. There are a few Linux, Centos 7+, Rocky, and Ubuntu 18.
the production XO I use, is behind an Apache reverse proxy, using https. The temporary one, is just XO, using http on 8080. Looks like all of these are PVHVM, (so, HVM?) it claims. Both of these XO's are VMs on the same host, and the other hosts are on the same subnet, but some are on a different VLAN. There are also a number that are in a co-location facility, connected via a point to point fiber link, on a different VLAN, but all behind our edge router.
Console working: XCP 8.1, XS 7.1
Console not working: XCP 7.6 - I can't say for sure that these have any updates applied, though.
@stormi at least it failed on 7.1CU2 installed this morning.
I implemented a fallback not checking the version, but catching the specific error message and going back to the old method