XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Xen orchestra VM Console broken?

    Scheduled Pinned Locked Moved Xen Orchestra
    36 Posts 6 Posters 7.7k Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ronivayR Offline
      ronivay Top contributor @olivierlambert
      last edited by ronivay

      Interestingly console for some old VM's with old OS are working, like centos 6, debian 8 but anything above those (centos7, debian 10, debian 11) aren't. I'm getting same errors in browser as listed by @bberndt earlier. This can be found from host xensource.log

      Apr 14 09:55:45 xs xapi: [debug|xs|33053260 INET :::80|Connection to VM console R:7839b84e856e|console] VM OpaqueRef:be204b50-6028-41f7-9744-e94ad7329046 console port: Some unix:/var/run/xen/vnc-290
      Apr 14 09:55:45 xs xapi: [error|xs|33053260 INET :::80|Connection to VM console R:7839b84e856e|console] No implementation for web-sockets console proxy to a Unix domain socket
      Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace] Connection to VM console R:7839b84e856e failed with exception (Failure "ws_proxy: not implemented")
      Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace] Raised (Failure "ws_proxy: not implemented")
      Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace] 1/1 xapi @ xs Raised at file (Thread 33053260 has no backtrace table. Was with_backtraces called?, line 0
      Apr 14 09:55:46 xs xapi: [error|xs|33053260 INET :::80||backtrace]
      

      So far i've tested with XO built with XenOrchestraInstallerUpdater (obviously ;)) and also built one manually, same symptoms. I've tried XO with and without reverse proxy, with http, with https, with one in same private network as XCP-ng host, with one connecting to it over VPN. No special firewalling or filtering in between and no HTTP proxy.

      But as the error from host says, it sounds like it’s simply missing support for this type of console connection.

      E: and just for the record. wsproxy package is installed:

      # rpm -q wsproxy
      wsproxy-1.6.0-4.el7.centos.x86_64
      
      florentF 1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates πŸͺ Co-Founder CEO
        last edited by olivierlambert

        @ronivay Do you mean it works for PV guests but not HVM ones?

        edit: that might be the reason why we can't reproduce since we probably test on PV guest πŸ€”

        ronivayR stormiS 2 Replies Last reply Reply Quote 0
        • ronivayR Offline
          ronivay Top contributor @olivierlambert
          last edited by

          Yeah good point, all those old ones are indeed PV guests and all the rest are HVM.

          florentF 1 Reply Last reply Reply Quote 0
          • florentF Offline
            florent Vates πŸͺ XO Team @ronivay
            last edited by

            @ronivay that's a great information. I can reproduce the comportment with xenserver 7.1 CU 2
            Thank you

            1 Reply Last reply Reply Quote 2
            • florentF Offline
              florent Vates πŸͺ XO Team @ronivay
              last edited by olivierlambert

              @ronivay @bberndt Could you try this branch fix_console_proxy_xcp_7x containing a quick fix (fallback for older hosts to the old method)?

              from this PR: https://github.com/vatesfr/xen-orchestra/pull/6191

              fbeauchamp opened this pull request in vatesfr/xen-orchestra

              closed fix(xo-server/proxy-console): fallback on TCP if WS not available #6191

              ronivayR 1 Reply Last reply Reply Quote 0
              • ronivayR Offline
                ronivay Top contributor @florent
                last edited by ronivay

                Thanks. This seems to fix it for me at least. XO logs UNEXPECTED RESPONSE and FALLBACK for those HVM VM's and console opens up fine.

                florentF 1 Reply Last reply Reply Quote 1
                • florentF Offline
                  florent Vates πŸͺ XO Team @ronivay
                  last edited by

                  @ronivay yeah

                  I'm making a little code cleanup before merging in master (removing console , adding changelog)
                  We'll probably make another patch release with this fix.

                  I'm glad you saw it and give us such a precise feedback.

                  1 Reply Last reply Reply Quote 0
                  • stormiS Offline
                    stormi Vates πŸͺ XCP-ng Team @olivierlambert
                    last edited by

                    @olivierlambert said in Xen orchestra VM Console broken?:

                    @ronivay Do you mean it works for PV guests but not HVM ones?

                    edit: that might be the reason why we can't reproduce since we probably test on PV guest πŸ€”

                    When I tested yesterday, this was on HVM guests.

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates πŸͺ Co-Founder CEO
                      last edited by

                      You need both HVM guest AND XS < 7.2

                      stormiS 1 Reply Last reply Reply Quote 0
                      • stormiS Offline
                        stormi Vates πŸͺ XCP-ng Team @olivierlambert
                        last edited by

                        @olivierlambert Yes, that was the case of one of the tests as far as I can tell.

                        florentF 1 Reply Last reply Reply Quote 0
                        • B Offline
                          bberndt
                          last edited by

                          I don't know if this helps:
                          most of my VMs are Windows. There are a few Linux, Centos 7+, Rocky, and Ubuntu 18.
                          the production XO I use, is behind an Apache reverse proxy, using https. The temporary one, is just XO, using http on 8080. Looks like all of these are PVHVM, (so, HVM?) it claims. Both of these XO's are VMs on the same host, and the other hosts are on the same subnet, but some are on a different VLAN. There are also a number that are in a co-location facility, connected via a point to point fiber link, on a different VLAN, but all behind our edge router.
                          Console working: XCP 8.1, XS 7.1
                          Console not working: XCP 7.6 - I can't say for sure that these have any updates applied, though.

                          florentF 1 Reply Last reply Reply Quote 0
                          • florentF Offline
                            florent Vates πŸͺ XO Team @stormi
                            last edited by

                            @stormi at least it failed on 7.1CU2 installed this morning.
                            I implemented a fallback not checking the version, but catching the specific error message and going back to the old method

                            1 Reply Last reply Reply Quote 1
                            • florentF Offline
                              florent Vates πŸͺ XO Team @bberndt
                              last edited by

                              @bberndt are you able to test the fix ?

                              B 1 Reply Last reply Reply Quote 0
                              • B Offline
                                bberndt @florent
                                last edited by

                                @florent
                                I ran the install upgrade script from @ronivay with the fix_console_proxy_xcp_7x branch. Seems to be working.

                                1 Reply Last reply Reply Quote 1
                                • ronivayR Offline
                                  ronivay Top contributor
                                  last edited by ronivay

                                  @florent I see that the issue is still open, but looks like PR https://github.com/vatesfr/xen-orchestra/pull/6191 was merged so i assume it's not WIP anymore? Just wanted to let you know that some of the changes in that PR before merge made the initial fix non functional. Tested with installation from latest master branch commit.

                                  Errors in browser have changed slightly:

                                  Failed when connecting: Failed to connect to server ( (code: 1005)) util.js:44:54
                                      Error util.js:44
                                      _fail rfb.js:578
                                      f rfb.js:233
                                      onclose websock.js:305
                                  

                                  and

                                  Tried changing state of a disconnected RFB object util.js:44:54
                                      Error util.js:44
                                      _updateConnectionState rfb.js:473
                                      disconnect rfb.js:284
                                      _clean react-novnc.js:66
                                      componentWillUnmount react-novnc.js:115
                                      React 139
                                  

                                  XO doesn't log anything related to fallback method, just a normal console proxy INFO message. Host still gives the same Failure "ws_proxy: not implemented" error and seems that the fallback to TCP doesn't happen and console doesn't open. Still works for PV guests as it did before.

                                  fbeauchamp opened this pull request in vatesfr/xen-orchestra

                                  closed fix(xo-server/proxy-console): fallback on TCP if WS not available #6191

                                  florentF 1 Reply Last reply Reply Quote 0
                                  • florentF Offline
                                    florent Vates πŸͺ XO Team @ronivay
                                    last edited by

                                    @ronivay hi , something should not have worked as intended, I'm working on it. Your server is a 7.6 ?

                                    ronivayR 1 Reply Last reply Reply Quote 0
                                    • ronivayR Offline
                                      ronivay Top contributor @florent
                                      last edited by

                                      yeah it’s 7.6 i see this with

                                      florentF 1 Reply Last reply Reply Quote 0
                                      • florentF Offline
                                        florent Vates πŸͺ XO Team @ronivay
                                        last edited by olivierlambert

                                        @ronivay hi, can you test this branch fix_console_bis before we merge it?

                                        thank you for your report

                                        ronivayR 1 Reply Last reply Reply Quote 0
                                        • ronivayR Offline
                                          ronivay Top contributor @florent
                                          last edited by

                                          @florent thanks, yeah this seems to fix it again.

                                          1 Reply Last reply Reply Quote 2
                                          • B Offline
                                            bberndt
                                            last edited by

                                            Updated to Master build, a66ae, xo-server 5.92.0, xo-web 5.96.0.
                                            Consoles seem to work fine again.
                                            thanks all!

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post