XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XO5 breaks after defaulting to XO6 (from source)

    Scheduled Pinned Locked Moved Xen Orchestra
    36 Posts 6 Posters 341 Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • MathieuRAM Offline
      MathieuRA Vates 🪐 XO Team @probain
      last edited by

      @probain Do you have a configuration file that overrides xo-server configuration for [http.mounts] and [http.proxies]?
      `

      P 1 Reply Last reply Reply Quote 0
      • P Offline
        probain @MathieuRA
        last edited by

        @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

        @probain Do you have a configuration file that overrides xo-server configuration for [http.mounts] and [http.proxies]?
        `

        Adding the following, breaks XO6 ("Cannot get /"). And XO5 still is stuck on "Still Loading.

        [http.upgrade]
'/v5/api' = true
'/v5/api/updater' = true

[http.routes]
'/v5/api' = { upgrade = true }
'/v5/api/updater' = { upgrade = true }

# List of files/directories which will be served.
[http.mounts]
#'/any/url' = '/path/to/directory'
'/v6' = '../../@xen-orchestra/web/dist/'
'/' = '../xo-web/dist/'

        If you're curious as to how I set everything up. I use my ansible-role (shared previously).
        Link for reference: https://github.com/cloudrootab/ansible_role_xoce/

        MathieuRAM 1 Reply Last reply Reply Quote 0
        • MathieuRAM Offline
          MathieuRA Vates 🪐 XO Team @probain
          last edited by

          @probain Do you have any xo-server logs?

          can you fetch /rest/v0/gui-routes endpoint?

          P 1 Reply Last reply Reply Quote 0
          • P Offline
            probain @MathieuRA
            last edited by olivierlambert

            @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

            @probain Do you have any xo-server logs?

            can you fetch /rest/v0/gui-routes endpoint?

            This is what results on the page

            {
  "xo5": "/v5",
  "xo6": "/v6"
}
            MathieuRAM 1 Reply Last reply Reply Quote 0
            • MathieuRAM Offline
              MathieuRA Vates 🪐 XO Team @probain
              last edited by

              @probain Can you provide me with the logs from xo-server at its launch?

              P 1 Reply Last reply Reply Quote 0
              • P Offline
                probain @MathieuRA
                last edited by probain

                @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

                @probain Can you provide me with the logs from xo-server at its launch?

                I would love to. However, I'm unsure which logs you're refering to.
                Since I can't work with XO5. The logs from therein aren't available.

                However. Doing a journalctl -ef when doing a systemctl start xo-server results in

                Dec 16 16:54:00 xo sudo[50689]: pam_unix(sudo:session): session opened for user root(uid=0) by REDACTED(uid=1002)
Dec 16 16:54:00 xo systemd[1]: Started xo-server.service - XO Server.
Dec 16 16:54:00 xo sudo[50689]: pam_unix(sudo:session): session closed for user root
Dec 16 16:54:02 xo xo-server[50694]: 2025-12-16T15:54:02.774Z xo:main WARN could not detect current commit {
Dec 16 16:54:02 xo xo-server[50694]:   error: Error: Command failed: git rev-parse --short HEAD
Dec 16 16:54:02 xo xo-server[50694]:   fatal: detected dubious ownership in repository at '/opt/xen-orchestra'
Dec 16 16:54:02 xo xo-server[50694]:   To add an exception for this directory, call:
Dec 16 16:54:02 xo xo-server[50694]:
Dec 16 16:54:02 xo xo-server[50694]:           git config --global --add safe.directory /opt/xen-orchestra
Dec 16 16:54:02 xo xo-server[50694]:
Dec 16 16:54:02 xo xo-server[50694]:       at genericNodeError (node:internal/errors:983:15)
Dec 16 16:54:02 xo xo-server[50694]:       at wrappedFn (node:internal/errors:537:14)
Dec 16 16:54:02 xo xo-server[50694]:       at ChildProcess.exithandler (node:child_process:417:12)
Dec 16 16:54:02 xo xo-server[50694]:       at ChildProcess.emit (node:events:519:28)
Dec 16 16:54:02 xo xo-server[50694]:       at ChildProcess.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:52:17)
Dec 16 16:54:02 xo xo-server[50694]:       at maybeClose (node:internal/child_process:1101:16)
Dec 16 16:54:02 xo xo-server[50694]:       at Socket.<anonymous> (node:internal/child_process:456:11)
Dec 16 16:54:02 xo xo-server[50694]:       at Socket.emit (node:events:519:28)
Dec 16 16:54:02 xo xo-server[50694]:       at Socket.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:52:17)
Dec 16 16:54:02 xo xo-server[50694]:       at Pipe.<anonymous> (node:net:346:12)
Dec 16 16:54:02 xo xo-server[50694]:       at Pipe.callbackTrampoline (node:internal/async_hooks:130:17) {
Dec 16 16:54:02 xo xo-server[50694]:     code: 128,
Dec 16 16:54:02 xo xo-server[50694]:     killed: false,
Dec 16 16:54:02 xo xo-server[50694]:     signal: null,
Dec 16 16:54:02 xo xo-server[50694]:     cmd: 'git rev-parse --short HEAD'
Dec 16 16:54:02 xo xo-server[50694]:   }
Dec 16 16:54:02 xo xo-server[50694]: }
Dec 16 16:54:02 xo xo-server[50694]: 2025-12-16T15:54:02.777Z xo:main INFO Starting xo-server v5.194.2 (https://github.com/vatesfr/xen-orchestra/commit/7c1764a39)
Dec 16 16:54:02 xo xo-server[50694]: 2025-12-16T15:54:02.794Z xo:main INFO Configuration loaded.
Dec 16 16:54:02 xo xo-server[50694]: 2025-12-16T15:54:02.818Z xo:main INFO Web server listening on https://[::]:8443
Dec 16 16:54:02 xo xo-server[50694]: 2025-12-16T15:54:02.819Z xo:main INFO Group changed to xo-user
Dec 16 16:54:02 xo xo-server[50694]: 2025-12-16T15:54:02.820Z xo:main INFO User changed to xo-user
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.296Z xo:mixins:hooks WARN start failure {
Dec 16 16:54:03 xo xo-server[50694]:   error: Error: spawn xenstore-read ENOENT
Dec 16 16:54:03 xo xo-server[50694]:       at Process.ChildProcess._handle.onexit (node:internal/child_process:285:19)
Dec 16 16:54:03 xo xo-server[50694]:       at onErrorNT (node:internal/child_process:483:16)
Dec 16 16:54:03 xo xo-server[50694]:       at processTicksAndRejections (node:internal/process/task_queues:90:21) {
Dec 16 16:54:03 xo xo-server[50694]:     errno: -2,
Dec 16 16:54:03 xo xo-server[50694]:     code: 'ENOENT',
Dec 16 16:54:03 xo xo-server[50694]:     syscall: 'spawn xenstore-read',
Dec 16 16:54:03 xo xo-server[50694]:     path: 'xenstore-read',
Dec 16 16:54:03 xo xo-server[50694]:     spawnargs: [ 'vm' ],
Dec 16 16:54:03 xo xo-server[50694]:     cmd: 'xenstore-read vm'
Dec 16 16:54:03 xo xo-server[50694]:   }
Dec 16 16:54:03 xo xo-server[50694]: }
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.302Z xo:mixins:hooks WARN start failure {
Dec 16 16:54:03 xo xo-server[50694]:   error: Error: Command failed with exit code 1: losetup -D
Dec 16 16:54:03 xo xo-server[50694]:   losetup: /dev/loop1: detach failed: Permission denied
Dec 16 16:54:03 xo xo-server[50694]:   losetup: /dev/loop4: detach failed: Permission denied
Dec 16 16:54:03 xo xo-server[50694]:   losetup: /dev/loop2: detach failed: Permission denied
Dec 16 16:54:03 xo xo-server[50694]:   losetup: /dev/loop0: detach failed: Permission denied
Dec 16 16:54:03 xo xo-server[50694]:   losetup: /dev/loop5: detach failed: Permission denied
Dec 16 16:54:03 xo xo-server[50694]:   losetup: /dev/loop3: detach failed: Permission denied
Dec 16 16:54:03 xo xo-server[50694]:       at makeError (file:///opt/xen-orchestra/packages/xo-server/node_modules/execa/lib/error.js:60:11)
Dec 16 16:54:03 xo xo-server[50694]:       at handlePromise (file:///opt/xen-orchestra/packages/xo-server/node_modules/execa/index.js:124:26) {
Dec 16 16:54:03 xo xo-server[50694]:     shortMessage: 'Command failed with exit code 1: losetup -D',
Dec 16 16:54:03 xo xo-server[50694]:     command: 'losetup -D',
Dec 16 16:54:03 xo xo-server[50694]:     escapedCommand: 'losetup -D',
Dec 16 16:54:03 xo xo-server[50694]:     exitCode: 1,
Dec 16 16:54:03 xo xo-server[50694]:     signal: undefined,
Dec 16 16:54:03 xo xo-server[50694]:     signalDescription: undefined,
Dec 16 16:54:03 xo xo-server[50694]:     stdout: '',
Dec 16 16:54:03 xo xo-server[50694]:     stderr: 'losetup: /dev/loop1: detach failed: Permission denied\n' +
Dec 16 16:54:03 xo xo-server[50694]:       'losetup: /dev/loop4: detach failed: Permission denied\n' +
Dec 16 16:54:03 xo xo-server[50694]:       'losetup: /dev/loop2: detach failed: Permission denied\n' +
Dec 16 16:54:03 xo xo-server[50694]:       'losetup: /dev/loop0: detach failed: Permission denied\n' +
Dec 16 16:54:03 xo xo-server[50694]:       'losetup: /dev/loop5: detach failed: Permission denied\n' +
Dec 16 16:54:03 xo xo-server[50694]:       'losetup: /dev/loop3: detach failed: Permission denied',
Dec 16 16:54:03 xo xo-server[50694]:     cwd: '/',
Dec 16 16:54:03 xo xo-server[50694]:     failed: true,
Dec 16 16:54:03 xo xo-server[50694]:     timedOut: false,
Dec 16 16:54:03 xo xo-server[50694]:     isCanceled: false,
Dec 16 16:54:03 xo xo-server[50694]:     killed: false
Dec 16 16:54:03 xo xo-server[50694]:   }
Dec 16 16:54:03 xo xo-server[50694]: }
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.578Z xo:main INFO Setting up /robots.txt → /opt/xen-orchestra/packages/xo-server/robots.txt
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.769Z xo:plugin INFO register audit
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.770Z xo:plugin INFO register auth-github
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.770Z xo:plugin INFO register auth-google
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.770Z xo:plugin INFO register auth-ldap
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.771Z xo:plugin INFO register auth-oidc
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.771Z xo:plugin INFO register auth-saml
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.771Z xo:plugin INFO register backup-reports
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.772Z xo:plugin INFO register load-balancer
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.772Z xo:plugin INFO register netbox
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.772Z xo:plugin INFO register perf-alert
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.773Z xo:plugin INFO register sdn-controller
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.773Z xo:plugin INFO register test-plugin
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.773Z xo:plugin INFO register transport-email
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.774Z xo:plugin INFO register transport-icinga2
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.774Z xo:plugin INFO register transport-nagios
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.774Z xo:plugin INFO register transport-slack
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.775Z xo:plugin INFO register transport-xmpp
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.775Z xo:plugin INFO register usage-report
Dec 16 16:54:03 xo xo-server[50694]: 2025-12-16T15:54:03.775Z xo:plugin INFO register web-hooks
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.422Z xo:plugin INFO successfully register auth-github
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register auth-google
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register auth-ldap
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register test-plugin
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register transport-icinga2
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register auth-oidc
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register netbox
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register transport-nagios
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register transport-slack
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register transport-xmpp
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register web-hooks
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.423Z xo:plugin INFO successfully register load-balancer
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.424Z xo:plugin INFO successfully register usage-report
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.424Z xo:plugin INFO successfully register backup-reports
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.424Z xo:plugin INFO successfully register auth-saml
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.424Z xo:plugin INFO successfully register transport-email
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.424Z xo:plugin INFO successfully register perf-alert
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.429Z xo:plugin INFO successfully register audit
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.451Z xo:plugin INFO successfully register sdn-controller
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.452Z xo:main INFO Setting up /v5 → /opt/xen-orchestra/packages/xo-web/dist
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.452Z xo:main INFO Setting up /v6 → /opt/xen-orchestra/@xen-orchestra/web/dist
Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.452Z xo:main INFO Setting up / → /opt/xen-orchestra/@xen-orchestra/web/dist

                Update:
                I have no idea why your PR is shown.

                0 MathieuRA committed to vatesfr/xen-orchestra 
                fix(xo-server): fix proxied urls when using ssl (#9314)

introduced by 03cea8b24d48e685a2a5a04a39db70d67d87eca0

Fixed [forum#11681](https://xcp-ng.org/forum/topic/11681/xo5-breaks-after-defaulting-to-xo6-from-source/16?_=1765875445829)
                MathieuRAM 1 Reply Last reply Reply Quote 0
                • MathieuRAM Offline
                  MathieuRA Vates 🪐 XO Team @probain
                  last edited by MathieuRA

                  @probain That's what I was looking for.

                  It's strange you are saying you receive an error: ("Cannot get /") because logs show / is mounted and exposes XO6.

                  Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.452Z xo:main INFO Setting up / → /opt/xen-orchestra/@xen-orchestra/web/dist

                  Even when I did the same "fixes" that helped my second lab.

                  The only difference with your second lab is that you use HAProxy?
                  Both are listening on IPv6?

                  P 1 Reply Last reply Reply Quote 0
                  • P Offline
                    probain @MathieuRA
                    last edited by

                    @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

                    @probain That's what I was looking for.

                    It's strange you are saying you receive an error: ("Cannot get /") because logs show / is mounted and exposes XO6.

                    Dec 16 16:54:04 xo xo-server[50694]: 2025-12-16T15:54:04.452Z xo:main INFO Setting up / → /opt/xen-orchestra/@xen-orchestra/web/dist

                    Even when I did the same "fixes" that helped my second lab.

                    The only difference with your second lab is that you use HAProxy?
                    Both are listening on IPv6?

                    I reverted those changes from above with the [http.mounts] and [http.proxies] parts. Since that broke things even more. Sorry, I should've clarified that. I've pretty much reverted all of my troubleshooting experiments. In hopes to make it more clear if certain troubleshooting steps are needed.

                    My primary lab uses Haproxy. The second one doesn't (second lab doesn't run unprivilged either).

                    The logs I shared, were from the latest commit 32b3c0b5fcba17585566f0981ef62bef74c56451 (I believe).

                    MathieuRAM 1 Reply Last reply Reply Quote 0
                    • MathieuRAM Offline
                      MathieuRA Vates 🪐 XO Team @probain
                      last edited by

                      @probain

                      Web server listening on https://[::]:8443

                      Well, I may have an idea about your issue.
                      The code is looking for the port 443 to know if he should use secure protocol ('wss:' / 'https:')
                      Is your second lab also in https on the 8443 port?

                      P 1 Reply Last reply Reply Quote 0
                      • P Offline
                        probain @MathieuRA
                        last edited by

                        @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

                        @probain

                        Web server listening on https://[::]:8443

                        Well, I may have an idea about your issue.
                        The code is looking for the port 443 to know if he should use secure protocol ('wss:' / 'https:')
                        Is your second lab also in https on the 8443 port?

                        Hmm, no the second lab is more "standard" and uses 443.
                        I can try and set 443 on the primary lab as well to test.

                        MathieuRAM 1 Reply Last reply Reply Quote 0
                        • MathieuRAM Offline
                          MathieuRA Vates 🪐 XO Team @probain
                          last edited by

                          @probain If you can, it will help a lot. Thanks

                          P 1 Reply Last reply Reply Quote 0
                          • P Offline
                            probain @MathieuRA
                            last edited by

                            @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

                            @probain If you can, it will help a lot. Thanks

                            I remember why I chose to use 8443. It is because the xo-user running the service isn't running with root privileges. So doing it with an ordinary user was way more difficult to setup than just use 8443.

                            But...
                            Switching over so that service-user runs as root. 443 becomes possible. And things work again as expected. However, I really did like not running xo-server as root.

                            MathieuRAM 1 Reply Last reply Reply Quote 0
                            • MathieuRAM Offline
                              MathieuRA Vates 🪐 XO Team @probain
                              last edited by

                              @probain I will prepare a branch for a potential fix, so you can still use the 8443 port

                              P 1 Reply Last reply Reply Quote 0
                              • P Offline
                                probain @MathieuRA
                                last edited by

                                @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

                                @probain I will prepare a branch for a potential fix, so you can still use the 8443 port

                                Absolutely fantastic. I will test it as soon as possible to help out however I can.

                                Cheers!

                                MathieuRAM 1 Reply Last reply Reply Quote 0
                                • MathieuRAM Offline
                                  MathieuRA Vates 🪐 XO Team @probain
                                  last edited by

                                  @probain branch mra-fix-secure-port 🤞

                                  P 1 Reply Last reply Reply Quote 0
                                  • P Offline
                                    probain @MathieuRA
                                    last edited by

                                    @MathieuRA said in XO5 breaks after defaulting to XO6 (from source):

                                    @probain branch mra-fix-secure-port 🤞

                                    Success!! Amazing. Thank you so much

                                    1 Reply Last reply Reply Quote 1
                                    • A Offline
                                      acebmxer
                                      last edited by acebmxer

                                      @mathieura

                                      Maybe this has something to do with the script i used to install xo but i can not get v6 to be the default. The links going from v6 to v5 work but just defaults to v5 and accessing. i use port 443 for xo and i do use a proxy.

                                      from the config.toml file

                                      [http.mounts]
164 # Uncomment to setup a default version.
165 # Otherwise, XO5 will be the default for stable channel and XO6 for latest and source
166 # '/' = '../xo-web/dist/'
167 
168 '/' = '../xo-web/dist/'
169 '/v6' = '../../@xen-orchestra/web/dist/'
170 
171 [http.proxies]
172 # [port] is used to reuse the same port declared in [http.listen.0]
173 '/v5/api' = '[protocol]//localhost:[port]/api'
174 '/v5/api/updater' = 'ws://localhost:9001'
175 '/v5/rest' = 'http://localhost:[port]/rest'

                                      Screenshot 2025-12-16 183125.png

                                      this config breaks everything. Nothing works.

                                      [http.mounts]
164 # Uncomment to setup a default version.
165 # Otherwise, XO5 will be the default for stable channel and XO6 for latest and source
166 # '/' = '../xo-web/dist/'
167 
168 '/v6' = '../xo-web/dist/'
169 '/' = '../../@xen-orchestra/web/dist/'
170 
171 [http.proxies]
172 # [port] is used to reuse the same port declared in [http.listen.0]
173 '/v5/api' = '[protocol]//localhost:[port]/api'
174 '/v5/api/updater' = 'ws://localhost:9001'
175 '/v5/rest' = 'http://localhost:[port]/rest'

                                      Original config

                                      [http.mounts]
164 # Uncomment to setup a default version.
165 # Otherwise, XO5 will be the default for stable channel and XO6 for latest and source
166 # '/' = '../xo-web/dist/'
167 
168 '/v5' = '../xo-web/dist/'
169 '/v6' = '../../@xen-orchestra/web/dist/'
170 
171 [http.proxies]
172 # [port] is used to reuse the same port declared in [http.listen.0]
173 '/v5/api' = '[protocol]//localhost:[port]/api'
174 '/v5/api/updater' = 'ws://localhost:9001'
175 '/v5/rest' = 'http://localhost:[port]/rest'
                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post