XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Password in plain text required for the connection between XOCE and XO-SERVER. Is this normal ?

    Scheduled Pinned Locked Moved Unsolved Management
    5 Posts 3 Posters 52 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F Offline
      FACE_DVLP
      last edited by FACE_DVLP

      Hi,

      I’ve updated XOCE 6 but I’m getting the error message “Unable to connect to the XO server” in the GUI.

      I ran the command redis-cli followed by the command
      GET xo:server:7315c3b1-d9c6-4cba-a207-e80ee4070e8f

      Command output with plain text password:
      ‘{\’allowUnauthorized\‘:\’true\‘,\’enabled\‘:\’true\‘,\’host\‘:\’192.168.100.100\‘,\’ label\‘:\’sxcp01\‘,\’password\‘:\’E😧 😧 😧 😧 😧 😧 😧 ;\‘,\’username\‘:\’root\‘,\’poolNameDescription\‘:\’XCP-Ng Server S01\‘,\’poolMembersAddresses\‘:\’[\" 192.168.100.100\\‘]\’,\‘error\’:\‘{\\’poolId\\‘:\\’75d19959-fe1f-ed43-f731-77d4ac6d5ec6\\‘,\\’connectedServerId\\‘:\\’ 1f612313-c8ec-4767-97c1-03347d510759\\‘,\\’connectingServerId\\‘:\\’7315c3b1-d9c6-4cba-a207-e80ee4070e8f\\‘}\’}"

      Is it normal ?

      Second problem
      I checked the log files
      root@MLMS06:/home/administrateur# sudo journalctl -u xo-server -f -n 50
      août 30 06:06:35 MLMS06 xo-server[812077]: }
      août 30 06:06:35 MLMS06 xo-server[812077]: 2026-08-30T04:06:35.273Z xo:xo-mixins:xen-servers INFO auto-reconnect stopped {
      août 30 06:06:35 MLMS06 xo-server[812077]: serverId: '7315c3b1-d9c6-4cba-a207-e80ee4070e8f',
      août 30 06:06:35 MLMS06 xo-server[812077]: outcome: 'permanent error'
      août 30 06:06:35 MLMS06 xo-server[812077]: }
      août 30 06:06:36 MLMS06 xo-server[812077]: 2026-08-30T04:06:36.412Z xo:xo-mixins:xen-servers WARN auto-reconnect aborted: permanent error {
      août 30 06:06:36 MLMS06 xo-server[812077]: serverId: '7315c3b1-d9c6-4cba-a207-e80ee4070e8f',
      août 30 06:06:36 MLMS06 xo-server[812077]: error: PoolAlreadyConnected: this pool is already connected
      août 30 06:06:36 MLMS06 xo-server[812077]: at XenServers._connectXenServer (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/xen-servers.mjs:473:15)
      août 30 06:06:36 MLMS06 xo-server[812077]: at XenServers.connectXenServer (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/xen-servers.mjs:423:7)
      août 30 06:06:36 MLMS06 xo-server[812077]: at autoReconnect (file:///opt/xen-orchestra/packages/xo-server/src/_xenServerAutoReconnect.mjs:49:7) {
      août 30 06:06:36 MLMS06 xo-server[812077]: poolId: '75d19959-fe1f-ed43-f731-77d4ac6d5ec6',
      août 30 06:06:36 MLMS06 xo-server[812077]: connectedServerId: '1f612313-c8ec-4767-97c1-03347d510759',
      août 30 06:06:36 MLMS06 xo-server[812077]: connectingServerId: '7315c3b1-d9c6-4cba-a207-e80ee4070e8f'
      août 30 06:06:36 MLMS06 xo-server[812077]: }
      août 30 06:06:36 MLMS06 xo-server[812077]: }
      août 30 06:06:36 MLMS06 xo-server[812077]: 2026-08-30T04:06:36.413Z xo:xo-mixins:xen-servers INFO auto-reconnect stopped {
      août 30 06:06:36 MLMS06 xo-server[812077]: serverId: '7315c3b1-d9c6-4cba-a207-e80ee4070e8f',
      août 30 06:06:36 MLMS06 xo-server[812077]: outcome: 'permanent error'
      août 30 06:06:36 MLMS06 xo-server[812077]: }
      août 30 06:06:37 MLMS06 xo-server[812077]: 2026-08-30T04:06:37.537Z xo:xo-mixins:xen-servers WARN auto-reconnect aborted: permanent error {
      août 30 06:06:37 MLMS06 xo-server[812077]: serverId: '7315c3b1-d9c6-4cba-a207-e80ee4070e8f',
      août 30 06:06:37 MLMS06 xo-server[812077]: error: PoolAlreadyConnected: this pool is already connected
      août 30 06:06:37 MLMS06 xo-server[812077]: at XenServers._connectXenServer (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/xen-servers.mjs:473:15)
      août 30 06:06:37 MLMS06 xo-server[812077]: at XenServers.connectXenServer (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/xen-servers.mjs:423:7)
      août 30 06:06:37 MLMS06 xo-server[812077]: at autoReconnect (file:///opt/xen-orchestra/packages/xo-server/src/_xenServerAutoReconnect.mjs:49:7) {
      août 30 06:06:37 MLMS06 xo-server[812077]: poolId: '75d19959-fe1f-ed43-f731-77d4ac6d5ec6',
      août 30 06:06:37 MLMS06 xo-server[812077]: connectedServerId: '1f612313-c8ec-4767-97c1-03347d510759',
      août 30 06:06:37 MLMS06 xo-server[812077]: connectingServerId: '7315c3b1-d9c6-4cba-a207-e80ee4070e8f'

      I deleted the entry using the command
      DEL xo:server:7315c3b1-d9c6-4cba-a207-e80ee4070e8f

      The xo-server service is running and I restarted it

      but I’m still getting the error message
      “Unable to connect to the XO server” in the GUI.

      Best regards

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Hi!

        1. It is not the XOCE to xo-server link, it is the credential xo-server uses to log into XAPI on your host.
        2. It has to be reversible, not hashed, because xo-server replays it to XAPI on every connect and auto-reconnect. So yes, plaintext by default is expected. The security perimeter here is Redis itself (bound to localhost or a unix socket, never exposed) and root access on the XO VM.

        That said, since XO 6.5 there is an opt-in encryption at rest. In your xo-server config:

        [redis]
        encryptCredentialDatabase = true
        

        On the next start, every record is encrypted with AES-256-GCM (values get an enc: prefix) and index keys become HMAC-SHA256 blind indexes, so hostnames and emails are not readable either. The key is split in two halves, one in XenStore (vm-data/xo-encryption-key) and one in /var/lib/xo-server/data/xo-encryption-key.

        Caveats for a source install:

        • XO must run as a VM on XCP-ng/XenServer with xen-tools, and xo-server needs access to the xenstored socket (root, typically).
        • Config export then requires a passphrase.
        • If you lose one key half while encryption is on, do not restart: XO would regenerate both halves and the existing records would become undecryptable.

        Full doc: https://docs.xen-orchestra.com/credential-encryption

        About the PoolAlreadyConnected part, that is a separate issue. It means two server entries resolve to the same pool, typically a slave host registered next to the master, or the pool re-added after a master change. Removing the entry with DEL xo:server:<id> is not the right fix: it drops the hash but leaves the id in xo:server_ids and in the host index. Remove the server from the UI instead. If the error persists, it is the other entry pointing to that same pool that needs to go.

        acebmxerA 1 Reply Last reply Reply Quote 1
        • poddingueP poddingue marked this topic as a question
        • acebmxerA Offline
          acebmxer @olivierlambert
          last edited by acebmxer

          @olivierlambert

          I ran your statement though Claude on my Install script and this is what it came back with.

          Credential Encryption Review — install-xen-orchestra.sh

          Notes from reviewing our installer's handling of ENCRYPT_REDIS_CREDENTIALS, and what (if anything) belongs in the Vates forum thread.


          1. What's already there

          ENCRYPT_REDIS_CREDENTIALS is a first-class config key:

          Behavior Location
          Defaulted and validated as a boolean install-xen-orchestra.sh:419, :479-481
          Auto-migrated into existing xo-config.cfg files as part of the v1→v2 schema bump install-xen-orchestra.sh:547-565
          Emits encryptCredentialDatabase = true under [redis] in the generated config.toml install-xen-orchestra.sh:1713-1718
          Preflight warns when systemd-detect-virt isn't xen install-xen-orchestra.sh:1623-1642
          Token flush treats non-JSON Redis values as possibly-encrypted and preserves them rather than misclassifying them install-xen-orchestra.sh:1086-1100

          The part that isn't in Vates' doc at all

          With a non-root SERVICE_USER, /dev/xen/xenbus is root-only 0600, so xo-server can't reach XenStore and drops into degraded mode.

          configure_xenstore_access() handles this: it creates a xenstore group, adds the service user, installs a udev rule, and chgrp/chmods the live device — so it works without a reboot and without running XO as root.

          That's a real gap-fill.


          2. Documentation gaps

          The docs (README.md:345-353, sample-xo-config.cfg:174-195) explain what the flag does, the XCP-ng-only constraint, and the root/non-root XenStore story. What's missing:

          2.1 The "why"

          The actual question in that thread. Nothing anywhere says the credential must be reversible because xo-server replays it to XAPI on every connect — so plaintext-by-default is expected behavior, not a bug, and the real perimeter is localhost-bound Redis plus root on the VM.

          Someone reading the README sees a security-flavored flag defaulting to false with no explanation of why that default is defensible.

          2.2 Remote Redis is offered with no warning

          sample-xo-config.cfg:167-172 gives this as an example:

          REDIS_URI=redis://redis.company.lan:6379/42
          

          That sends XAPI root credentials over the network in cleartext, to a Redis that is by definition not localhost-bound. There's no note about it, and the installer never checks or hardens bind/protected-mode — setup_redis() only starts the service and pings it.

          2.3 The key-loss caveat is absent, and backups don't cover the key

          This is the sharpest one.

          "If you lose one half while encryption is on, do not restart" appears nowhere. Worse:

          • create_backup() copies only $INSTALL_DIR
          • /var/lib/xo-server/data/xo-encryption-key is never captured
          • neither is the XenStore half

          So --backup/--restore is not a recovery path for an encrypted database, and nothing says so.

          2.4 Config export requires a passphrase once encryption is on

          Not mentioned.

          2.5 The preflight checks virt type, not tooling

          It never verifies xenstore-read/xenstore-write exist (i.e. that xen-guest-utilities is installed). A Xen guest without them passes the check and then fails at xo-server startup. The binaries are named only in comments and generated config text (install-xen-orchestra.sh:563, :1716).

          2.6 --uninstall destroys the on-disk key half silently

          install-xen-orchestra.sh:6004-6008 does rm -rf /var/lib/xo-server with no encryption-specific note that this permanently destroys the on-disk key half. Defensible for a full uninstall, but worth stating.


          3. Should any of this go to Vates?

          Almost all of it is on us

          Their page already covers the two caveats flagged as undocumented in our repo — verbatim:

          If either key half is lost while encryption is enabled, do not restart xo-server: at startup, XO treats a missing key half as a fresh setup, generates two new halves (overwriting the one that survived) and re-runs the encryption migration, which skips records that are already encrypted. Those records become permanently undecryptable.

          When credential encryption is enabled, exporting the XO configuration requires a passphrase.

          So gaps 2.1, 2.3, 2.4, 2.5, 2.6 aren't Vates' problem — they're ours, for surfacing an option whose failure modes we don't repeat locally.

          Gap 2.3 in particular is squarely ours: we ship a --backup/--restore that never captures /var/lib/xo-server/data/xo-encryption-key, so a restore silently produces exactly the "permanently undecryptable" state their doc warns about. That's our bug, not a doc issue.

          One thing is genuinely worth relaying

          Their entire treatment of non-root is a single clause:

          This requires running xo-server as root or granting it access to the xenstored socket.

          The "or" branch is unspecified — no device path, no group, no udev rule. That matters more than it looks, because their own install-from-sources guide recommends running non-root. Follow both docs and you get a broken install with no explanation.

          We had to work out:

          • /dev/xen/xenbus being root:root 0600
          • the xen!xenbus udev sysname encoding
          • the group + rule to fix it (install-xen-orchestra.sh:1861-1923)

          That's the concrete content their clause is missing, and it's reusable by anyone doing a source install.

          That thread is a good place for it. The answer you quoted says "xo-server needs access to the xenstored socket (root, typically)" — the parenthetical is the exact spot the detail belongs, so a reply reads as filling in a known gap rather than a drive-by critique.


          4. Verify before posting

          Our claim that a non-root xo-server without XenStore access "enters degraded mode and logins are rejected" (install-xen-orchestra.sh:1854-1855) traces only to our own script comments — not to XO source or an observed run.

          If you're asserting a specific failure mode to the people who wrote the code: confirm it first, or soften it to what you actually saw. If the startup error doesn't name XenStore as the cause, that's a second, separately useful report for them.


          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            I have no issue using Claude, but I would prefer your own conclusions/recommendations in the end and less text 😉

            acebmxerA 1 Reply Last reply Reply Quote 0
            • acebmxerA Offline
              acebmxer @olivierlambert
              last edited by acebmxer

              @olivierlambert said:

              I have no issue using Claude, but I would prefer your own conclusions/recommendations in the end and less text 😉

              Sory its early morning. I wanted to show the statement what it said about what was missing on Vates side along what was missing from my script. Also to verify if Claude was correct or not. I have been battle with that alot lately.

              1 Reply Last reply Reply Quote 0

              Hello! It looks like you're interested in this conversation, but you don't have an account yet.

              Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

              With your input, this post could be even better 💗

              Register Login
              • First post
                Last post