XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backups failed after changing masters?

    Scheduled Pinned Locked Moved Xen Orchestra
    6 Posts 2 Posters 532 Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • tsettleT Offline
      tsettle
      last edited by

      Had a hardware issue, so I changed master to another server in the pool, evacuated all the VMs and detached the host and removed it from the pool.

      Backup jobs in XO started failing with an error about not being able to connect to the detached host: Error: connect EHOSTUNREACH 172.23.6.101:443.

      Of course it's unreachable. The master was changed and the original server was removed from the pool and taken off line.

      So, what did I miss? I know it's not a lot to go on, but this is all I'm seeing. Not sure where else to look.

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Check on which IP XO is connecting to the pool.

        tsettleT 1 Reply Last reply Reply Quote 0
        • tsettleT Offline
          tsettle
          last edited by

          I inadvertently fixed it, but didn't take any notes.

          Changed master using XO. XO still sees all 3 hosts.

          Detach old master, power down, fixed hardware issue, re-install XCP.

          When I tried to add the host back into the pool, I found that it was still listed on the servers page, but saw no way to re-connect it, so I deleted it.

          Now the entire pool is missing?

          *** Now I know that servers and hosts are two completely different things. Odd nomenclature, but I can deal with it.

          Re-added the new master server, now the pool is back. Added the old master back and joined it to the pool, and all is good.

          I'm sure I missed something when changing masters and removing the old one, but not sure what. I would think that changing masters in XO would change all the things that needed to be changed. Guess not.

          1 Reply Last reply Reply Quote 0
          • tsettleT Offline
            tsettle @olivierlambert
            last edited by

            @olivierlambert

            Well crud, just read the thread from a few days ago about "backups failing if one host is down."

            Reading your responses to that, I now understand what happened. Even after changing masters, XO was still connecting to the old master, which re-directed it to the new master.

            When the old master went offline, XO continued to function, but when the backup job was launched, it tried to connect to the IP of the old master.

            I like that XO will redirect itself when connecting to a host rather than the master, but in the event that XO changes the master, shouldn't it be able to update it's own configuration?

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by olivierlambert

              No it doesn't modify its own "server" record, it will just be redirected by a slave. This redirect info doesn't persist, every time it's reconnecting, there's no recollection on the pool status/members.

              Note this will be "fixed" in our next release; this time, on first valid connection to the pool, XO will store all host IPs of the pool, so it can cycle on recorded IPs in case the main IP doesn't answer at all.

              1 Reply Last reply Reply Quote 3
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                Source: https://github.com/vatesfr/xen-orchestra/issues/5969

                olivierlambert created this issue in vatesfr/xen-orchestra

                closed Save all associated host IPs to a pool and use them as fallback for XAPI connection #5969

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post