XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Migration woes - SR_BACKEND_FAILURE_78

    Scheduled Pinned Locked Moved XCP-ng
    13 Posts 3 Posters 1.0k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G Offline
      garethw
      last edited by

      Hi All,

      I'm in the process of migrating some VMs from one server to another. After a couple of initial hicups I have moved 3 of the 4 VMs that I need leaving one large (circa 2tb) one to move.

      I was initially having problems because the migration was taking more than 24 hours, so have added "pending_task_timeout = 172800" to a file in /etc/xapi.conf.d and restarted the toolstack.

      I am now getting a SR_BACKEND_FAILURE_78 error. Google hasn't helped much (other than a post on the LTS forums from a couple of years ago that I tried to help answer!) but I wondered if anyone could give me some more details on what that specific failure means. Full log is below.

      Thanks in advance for any assistance.

      vm.migrate
      {
        "vm": "bb7c8076-728d-4a09-38f1-b227c3d16c2e",
        "mapVdisSrs": {
          "0c793c82-7768-4490-9c42-48b2abd8c819": "baf6a596-d7bd-c9fe-1a16-e1296938194a",
          "582c36f0-a599-4546-9011-a545674dc55e": "ee1d59e1-3036-3764-a152-0b99812b81e7"
        },
        "mapVifsNetworks": {
          "a8ce03de-1675-f19a-a94b-1b50822bd3ff": "d121db64-65ab-ce28-ea15-4e80fd60be5d"
        },
        "migrationNetwork": "aa838565-8ec5-c814-4b17-c8c9d715d54a",
        "sr": "ee1d59e1-3036-3764-a152-0b99812b81e7",
        "targetHost": "b9ede076-9a6c-42f4-b08b-50d84952f8fc"
      }
      {
        "code": 21,
        "data": {
          "objectId": "bb7c8076-728d-4a09-38f1-b227c3d16c2e",
          "code": "SR_BACKEND_FAILURE_78"
        },
        "message": "operation failed",
        "name": "XoError",
        "stack": "XoError: operation failed
          at operationFailed (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
          at file:///usr/local/lib/node_modules/xo-server/src/api/vm.mjs:482:15
          at Object.migrate (file:///usr/local/lib/node_modules/xo-server/src/api/vm.mjs:469:3)
          at Api.callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:304:20)"
      }
      
      G 1 Reply Last reply Reply Quote 0
      • G Offline
        garethw @garethw
        last edited by

        Stand by...

        I realised that the failure happened within seconds of me starting the migration so did some more digging.

        It might be because the previous failed migrations left the partially migrated disks on the SR of the destination and I had run out of disk space as a result... I've cleared the failed disks and started again.

        Why do you always notice these things just after you post to a forum...

        1 Reply Last reply Reply Quote 0
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          Because writing things down helps to realize some stuff 😉 It's fine, keep us posted!

          G 1 Reply Last reply Reply Quote 0
          • G Offline
            garethw @olivierlambert
            last edited by

            @olivierlambert it's now 22 hours in with an estimated 13 hours to go... Fingers crossed.

            1 Reply Last reply Reply Quote 1
            • G Offline
              garethw
              last edited by olivierlambert

              Sad times...

              Failed again, rhis probably needs a different post as it's a totally different error but if anyone spots this and has any ideas I'd appreciate the pointers;

              vm.migrate
              {
                "vm": "bb7c8076-728d-4a09-38f1-b227c3d16c2e",
                "mapVdisSrs": {
                  "0c793c82-7768-4490-9c42-48b2abd8c819": "baf6a596-d7bd-c9fe-1a16-e1296938194a",
                  "582c36f0-a599-4546-9011-a545674dc55e": "ee1d59e1-3036-3764-a152-0b99812b81e7"
                },
                "mapVifsNetworks": {
                  "a8ce03de-1675-f19a-a94b-1b50822bd3ff": "d121db64-65ab-ce28-ea15-4e80fd60be5d"
                },
                "migrationNetwork": "aa838565-8ec5-c814-4b17-c8c9d715d54a",
                "sr": "ee1d59e1-3036-3764-a152-0b99812b81e7",
                "targetHost": "b9ede076-9a6c-42f4-b08b-50d84952f8fc"
              }
              {
                "code": 21,
                "data": {
                  "objectId": "bb7c8076-728d-4a09-38f1-b227c3d16c2e",
                  "code": "INTERNAL_ERROR"
                },
                "message": "operation failed",
                "name": "XoError",
                "stack": "XoError: operation failed
                  at operationFailed (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                  at file:///usr/local/lib/node_modules/xo-server/src/api/vm.mjs:482:15
                  at runMicrotasks (<anonymous>)
                  at runNextTicks (internal/process/task_queues.js:60:5)
                  at processImmediate (internal/timers.js:435:9)
                  at process.topLevelDomainCallback (domain.js:147:15)
                  at process.callbackTrampoline (internal/async_hooks.js:132:14)
                  at Object.migrate (file:///usr/local/lib/node_modules/xo-server/src/api/vm.mjs:469:3)
                  at Api.callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:304:20)"
              }
              
              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                At least, it's not related to XO 😄

                Something went wrong during the migration, could be many things. The logs on source and destination hosts might provide details on what went wrong.

                G 1 Reply Last reply Reply Quote 0
                • DanpD Offline
                  Danp Pro Support Team
                  last edited by

                  Maybe you should consider some alternative methods to achieve your migration goal, such as using one of XO's backup methods to replicate the VM on the target server.

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    Warm migration is probably the easiest one 🙂

                    1 Reply Last reply Reply Quote 0
                    • G Offline
                      garethw @olivierlambert
                      last edited by

                      @olivierlambert said in Migration woes - SR_BACKEND_FAILURE_78:

                      logs on source and destination hosts

                      Are we talking, looking through /var/log or something that is in the XO interface somewhere? Going home, hosts, hostname, logs shows "Nothing so far" for both hosts.

                      @Danp said in Migration woes - SR_BACKEND_FAILURE_78:

                      consider some alternative methods to achieve your migration goal

                      @olivierlambert said in Migration woes - SR_BACKEND_FAILURE_78:

                      Warm migration is probably the easiest one

                      Migrate whilst the VM is running? I was trying a migrate whilst shutdown as I assumed that would be easier.

                      Trying one more time.

                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        No, I specifically said warm, not live or hot migration 😉 It's an XO feature allowing to migrate in a kind of "safer" way.

                        G 1 Reply Last reply Reply Quote 0
                        • G Offline
                          garethw @olivierlambert
                          last edited by

                          @olivierlambert Ok, thanks. I'll look up warm migration and go from there.

                          G 1 Reply Last reply Reply Quote 0
                          • G Offline
                            garethw @garethw
                            last edited by

                            I'm struggling to find the warm migration option (looking in advanced) is it on the features that's part of Starter/Enterprise/Premium? I have to admit that this XO is only on the Free account.

                            1 Reply Last reply Reply Quote 0
                            • olivierlambertO Offline
                              olivierlambert Vates 🪐 Co-Founder CEO
                              last edited by

                              You need a fully up to date XO (check that first). If you are on XOA Free, you might need a trial (send me a private chat message with your email registered to this XOA, so I can unlock you the free trial). Alternatively, you can also use XO from the sources 🙂

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post