XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Error: couldn't instantiate any remote

    Scheduled Pinned Locked Moved Xen Orchestra
    33 Posts 8 Posters 5.1k Views 7 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JamfoFLJ Offline
      JamfoFL @Danp
      last edited by

      @Danp Yuuuup! I'm a flippin' idiot on that front. Thanks, Danp! I don't know why my screen is now reformat in those windows (it hadn't been like this previously, and doesn't look like this for the Delta Backups), but if it works and only looks funny, I can live with that. You can see how the border that usually surrounds all the fields for the scheduler is now "too short" and doesn't show stuff off of the screen.

      Regardless... as I said, I can live with a slightly off-kilter screen. It's the complete loss of the ability to run continuous replications that of the greatest concern. I just want to throw in as much detail as possible.

      Thanks again!!

      JamfoFLJ 1 Reply Last reply Reply Quote 0
      • JamfoFLJ Offline
        JamfoFL @JamfoFL
        last edited by

        Just for completeness... here is the log for the failure of one of the continuous replications jobs that had been working for quite some time until after the update:

        {
          "data": {
            "mode": "delta",
            "reportWhen": "failure"
          },
          "id": "1664289600023",
          "jobId": "ab9ec0fb-43d5-4489-9f1b-b38e39b80e1a",
          "jobName": "VXCTEST-22-03 Replication",
          "message": "backup",
          "scheduleId": "99e24d76-1455-4e18-9f47-bc569cfe74e4",
          "start": 1664289600023,
          "status": "failure",
          "end": 1664289600028,
          "result": {
            "errors": {},
            "message": "couldn't instantiate any remote",
            "name": "Error",
            "stack": "Error: couldn't instantiate any remote\n    at executor (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.mjs:233:27)\n    at file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/jobs/index.mjs:263:30\n    at Jobs._runJob (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/jobs/index.mjs:292:22)\n    at Jobs.runJobSequence (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/jobs/index.mjs:332:7)"
          }
        }
        

        And here is the log for the brand-new continuous replication job I set up just now to see if that would make a difference:

        backupNg.runJob
        {
          "id": "905e5413-f550-40af-afbb-027cfc2133f7",
          "schedule": "23b7fbac-77f0-4a0a-99cf-bb39ddb514ab"
        }
        {
          "errors": {},
          "message": "couldn't instantiate any remote",
          "name": "Error",
          "stack": "Error: couldn't instantiate any remote
            at executor (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/backups-ng/index.mjs:233:27)
            at file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/jobs/index.mjs:263:30
            at Jobs._runJob (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
            at Jobs.runJobSequence (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/jobs/index.mjs:332:7)
            at Api.#callApiMethod (file:///root/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:394:20)"
        }
        
        JamfoFLJ 1 Reply Last reply Reply Quote 0
        • JamfoFLJ Offline
          JamfoFL @JamfoFL
          last edited by

          One last bit of information to add, then I'll shut up and let the professionals go at it!

          As I have been running these jobs successfully for some time before this issue popped up, I did have a number of VMs sitting ready from prior continuous replication jobs. I can confirm that I was able to successfully power on and boot these already existing continuous replication VMs, so whatever the issue is, it is only affecting the ability to complete continuous replication jobs going forward, but has not affected the ability to access and utilize the results of previous jobs.

          DanpD 1 Reply Last reply Reply Quote 0
          • AtaxyaNetworkA Offline
            AtaxyaNetwork Ambassador @JamfoFL
            last edited by

            @JamfoFL Nice catch !
            i can't help anymore since i'm not a great dev, but @florent or @olivierlambert will be better than me to diagnostic these ! ^^

            1 Reply Last reply Reply Quote 0
            • DanpD Offline
              Danp Pro Support Team @JamfoFL
              last edited by Danp

              @JamfoFL What commit is showing under the XO About screen?

              Error is likely related to this commit

              JamfoFLJ 1 Reply Last reply Reply Quote 0
              • JamfoFLJ Offline
                JamfoFL @Danp
                last edited by JamfoFL

                @Danp Current commit is 3d3b6.

                xo-server: 5.103.0
                xo-web: 5.104.0

                Yes... as I had mentioned, everything worked until I updated yesterday at around 3:50 PM. I noticed there were five different commits that were released yesterday, and the one you linked to is one of those. So, as I figured, one of the commits from yesterday "broke" the ability to run continuous replications.

                Now that it looks like you've focused on a cause, I'm sure it's only a matter of time until a new commit is published to fix the issue. Once it's released, I'll get it installed and, hopefully, we can put this behind us.

                Thanks!!

                A 1 Reply Last reply Reply Quote 2
                • T Offline
                  thedobe
                  last edited by

                  Seems that the issue is on the latest version. I had the same issue and rolled back to:

                  xo-server 5.102.3
                  xo-web 5.103.0
                  Xen Orchestra, commit 9d09a

                  Rolling back fixed the issue. Guess I'll wait a bit for the issue to resolve in a further patch.

                  1 Reply Last reply Reply Quote 0
                  • A Offline
                    alcaraz73 @JamfoFL
                    last edited by

                    @JamfoFL Same issue here, it seems to be a version problem.

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by

                      Just to be sure: it's when doing BOTH CR and delta backup?

                      JamfoFLJ 1 Reply Last reply Reply Quote 0
                      • florentF Offline
                        florent Vates 🪐 XO Team
                        last edited by

                        Hello,

                        We could reproduce it with a job with only continuous replication. the PR with the fix is here : https://github.com/vatesfr/xen-orchestra/pull/6437/files

                        Can you confirm it fixes the problem ?

                        Regards
                        Florent

                        fbeauchamp opened this pull request in vatesfr/xen-orchestra

                        closed fix(xo-server): fix running replication job #6437

                        JamfoFLJ screame1S A 3 Replies Last reply Reply Quote 1
                        • JamfoFLJ Offline
                          JamfoFL @olivierlambert
                          last edited by

                          @olivierlambert No... only with CR. Delta backups still work like a champ!

                          1 Reply Last reply Reply Quote 0
                          • JamfoFLJ Offline
                            JamfoFL @florent
                            last edited by

                            @florent Thanks for the update, Florent!

                            Unfortunately, my office is in the direct path of Hurricane Ian, so we are going to be closed for the next day or two, and I won't have an opportunity to try the patch until after we return from the storm!

                            Hopefully Screame1 (who originally submitted the error) will be able to test and verify, and I will do the same once I am back to the office.

                            Thanks again for you post, and I'll be sure to update you as soon as I can!

                            screame1S 1 Reply Last reply Reply Quote 0
                            • screame1S Offline
                              screame1 @florent
                              last edited by screame1

                              @florent @JamfoFL I'd be happy to try to test. I have changed the source code on my server per the change you made.

                              That's where my knowledge turns fuzzy. How do I go about compiling that change and trying to run it?

                              Thanks for the quick response on this issue!

                              AtaxyaNetworkA 1 Reply Last reply Reply Quote 0
                              • screame1S Offline
                                screame1 @JamfoFL
                                last edited by

                                @JamfoFL I've been watching the coverage of Ian from here in Indiana. You and everyone in the path are in my thoughts. Stay safe!

                                1 Reply Last reply Reply Quote 0
                                • AtaxyaNetworkA Offline
                                  AtaxyaNetwork Ambassador @screame1
                                  last edited by

                                  @screame1 You can do something like this:

                                  cd your_folder_where_xo_is
                                  git checkout fix_continuous_replication
                                  git pull -r
                                  yarn
                                  yarn build
                                  

                                  (i didn't use XO source since a long time, so maybe i'm wrong ^^)

                                  screame1S 1 Reply Last reply Reply Quote 2
                                  • screame1S Offline
                                    screame1 @AtaxyaNetwork
                                    last edited by

                                    @AtaxyaNetwork Thanks! I was able to find instructions in another thread, so I was down the path when you replied. I've applied the patch, restarted the server, and it appears to have started the backup correctly. It will obviously take a few minutes to know if the backups work, but this patch has moved me past the error reported.

                                    Thanks again for the efforts @florent @JamfoFL and @AtaxyaNetwork !

                                    1 Reply Last reply Reply Quote 1
                                    • A Offline
                                      alcaraz73 @florent
                                      last edited by

                                      @florent Thanks, patch applied and CR is working now.

                                      JamfoFLJ 1 Reply Last reply Reply Quote 1
                                      • JamfoFLJ Offline
                                        JamfoFL @alcaraz73
                                        last edited by

                                        @alcaraz73 @screame1 @AtaxyaNetwork @florent @Danp @olivierlambert I know I'm late back to the party, but just got back into our office after Hurricane Ian. Thanks to all for your well wishes... I was very fortunate to come out with no damage, and was only without power for about 12 hours.

                                        Now that I'm back, I also applied the fixes here (and updated to the latest commit) and can confirm continuous replication is working like a champ.

                                        Thanks to everyone involved for the help!

                                        1 Reply Last reply Reply Quote 2
                                        • olivierlambertO Offline
                                          olivierlambert Vates 🪐 Co-Founder CEO
                                          last edited by

                                          Oh wow, you were lucky indeed!

                                          Happy to know you are safe 🙂

                                          1 Reply Last reply Reply Quote 1
                                          • First post
                                            Last post