XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup fails with "Body Timeout Error", "all targets have failed, step: writer.run()"

    Scheduled Pinned Locked Moved Unsolved Backup
    74 Posts 19 Posters 6.3k Views 19 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • marcoiM Offline
      marcoi
      last edited by marcoi

      i switched one of the backups for my extra pool from nfs to smb and its been sitting so i expect it to fail. might be a smb issue. ill update this post with results once it either fails or works

      2d4d9b1d-9aa5-4f1d-a1c1-d8ea5e079047-image.jpeg

      i did test smb and it works.

      back to nfs works
      b2b807b8-d422-4719-8088-817b08d3c5ad-image.jpeg

      1 Reply Last reply Reply Quote 0
      • M Online
        MajorP93 @olivierlambert
        last edited by

        @olivierlambert said:

        Hi! Can you try to replicate on the latest commit?

        Hello!
        Yes I was able to replicate this issue after updating my Xen Orchestra.

        188a7d72-1210-4077-b5ef-aa7ebe60ef6d-image.jpeg

        949e05f6-184d-4f11-9e53-23ddbd662150-image.jpeg

        The thing is that the "XO-config-and-pool-metadata" does not always fail.
        It fails from time to time then it looks exactly like shown in the screenshot.
        Attached logs please find above in my earlier post.

        This exact job has been working for a long time before.
        After some Xen Orchestra commit this issue got introduced.

        I use the same backup remotes for multiple backup jobs.
        Only the "XO-config-and-pool-metadata" job is failing with this body timeout error.

        Best regards

        P 1 Reply Last reply Reply Quote 0
        • P Offline
          pierrebrunet Vates 🪐 XO Team @MajorP93
          last edited by

          @MajorP93 Hi,
          Thank you, thank you! I think you found it! It seems to be SMB, @acebmxer can you confirm that you also use smb please?
          If this is the case, the bad news is I think we will need to update our docs to not advise SMB at all: it was already said for delta but maybe we will have to extend this to all kind of BR 😕

          M acebmxerA 3 Replies Last reply Reply Quote 0
          • M Online
            MajorP93 @pierrebrunet
            last edited by

            @pierrebrunet Hello, thank you for working on this issue.

            Are you completely sure that this isn't something that got introduced within Xen Orchestra code?

            Since I started using XCP-ng / Xen Orchestra I have been using these backup repositories via SMB.
            All backup jobs (including this one) have been working fine for almost a year.

            This repeated body timeout error is a rather new issue which is why I was wondering if this could have been introduced by XO code changes?

            I did not switch to a different linux kernel major version or similar. So the SMB client kernel module is at the same version number.

            Best regards

            1 Reply Last reply Reply Quote 0
            • acebmxerA Offline
              acebmxer @pierrebrunet
              last edited by acebmxer

              @pierrebrunet

              I use nfs for my remotes and still get the error occasionally.

              Edit - Note i am only seeing this error in my work production environment. I am not seeing this issue in my home lab. Both use NFS for remote storage and vm storage.

              P 1 Reply Last reply Reply Quote 0
              • M Online
                MajorP93 @pierrebrunet
                last edited by

                @pierrebrunet Oh one more thing that I just remembered.
                If I recall correctly the first time I saw this issue was right after I patched my XCP-ng hosts without updating Xen Orchestra.
                So maybe this is caused by XAPI code changes?

                1 Reply Last reply Reply Quote 0
                • J Offline
                  JB
                  last edited by

                  6b58102f-353c-4da8-aed5-6817621e13e1-image.jpeg

                  a4477b40-7812-4f27-8711-917c81c818da-image.jpeg

                  commit 20bed

                  SMB

                  1 Reply Last reply Reply Quote 0
                  • P Offline
                    pierrebrunet Vates 🪐 XO Team @acebmxer
                    last edited by

                    @acebmxer ok, too bad... I thought we had it...
                    @majorp93 this can be a lead. I will ask xapi team if they changed something in pool metadata export.

                    acebmxerA 1 Reply Last reply Reply Quote 1
                    • acebmxerA Offline
                      acebmxer @pierrebrunet
                      last edited by

                      @pierrebrunet

                      No worries,
                      Mine seems to be intermittent and restarting that pools tool stack resolves the issue for now.

                      J 1 Reply Last reply Reply Quote 0
                      • J Offline
                        JB @acebmxer
                        last edited by

                        @acebmxer

                        514388c6-4dc9-4602-a72d-8ec157d492ab-image.jpeg

                        Exactly! I restarted the pools tool stack and made 3 backups. If I restart it again, I'll be able to make more backups.

                        1 Reply Last reply Reply Quote 0

                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                        With your input, this post could be even better 💗

                        Register Login
                        • First post
                          Last post