XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup fails with "Body Timeout Error", "all targets have failed, step: writer.run()"

    Scheduled Pinned Locked Moved Unsolved Backup
    74 Posts 19 Posters 6.3k Views 19 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J Offline
      JB
      last edited by

      This post is deleted!
      1 Reply Last reply Reply Quote 0
      • J Offline
        JB
        last edited by

        070b68c3-bcf2-40ca-8667-ad41a9dd6f42-image.jpeg

        1 Reply Last reply Reply Quote 0
        • marcoiM Offline
          marcoi
          last edited by marcoi

          i switched one of the backups for my extra pool from nfs to smb and its been sitting so i expect it to fail. might be a smb issue. ill update this post with results once it either fails or works

          2d4d9b1d-9aa5-4f1d-a1c1-d8ea5e079047-image.jpeg

          i did test smb and it works.

          back to nfs works
          b2b807b8-d422-4719-8088-817b08d3c5ad-image.jpeg

          1 Reply Last reply Reply Quote 0
          • M Online
            MajorP93 @olivierlambert
            last edited by

            @olivierlambert said:

            Hi! Can you try to replicate on the latest commit?

            Hello!
            Yes I was able to replicate this issue after updating my Xen Orchestra.

            188a7d72-1210-4077-b5ef-aa7ebe60ef6d-image.jpeg

            949e05f6-184d-4f11-9e53-23ddbd662150-image.jpeg

            The thing is that the "XO-config-and-pool-metadata" does not always fail.
            It fails from time to time then it looks exactly like shown in the screenshot.
            Attached logs please find above in my earlier post.

            This exact job has been working for a long time before.
            After some Xen Orchestra commit this issue got introduced.

            I use the same backup remotes for multiple backup jobs.
            Only the "XO-config-and-pool-metadata" job is failing with this body timeout error.

            Best regards

            P 1 Reply Last reply Reply Quote 0
            • P Offline
              pierrebrunet Vates 🪐 XO Team @MajorP93
              last edited by

              @MajorP93 Hi,
              Thank you, thank you! I think you found it! It seems to be SMB, @acebmxer can you confirm that you also use smb please?
              If this is the case, the bad news is I think we will need to update our docs to not advise SMB at all: it was already said for delta but maybe we will have to extend this to all kind of BR 😕

              M acebmxerA 3 Replies Last reply Reply Quote 0
              • M Online
                MajorP93 @pierrebrunet
                last edited by

                @pierrebrunet Hello, thank you for working on this issue.

                Are you completely sure that this isn't something that got introduced within Xen Orchestra code?

                Since I started using XCP-ng / Xen Orchestra I have been using these backup repositories via SMB.
                All backup jobs (including this one) have been working fine for almost a year.

                This repeated body timeout error is a rather new issue which is why I was wondering if this could have been introduced by XO code changes?

                I did not switch to a different linux kernel major version or similar. So the SMB client kernel module is at the same version number.

                Best regards

                1 Reply Last reply Reply Quote 0
                • acebmxerA Offline
                  acebmxer @pierrebrunet
                  last edited by acebmxer

                  @pierrebrunet

                  I use nfs for my remotes and still get the error occasionally.

                  Edit - Note i am only seeing this error in my work production environment. I am not seeing this issue in my home lab. Both use NFS for remote storage and vm storage.

                  P 1 Reply Last reply Reply Quote 0
                  • M Online
                    MajorP93 @pierrebrunet
                    last edited by

                    @pierrebrunet Oh one more thing that I just remembered.
                    If I recall correctly the first time I saw this issue was right after I patched my XCP-ng hosts without updating Xen Orchestra.
                    So maybe this is caused by XAPI code changes?

                    1 Reply Last reply Reply Quote 0
                    • J Offline
                      JB
                      last edited by

                      6b58102f-353c-4da8-aed5-6817621e13e1-image.jpeg

                      a4477b40-7812-4f27-8711-917c81c818da-image.jpeg

                      commit 20bed

                      SMB

                      1 Reply Last reply Reply Quote 0
                      • P Offline
                        pierrebrunet Vates 🪐 XO Team @acebmxer
                        last edited by

                        @acebmxer ok, too bad... I thought we had it...
                        @majorp93 this can be a lead. I will ask xapi team if they changed something in pool metadata export.

                        acebmxerA 1 Reply Last reply Reply Quote 1
                        • acebmxerA Offline
                          acebmxer @pierrebrunet
                          last edited by

                          @pierrebrunet

                          No worries,
                          Mine seems to be intermittent and restarting that pools tool stack resolves the issue for now.

                          J 1 Reply Last reply Reply Quote 0
                          • J Offline
                            JB @acebmxer
                            last edited by

                            @acebmxer

                            514388c6-4dc9-4602-a72d-8ec157d492ab-image.jpeg

                            Exactly! I restarted the pools tool stack and made 3 backups. If I restart it again, I'll be able to make more backups.

                            1 Reply Last reply Reply Quote 0

                            Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                            Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                            With your input, this post could be even better 💗

                            Register Login
                            • First post
                              Last post