XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup fails with "Body Timeout Error", "all targets have failed, step: writer.run()"

    Scheduled Pinned Locked Moved Unsolved Backup
    70 Posts 19 Posters 6.2k Views 19 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J Offline
      JB @marcoi
      last edited by

      @marcoi Does your backup include METADATA and XO CONFIG?

      marcoiM 1 Reply Last reply Reply Quote 0
      • marcoiM Offline
        marcoi @JB
        last edited by marcoi

        @JB both
        Also Xen Orchestra, commit be960

        ab69a1e2-43cd-4491-8a2f-b48765886b8c-image.jpeg

        J 2 Replies Last reply Reply Quote 0
        • J Offline
          JB @marcoi
          last edited by

          @marcoi The problem is with the METADATA backup. XO Config is working fine!

          1 Reply Last reply Reply Quote 0
          • J Offline
            JB @marcoi
            last edited by

            @marcoi So the problem is with my infrastructure.

            1 Reply Last reply Reply Quote 0
            • J Offline
              JB @marcoi
              last edited by

              @marcoi I see that you use NFS to store the backup. I use SMB.

              1 Reply Last reply Reply Quote 0
              • J Offline
                JB
                last edited by

                This post is deleted!
                1 Reply Last reply Reply Quote 0
                • J Offline
                  JB
                  last edited by

                  070b68c3-bcf2-40ca-8667-ad41a9dd6f42-image.jpeg

                  1 Reply Last reply Reply Quote 0
                  • marcoiM Offline
                    marcoi
                    last edited by marcoi

                    i switched one of the backups for my extra pool from nfs to smb and its been sitting so i expect it to fail. might be a smb issue. ill update this post with results once it either fails or works

                    2d4d9b1d-9aa5-4f1d-a1c1-d8ea5e079047-image.jpeg

                    i did test smb and it works.

                    back to nfs works
                    b2b807b8-d422-4719-8088-817b08d3c5ad-image.jpeg

                    1 Reply Last reply Reply Quote 0
                    • M Offline
                      MajorP93 @olivierlambert
                      last edited by

                      @olivierlambert said:

                      Hi! Can you try to replicate on the latest commit?

                      Hello!
                      Yes I was able to replicate this issue after updating my Xen Orchestra.

                      188a7d72-1210-4077-b5ef-aa7ebe60ef6d-image.jpeg

                      949e05f6-184d-4f11-9e53-23ddbd662150-image.jpeg

                      The thing is that the "XO-config-and-pool-metadata" does not always fail.
                      It fails from time to time then it looks exactly like shown in the screenshot.
                      Attached logs please find above in my earlier post.

                      This exact job has been working for a long time before.
                      After some Xen Orchestra commit this issue got introduced.

                      I use the same backup remotes for multiple backup jobs.
                      Only the "XO-config-and-pool-metadata" job is failing with this body timeout error.

                      Best regards

                      P 1 Reply Last reply Reply Quote 0
                      • P Offline
                        pierrebrunet Vates 🪐 XO Team @MajorP93
                        last edited by

                        @MajorP93 Hi,
                        Thank you, thank you! I think you found it! It seems to be SMB, @acebmxer can you confirm that you also use smb please?
                        If this is the case, the bad news is I think we will need to update our docs to not advise SMB at all: it was already said for delta but maybe we will have to extend this to all kind of BR 😕

                        M acebmxerA 3 Replies Last reply Reply Quote 0
                        • M Offline
                          MajorP93 @pierrebrunet
                          last edited by

                          @pierrebrunet Hello, thank you for working on this issue.

                          Are you completely sure that this isn't something that got introduced within Xen Orchestra code?

                          Since I started using XCP-ng / Xen Orchestra I have been using these backup repositories via SMB.
                          All backup jobs (including this one) have been working fine for almost a year.

                          This repeated body timeout error is a rather new issue which is why I was wondering if this could have been introduced by XO code changes?

                          I did not switch to a different linux kernel major version or similar. So the SMB client kernel module is at the same version number.

                          Best regards

                          1 Reply Last reply Reply Quote 0
                          • acebmxerA Online
                            acebmxer @pierrebrunet
                            last edited by acebmxer

                            @pierrebrunet

                            I use nfs for my remotes and still get the error occasionally.

                            Edit - Note i am only seeing this error in my work production environment. I am not seeing this issue in my home lab. Both use NFS for remote storage and vm storage.

                            1 Reply Last reply Reply Quote 0
                            • M Offline
                              MajorP93 @pierrebrunet
                              last edited by

                              @pierrebrunet Oh one more thing that I just remembered.
                              If I recall correctly the first time I saw this issue was right after I patched my XCP-ng hosts without updating Xen Orchestra.
                              So maybe this is caused by XAPI code changes?

                              1 Reply Last reply Reply Quote 0

                              Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                              Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                              With your input, this post could be even better 💗

                              Register Login
                              • First post
                                Last post