XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    backup mail report says INTERRUPTED but it's not ?

    Scheduled Pinned Locked Moved Backup
    125 Posts 11 Posters 49.8k Views 13 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      MajorP93 @florent
      last edited by

      @florent Thanks for the reply. I will rebuild XO and test with current master.

      M 1 Reply Last reply
      Reply Quote 1
      • M
        MajorP93 @MajorP93
        last edited by

        I can confirm that recent changes in Xen Orchestra improved the situation a lot.
        I am still monitoring and will keep testing before coming to a conclusion but so far the backups did not crash due to Node heap size being exhausted 🤞.

        P 1 Reply Last reply
        Reply Quote 1
        • P
          Pilow @MajorP93
          last edited by

          @MajorP93 what version of XOA are you up to ?
          i'm still in 6.1.2 because of other tiny bugs in new versions

          M 1 Reply Last reply
          Reply Quote 0
          • M
            MajorP93 @Pilow
            last edited by

            @Pilow I am using Xen Orchestra from sources. I think the commits that improved RAM / heap usage are not released for XOA. But I suspect a XOA release coming soon as they already pushed "technical release" commit which is mostly an indicator that a XOA release will follow soon.

            1 Reply Last reply
            Reply Quote 0
            • florentF
              florent Vates 🪐 XO Team
              last edited by florent

              yes, the last changes are released in latest ( 6.3) tomorrow if everything proceed as intended

              mostly : https://github.com/vatesfr/xen-orchestra/pull/9622
              https://github.com/vatesfr/xen-orchestra/pull/9557

              and this one was in 6.2 :
              https://github.com/vatesfr/xen-orchestra/commit/e36e1012e20c9678efa15148179941cb284c39a6

              that's nice to hear

              1 Reply Last reply
              Reply Quote 1
              • MathieuRAM
                MathieuRA Vates 🪐 XO Team
                last edited by

                @pilow @majorp93
                Hi, we've detected a memory leak at the REST API level.
                We pushed a fix on the branch: mra-fix-rest-memory-leak
                Can you test the branch?

                M 2 Replies Last reply
                Reply Quote 0
                • M
                  MajorP93 @MathieuRA
                  last edited by

                  @MathieuRA Sounds promising! Once my backup job finished I will update my XO instance and start testing. Thanks for the heads up and for working on this.
                  Best regards

                  M 1 Reply Last reply
                  Reply Quote 1
                  • M
                    MajorP93 @MajorP93
                    last edited by

                    I updated to branch "mra-fix-rest-memory-leak".
                    I will look at backup job results tomorrow and report back.

                    1 Reply Last reply
                    Reply Quote 2
                    • M MajorP93 referenced this topic on
                    • M
                      MajorP93 @MathieuRA
                      last edited by MajorP93

                      Hello @MathieuRA , @florent

                      Moving the discussion to this thread as it seems to be tackling the same issue: https://xcp-ng.org/forum/topic/11892/xoa-memory-usage/28?_=1776334896974

                      I reported my findings so far over there

                      Best regards

                      1 Reply Last reply
                      Reply Quote 0
                      • T
                        thomassangster
                        last edited by

                        If Xen Orchestra backup report displays an 'INTERRUPTED' status and the web console says clean completions, this often mean there’s a lag in syncing asynchronous metadata. This occurs when the underlying backup job finishes writing VM disk blocks, but the post-backup cleanup hook or secondary storage health-check times out before the notification worker receives confirmation from the remote storage target.

                        The reporting engine inside xo-server dispatches status emails based on strict task completion timeouts. If your backup storage repository takes extra time to coalesce snapshots or update remote NFS metadata indexes, the notification daemon flags the overall job as interrupted even though your virtual machine images remain completely intact.

                        To verify actual mail backup integrity, inspect the raw log stream via xo-cli backup.listLogs or review your xo-server system logs with journalctl -u xo-server. You will likely find that the disk transfer succeeded with zero dropped blocks, while only the final ping event was delayed.

                        When managing critical infrastructure, chasing ambiguous email reports is time wasting. If you require an airtight mail backup tool to archive and organize all infrastructure alert notifications across multiple monitoring systems, Mail Backup X really delivers. It automatically backs up incoming data, encrypts archives locally, and maintains records with high-speed search across all correspondence and attachments.

                        1 Reply Last reply
                        Reply Quote 0
                        • T
                          thomassangster
                          last edited by

                          @Pilow, I saw your update that the affected backups show SUCCESS in XOA, are present on the remote, and have been restored successfully. Given that, your suggestion about the report being generated before the final status is available seems reasonable.

                          Bastien’s delay test looks like a useful way to check that. You mentioned the first day after applying it was clear, although with the issue appearing intermittently, it would probably take several days of reports to know whether it helped.

                          I also saw your later test: one- and two-second delays still produced interrupted reports, while three seconds appeared to work over two days. The team has since merged a five-second delay. If you’re still seeing the original mismatch, can you confirm whether your current XO build includes that change? That would help establish whether there’s another timing issue, or whether the fix simply hasn’t reached the installation yet.

                          For the next occurrence, it may help to keep the email timestamp and the run log for that same job, particularly if only one VM in a larger job is marked interrupted in the report. Posting those alongside the XO version would give the team something specific to compare.

                          On a related note, Mail Backup X could be useful as a separate routine if you also want backups of your email accounts. It’s a convenient tool that works across the spectrum and is compatible with IMAP-based email accounts, offers incremental backups and encryption support.

                          1 Reply Last reply
                          Reply Quote 0

                          Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                          Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                          With your input, this post could be even better 💗

                          Register Login
                          • First post
                            Last post