XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Missing vdi which is a base for a delta

    Scheduled Pinned Locked Moved Unsolved Backup
    14 Posts 3 Posters 832 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      blueh2o @poddingue
      last edited by

      @poddingue Yes, thanks, I reviewed that post.

      1 Reply Last reply
      Reply Quote 0
      • B
        blueh2o @acebmxer
        last edited by

        @acebmxer Of course, sorry. We have XO from sources, currently commit 0be90. I'm not aware of any beta updates? I'm attaching the latest log for one of the jobs.
        2026-10-27T06_00_00.022Z - backup NG.json.txt

        poddingueP 1 Reply Last reply
        Reply Quote 0
        • poddingueP
          poddingue Vates 🪐 @blueh2o
          last edited by

          @blueh2o thanks for the log. In that run 16 of the 56 VMs failed this way, and each one names a different missing base, so it doesn't look like one damaged chain to me.
          That's past what I can work out on my own, so @Team-XO-Backend, this one has a version and a full job log attached above now if someone can take a look.

          1 Reply Last reply
          Reply Quote 0
          • B
            blueh2o
            last edited by

            FWIW it seems to be the same VMs and same VDIs each day. I'm doing some additional experimentation and testing.

            poddingueP 1 Reply Last reply
            Reply Quote 1
            • poddingueP
              poddingue Vates 🪐 @blueh2o
              last edited by

              @blueh2o same VMs and same disks every day is useful, I think, since whoever looks at it can then trigger it on demand instead of waiting for a bad night. 🤔

              While you experiment, do the failing VMs have something in common? The same source SR, for example, or a mirror target keeping a different retention than the source.
              I don't know the mirror code well enough to say which of those would matter. 🤷

              B 1 Reply Last reply
              Reply Quote 0
              • B
                blueh2o @poddingue
                last edited by

                @poddingue said:

                While you experiment, do the failing VMs have something in common? The same source SR, for example, or a mirror target keeping a different retention than the source.

                There are two backup jobs that are failing. One from Ark (local) to Synology, and one from Ark to S3. It seems to be the same VMs failing each night, but interestingly, there are some that fail to Synology but succeed to S3 and vice-versa. There are 9 VMs that fail for both. They do not all share the same SR. The backups on Ark are coming from several delta jobs that run daily, weekly, and monthly. Retention are different for all those.

                poddingueP 1 Reply Last reply
                Reply Quote 0
                • poddingueP
                  poddingue Vates 🪐 @blueh2o
                  last edited by poddingue

                  Thanks, that helps. Some VMs fail to the Synology but pass to S3, and some the other way round. 🤔
                  So I think the backups on Ark are probably fine (they pass health checks, as you said at the start) and it has more to do with what each target already holds, but I could be wrong. 🤷
                  I'd also flag the several delta jobs with different retentions all feeding Ark, I just don't know the mirror code well enough to say whether that matters.

                  @Team-XO-Backend, posts 6 and 10 have the log and the pattern if someone gets to this one.

                  1 Reply Last reply
                  Reply Quote 0
                  • B
                    blueh2o
                    last edited by

                    I don't know how, but there do seem to be chains on one remote that have no key. When I delete those, I can get a successful mirror run. I guess I'll just have to throw out all the 'bad' backup chains and start over.

                    poddingueP 1 Reply Last reply
                    Reply Quote 0
                    • poddingueP
                      poddingue Vates 🪐 @blueh2o
                      last edited by

                      Glad you got a clean run out of it, and thanks for coming back to say what worked. If I'm reading the mirror code right, it checks that the parent disk of each delta is already on every target, and stops with this exact error when one isn't there. How a chain ends up without its key in the first place, I can't tell you. Which remote were those chains on, Ark or one of the mirror targets? And before you wipe everything and start over, could you jot down which VMs had them? I think that's what someone would need to chase the cause.

                      B 1 Reply Last reply
                      Reply Quote 0
                      • B
                        blueh2o @poddingue
                        last edited by

                        @poddingue They were all on Ark. VMs:

                        7da38127-0ff5-1f76-d8b6-a5159a37d658
                        7eb23022-4f0b-f73e-b544-c4d860e54453
                        865f4634-76f6-48a3-938e-3adff4fb9e3f
                        9a64cc4e-0b46-d01e-bfcf-3fbedbd7e334
                        a1f1bee1-4076-b37f-006c-630d5c3a9f39
                        a86bb37b-a0ec-3d94-b52d-196da40b317d
                        a925c4c2-b3f3-6cc3-3c40-a836e3a24914
                        b27a40e4-ea4f-7706-fb4e-98551fa85d0d
                        b92636e5-c9f1-47d6-644f-70158dc39f0e
                        bbe64766-97f3-de14-43aa-77867b7f1fb0
                        d1a39422-78d6-56d1-28c3-65299cdc1a5e
                        d77691ef-a507-e120-e2b8-2c1807f0ef99
                        edc46393-eee4-6a39-89c0-0b4486df8879
                        f8ed66aa-e753-1271-84c6-a73819ff25f7
                        fb7fca7d-7355-4a48-f35c-d32c8dfa71ee

                        1 Reply Last reply
                        Reply Quote 0

                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                        With your input, this post could be even better 💗

                        Register Login
                        • First post
                          Last post