XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Our future backup code: test it!

    Scheduled Pinned Locked Moved Backup
    121 Posts 10 Posters 4.6k Views 11 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Offline
      manilx @florent
      last edited by

      @florent Pls advise here when it's fixed, so that I can retry. I will hold back until then.
      Backups are essential.

      florentF 1 Reply Last reply Reply Quote 1
      • florentF Offline
        florent Vates 🪐 XO Team @manilx
        last edited by florent

        @manilx the PR is in review, it should be merged in a few hours

        thank you for helping us improve the quality

        M 1 Reply Last reply Reply Quote 0
        • M Offline
          manilx @florent
          last edited by

          @florent Will update and try at the end of today then (GMT)

          florentF 1 Reply Last reply Reply Quote 0
          • florentF Offline
            florent Vates 🪐 XO Team @manilx
            last edited by

            @manilx this is merged

            M 2 Replies Last reply Reply Quote 2
            • M Offline
              manilx @florent
              last edited by

              @florent Updated.

              Did a backup job: OK
              Did the mirror job: OK

              Great!

              1 Reply Last reply Reply Quote 2
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                \o/

                1 Reply Last reply Reply Quote 0
                • M Offline
                  manilx @florent
                  last edited by

                  This post is deleted!
                  1 Reply Last reply Reply Quote 0
                  • Tristis OrisT Offline
                    Tristis Oris Top contributor
                    last edited by Tristis Oris

                    CR still failed with
                    Error: HANDLE_INVALID(SR, OpaqueRef:cc116d5f-0a91-b487-b43d-4ebd98495fcd)
                    Error: Error in the source generator Cannot read properties of undefined (reading 'write')
                    i found that old CR copies dissapeared from all hosts.

                    florentF 1 Reply Last reply Reply Quote 0
                    • florentF Offline
                      florent Vates 🪐 XO Team @Tristis Oris
                      last edited by

                      @Tristis-Oris I still don't have the solution I am working on it .

                      M 1 Reply Last reply Reply Quote 0
                      • M Offline
                        manilx @florent
                        last edited by

                        @florent Getting errors again:

                        ScreenShot 2025-05-25 at 10.06.44.png

                        {
                          "data": {
                            "mode": "delta",
                            "reportWhen": "failure"
                          },
                          "id": "1748134829731",
                          "jobId": "b7215ad7-5616-4620-a6fb-1c5b5b1bd3c6",
                          "jobName": "Mirror Mani",
                          "message": "backup",
                          "scheduleId": "9e8eeb18-65bb-4e12-b1a1-4f7c0490565c",
                          "start": 1748134829731,
                          "status": "failure",
                          "infos": [
                            {
                              "data": {
                                "vms": [
                                  "0c8fd092-ab36-478e-df85-16b1a01e1ca8",
                                  "19242523-e70b-bf39-820d-5e322435012b",
                                  "1a65641e-0976-07a8-30f8-3cbd2e4bc63d",
                                  "1c596fc8-26a8-7d95-d004-1e694392fc72",
                                  "1d7614fc-65f7-4e1c-71f9-da638efd8b4d",
                                  "284b64bc-8684-eb3c-b32e-93e455f5854b",
                                  "294b867a-e241-fe7a-10b0-aef1de6f693d",
                                  "341a6155-908b-66ba-76ee-56a1d5163af1",
                                  "43a4d905-7d13-85b8-bed3-f6b805ff26ac",
                                  "48ca6a7e-7acf-74fd-4ad9-714315563652",
                                  "4d8cad25-b6e5-4d8f-5226-b5908b21a1b6",
                                  "52e64134-62e3-9682-4e3f-296a1198db4d",
                                  "58af91d1-94cf-cbe5-88cf-560e652741bf",
                                  "592e229c-76d7-c0f2-426c-4ef9e80810d0",
                                  "5f49e0a4-9397-3da5-e44a-0ab3632cd9f1",
                                  "6c9f0275-07bd-a124-e187-47f8389ed93c",
                                  "8d573273-49b1-6b84-73a6-42c3346add52",
                                  "8e76ba5c-e190-753b-75d6-e67feb1dd85c",
                                  "9017c283-51d0-8d22-d13b-328cc9fe4ff7",
                                  "903d61e0-c07f-8163-47ae-8e212147a353",
                                  "931aaab6-5f7a-d0ef-26a1-1f54063d0941",
                                  "9e0c8a72-d0fd-6632-6d57-94224149af05",
                                  "9f8088b6-9a5d-0ec5-24d5-118c23100a4b",
                                  "a060d911-bf2b-61c5-56bf-11944af25c4a",
                                  "a2562d66-00c8-3955-ab43-6a5584044b98",
                                  "ac14b70f-9958-7add-f833-64486feba4e7",
                                  "b02697d4-23b5-598f-4def-d6b197bc1085",
                                  "b5d74e0b-388c-019a-6994-e174c9ca7a51",
                                  "bcb4bcfc-c77b-2068-4bec-2826e8ece6be",
                                  "c21e8541-c5ea-840e-31aa-095789600230",
                                  "c49a5398-38db-4fcd-481b-30181a5503cc",
                                  "c5fbe7d1-9f9c-0167-5ed2-1dff1104ef4b",
                                  "cc6b00b9-16ec-1990-e44a-503b6e3921c3",
                                  "d7964354-aff6-c7e1-0067-52e8e866a965",
                                  "ef53c584-f6c2-3aa3-b4c0-802fb20683d9",
                                  "f262e69e-f345-bc52-c2dc-60eea7b84c9a",
                                  "fab6b6c0-7ce7-91da-fe60-acbcea4ca5be",
                                  "fe2dd0ea-f063-9aba-30d5-d2de508b091f"
                                ]
                              },
                              "message": "vms"
                            }
                          ],
                          "tasks": [
                            {
                              "data": {
                                "type": "VM",
                                "id": "19242523-e70b-bf39-820d-5e322435012b"
                              },
                              "id": "1748134830645",
                              "message": "backup VM",
                              "start": 1748134830645,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748134830822",
                                  "message": "clean-vm",
                                  "start": 1748134830822,
                                  "status": "success",
                                  "end": 1748134831309,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748134831397",
                                  "message": "export",
                                  "start": 1748134831397,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748134831413",
                                      "message": "transfer",
                                      "start": 1748134831413,
                                      "status": "success",
                                      "end": 1748134937238,
                                      "result": {
                                        "size": 710934528
                                      }
                                    },
                                    {
                                      "id": "1748134947620",
                                      "message": "clean-vm",
                                      "start": 1748134947620,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/19242523-e70b-bf39-820d-5e322435012b/20250524T220611Z.json",
                                            "actual": 710934528,
                                            "expected": 713224192
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748134949566,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748134965928
                                }
                              ],
                              "end": 1748134965928
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "0c8fd092-ab36-478e-df85-16b1a01e1ca8"
                              },
                              "id": "1748134830649",
                              "message": "backup VM",
                              "start": 1748134830649,
                              "status": "interrupted",
                              "tasks": [
                                {
                                  "id": "1748134830750",
                                  "message": "clean-vm",
                                  "start": 1748134830750,
                                  "status": "success",
                                  "warnings": [
                                    {
                                      "data": {
                                        "path": "/xo-vm-backups/0c8fd092-ab36-478e-df85-16b1a01e1ca8/20250522T161036Z.json",
                                        "actual": 71360596992,
                                        "expected": 71360597504
                                      },
                                      "message": "cleanVm: incorrect backup size in metadata"
                                    }
                                  ],
                                  "end": 1748134831421,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748134834838",
                                  "message": "export",
                                  "start": 1748134834838,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748134834899",
                                      "message": "transfer",
                                      "start": 1748134834899,
                                      "status": "success",
                                      "end": 1748135312431,
                                      "result": {
                                        "size": 4857004032
                                      }
                                    },
                                    {
                                      "id": "1748135312559",
                                      "message": "clean-vm",
                                      "start": 1748135312559,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "parent": "/xo-vm-backups/0c8fd092-ab36-478e-df85-16b1a01e1ca8/vdis/b7215ad7-5616-4620-a6fb-1c5b5b1bd3c6/5cb71c46-ddf5-4058-b29c-3bb436816d45",
                                            "child": "/xo-vm-backups/0c8fd092-ab36-478e-df85-16b1a01e1ca8/vdis/b7215ad7-5616-4620-a6fb-1c5b5b1bd3c6/5cb71c46-ddf5-4058-b29c-3bb436816d45/20250524T220111Z.vhd"
                                          },
                                          "message": "parent VHD is missing"
                                        },
                                        {
                                          "data": {
                                            "backup": "/xo-vm-backups/0c8fd092-ab36-478e-df85-16b1a01e1ca8/20250524T220111Z.json",
                                            "missingVhds": [
                                              "/xo-vm-backups/0c8fd092-ab36-478e-df85-16b1a01e1ca8/vdis/b7215ad7-5616-4620-a6fb-1c5b5b1bd3c6/5cb71c46-ddf5-4058-b29c-3bb436816d45/20250524T220111Z.vhd"
                                            ]
                                          },
                                          "message": "some VHDs linked to the backup are missing"
                                        }
                                      ],
                                      "end": 1748135312819,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1748135312823
                                }
                              ],
                              "warnings": [
                                {
                                  "data": {
                                    "attempt": 1,
                                    "error": "Expected values to be strictly equal:\n\n0 !== 512\n"
                                  },
                                  "message": "Retry the VM mirror backup due to an error"
                                }
                              ]
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "1c596fc8-26a8-7d95-d004-1e694392fc72"
                              },
                              "id": "1748134966251",
                              "message": "backup VM",
                              "start": 1748134966251,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748134968074",
                                  "message": "clean-vm",
                                  "start": 1748134968074,
                                  "status": "success",
                                  "end": 1748134968813,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748134971622",
                                  "message": "export",
                                  "start": 1748134971622,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748134971997",
                                      "message": "transfer",
                                      "start": 1748134971997,
                                      "status": "success",
                                      "end": 1748135312426,
                                      "result": {
                                        "size": 3261071360
                                      }
                                    },
                                    {
                                      "id": "1748135314472",
                                      "message": "clean-vm",
                                      "start": 1748135314472,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/1c596fc8-26a8-7d95-d004-1e694392fc72/20250524T222347Z.json",
                                            "actual": 3261071360,
                                            "expected": 3264098816
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748135314652,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748135314707
                                }
                              ],
                              "end": 1748135314707
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "294b867a-e241-fe7a-10b0-aef1de6f693d"
                              },
                              "id": "1748135312961",
                              "message": "backup VM",
                              "start": 1748135312961,
                              "status": "interrupted",
                              "tasks": [
                                {
                                  "id": "1748135313037",
                                  "message": "clean-vm",
                                  "start": 1748135313037,
                                  "status": "success",
                                  "warnings": [
                                    {
                                      "data": {
                                        "path": "/xo-vm-backups/294b867a-e241-fe7a-10b0-aef1de6f693d/20250522T161022Z.json",
                                        "actual": 11216235520,
                                        "expected": 11216236032
                                      },
                                      "message": "cleanVm: incorrect backup size in metadata"
                                    }
                                  ],
                                  "end": 1748135313727,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135313903",
                                  "message": "export",
                                  "start": 1748135313903,
                                  "status": "failure",
                                  "tasks": [
                                    {
                                      "id": "1748135313921",
                                      "message": "transfer",
                                      "start": 1748135313921,
                                      "status": "failure",
                                      "end": 1748135318829,
                                      "result": {
                                        "errno": -9,
                                        "code": "EBADF",
                                        "syscall": "read",
                                        "message": "EBADF: bad file descriptor, read",
                                        "name": "Error",
                                        "stack": "Error: EBADF: bad file descriptor, read\nFrom:\n    at NfsHandler.addSyncStackTrace (/opt/xen-orchestra/@xen-orchestra/fs/dist/local.js:21:26)\n    at NfsHandler._read (/opt/xen-orchestra/@xen-orchestra/fs/dist/local.js:160:43)\n    at NfsHandler.read (/opt/xen-orchestra/@xen-orchestra/fs/dist/abstract.js:259:17)\n    at NfsHandler.read (/opt/xen-orchestra/node_modules/limit-concurrency-decorator/index.js:97:24)\n    at VhdFile._read (/opt/xen-orchestra/packages/vhd-lib/Vhd/VhdFile.js:126:55)\n    at VhdFile.readBlock (/opt/xen-orchestra/packages/vhd-lib/Vhd/VhdFile.js:212:17)\n    at RemoteVhd.readBlock (file:///opt/xen-orchestra/@xen-orchestra/backups/disks/RemoteVhd.mjs:144:38)\n    at RemoteVhd.buildDiskBlockGenerator (file:///opt/xen-orchestra/@xen-orchestra/disk-transform/dist/Disk.mjs:67:28)"
                                      }
                                    },
                                    {
                                      "id": "1748135318836",
                                      "message": "clean-vm",
                                      "start": 1748135318836,
                                      "status": "success",
                                      "end": 1748135318927,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1748135318948
                                }
                              ],
                              "warnings": [
                                {
                                  "data": {
                                    "attempt": 1,
                                    "error": "ENOENT: no such file or directory, open '/run/xo-server/mounts/337848e1-b52b-4e08-9bf0-ffce54de4276/xo-vm-backups/294b867a-e241-fe7a-10b0-aef1de6f693d/vdis/b7215ad7-5616-4620-a6fb-1c5b5b1bd3c6/c87b9871-9ce2-4b39-9fec-9dea2fac3220/20250524T220036Z.vhd'"
                                  },
                                  "message": "Retry the VM mirror backup due to an error"
                                }
                              ]
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "43a4d905-7d13-85b8-bed3-f6b805ff26ac"
                              },
                              "id": "1748135318587",
                              "message": "backup VM",
                              "start": 1748135318587,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135318860",
                                  "message": "clean-vm",
                                  "start": 1748135318860,
                                  "status": "success",
                                  "end": 1748135319739,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135319979",
                                  "message": "export",
                                  "start": 1748135319979,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748135319993",
                                      "message": "transfer",
                                      "start": 1748135319993,
                                      "status": "success",
                                      "end": 1748135458653,
                                      "result": {
                                        "size": 870318080
                                      }
                                    },
                                    {
                                      "id": "1748135473880",
                                      "message": "clean-vm",
                                      "start": 1748135473880,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/43a4d905-7d13-85b8-bed3-f6b805ff26ac/20250524T220144Z.json",
                                            "actual": 870318080,
                                            "expected": 872691712
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748135483581,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748135487189
                                }
                              ],
                              "end": 1748135487190
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "58af91d1-94cf-cbe5-88cf-560e652741bf"
                              },
                              "id": "1748135319145",
                              "message": "backup VM",
                              "start": 1748135319145,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135319356",
                                  "message": "clean-vm",
                                  "start": 1748135319356,
                                  "status": "success",
                                  "end": 1748135320011,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135331775",
                                  "message": "export",
                                  "start": 1748135331775,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748135331781",
                                      "message": "transfer",
                                      "start": 1748135331781,
                                      "status": "success",
                                      "end": 1748135561792,
                                      "result": {
                                        "size": 2801795072
                                      }
                                    },
                                    {
                                      "id": "1748135562458",
                                      "message": "clean-vm",
                                      "start": 1748135562458,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/58af91d1-94cf-cbe5-88cf-560e652741bf/20250524T221027Z.json",
                                            "actual": 2801795072,
                                            "expected": 2804644352
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748135562570,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748135562664
                                }
                              ],
                              "end": 1748135562665
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "6c9f0275-07bd-a124-e187-47f8389ed93c"
                              },
                              "id": "1748135487646",
                              "message": "backup VM",
                              "start": 1748135487646,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135488697",
                                  "message": "clean-vm",
                                  "start": 1748135488697,
                                  "status": "success",
                                  "end": 1748135499685,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "id": "1748135500529:0",
                                  "message": "clean-vm",
                                  "start": 1748135500529,
                                  "status": "success",
                                  "end": 1748135506654,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "end": 1748135507365
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "931aaab6-5f7a-d0ef-26a1-1f54063d0941"
                              },
                              "id": "1748135507681",
                              "message": "backup VM",
                              "start": 1748135507681,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135509790",
                                  "message": "clean-vm",
                                  "start": 1748135509790,
                                  "status": "success",
                                  "end": 1748135517538,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "id": "1748135518256",
                                  "message": "clean-vm",
                                  "start": 1748135518256,
                                  "status": "success",
                                  "end": 1748135523041,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "end": 1748135523539
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "a060d911-bf2b-61c5-56bf-11944af25c4a"
                              },
                              "id": "1748135525305",
                              "message": "backup VM",
                              "start": 1748135525305,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135527194",
                                  "message": "clean-vm",
                                  "start": 1748135527194,
                                  "status": "success",
                                  "end": 1748135534840,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "id": "1748135535382",
                                  "message": "clean-vm",
                                  "start": 1748135535382,
                                  "status": "success",
                                  "end": 1748135538329,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "end": 1748135538791
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "a2562d66-00c8-3955-ab43-6a5584044b98"
                              },
                              "id": "1748135539187",
                              "message": "backup VM",
                              "start": 1748135539187,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135561164",
                                  "message": "clean-vm",
                                  "start": 1748135561164,
                                  "status": "success",
                                  "end": 1748135562180,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135562446",
                                  "message": "export",
                                  "start": 1748135562446,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748135562453",
                                      "message": "transfer",
                                      "start": 1748135562453,
                                      "status": "success",
                                      "end": 1748135562527,
                                      "result": {
                                        "size": 0
                                      }
                                    },
                                    {
                                      "id": "1748135562801",
                                      "message": "clean-vm",
                                      "start": 1748135562801,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/a2562d66-00c8-3955-ab43-6a5584044b98/20250524T220635Z.json",
                                            "actual": 0,
                                            "expected": 134144
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748135563247,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748135563304
                                }
                              ],
                              "end": 1748135563304
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "b02697d4-23b5-598f-4def-d6b197bc1085"
                              },
                              "id": "1748135562737",
                              "message": "backup VM",
                              "start": 1748135562737,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135562853",
                                  "message": "clean-vm",
                                  "start": 1748135562853,
                                  "status": "success",
                                  "warnings": [
                                    {
                                      "data": {
                                        "path": "/xo-vm-backups/b02697d4-23b5-598f-4def-d6b197bc1085/20250522T161026Z.json",
                                        "actual": 11740651520,
                                        "expected": 11740652032
                                      },
                                      "message": "cleanVm: incorrect backup size in metadata"
                                    }
                                  ],
                                  "end": 1748135563349,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135563511",
                                  "message": "export",
                                  "start": 1748135563511,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748135563531",
                                      "message": "transfer",
                                      "start": 1748135563531,
                                      "status": "success",
                                      "end": 1748135898500,
                                      "result": {
                                        "size": 2713714688
                                      }
                                    },
                                    {
                                      "id": "1748135910743",
                                      "message": "clean-vm",
                                      "start": 1748135910743,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/b02697d4-23b5-598f-4def-d6b197bc1085/20250524T220039Z.json",
                                            "actual": 2713714688,
                                            "expected": 2716501504
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748135913811,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748135915372
                                }
                              ],
                              "end": 1748135915372
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "b5d74e0b-388c-019a-6994-e174c9ca7a51"
                              },
                              "id": "1748135563341",
                              "message": "backup VM",
                              "start": 1748135563341,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135563363",
                                  "message": "clean-vm",
                                  "start": 1748135563363,
                                  "status": "success",
                                  "end": 1748135563857,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135565987",
                                  "message": "export",
                                  "start": 1748135565987,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748135565996",
                                      "message": "transfer",
                                      "start": 1748135565996,
                                      "status": "success",
                                      "end": 1748136473233,
                                      "result": {
                                        "size": 13337886720
                                      }
                                    },
                                    {
                                      "id": "1748136476298",
                                      "message": "clean-vm",
                                      "start": 1748136476298,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/b5d74e0b-388c-019a-6994-e174c9ca7a51/20250524T220255Z.json",
                                            "actual": 13337886720,
                                            "expected": 13343341568
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748136477094,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748136477313
                                }
                              ],
                              "end": 1748136477313
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "c49a5398-38db-4fcd-481b-30181a5503cc"
                              },
                              "id": "1748135916054",
                              "message": "backup VM",
                              "start": 1748135916054,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135920380",
                                  "message": "clean-vm",
                                  "start": 1748135920380,
                                  "status": "success",
                                  "end": 1748135930666,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "id": "1748135930919",
                                  "message": "clean-vm",
                                  "start": 1748135930919,
                                  "status": "success",
                                  "end": 1748135935805,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "end": 1748135936610
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "c5fbe7d1-9f9c-0167-5ed2-1dff1104ef4b"
                              },
                              "id": "1748135937843",
                              "message": "backup VM",
                              "start": 1748135937843,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1748135940204",
                                  "message": "clean-vm",
                                  "start": 1748135940204,
                                  "status": "success",
                                  "end": 1748135948613,
                                  "result": {
                                    "merge": false
                                  }
                                },
                                {
                                  "data": {
                                    "id": "337848e1-b52b-4e08-9bf0-ffce54de4276",
                                    "type": "remote"
                                  },
                                  "id": "1748135951349",
                                  "message": "export",
                                  "start": 1748135951349,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1748135951810",
                                      "message": "transfer",
                                      "start": 1748135951810,
                                      "status": "success",
                                      "end": 1748136066962,
                                      "result": {
                                        "size": 960495616
                                      }
                                    },
                                    {
                                      "id": "1748136083459",
                                      "message": "clean-vm",
                                      "start": 1748136083459,
                                      "status": "success",
                                      "warnings": [
                                        {
                                          "data": {
                                            "path": "/xo-vm-backups/c5fbe7d1-9f9c-0167-5ed2-1dff1104ef4b/20250524T220054Z.json",
                                            "actual": 960495616,
                                            "expected": 962891264
                                          },
                                          "message": "cleanVm: incorrect backup size in metadata"
                                        }
                                      ],
                                      "end": 1748136093352,
                                      "result": {
                                        "merge": true
                                      }
                                    }
                                  ],
                                  "end": 1748136096139
                                }
                              ],
                              "end": 1748136096139
                            }
                          ],
                          "end": 1748136477317,
                          "result": {
                            "errno": -9,
                            "code": "EBADF",
                            "syscall": "scandir",
                            "path": "/run/xo-server/mounts/5dba60dc-c4fc-4d49-9a1e-453522c0d8f7/xo-vm-backups/341a6155-908b-66ba-76ee-56a1d5163af1"
                          }
                        }
                        
                        1 Reply Last reply Reply Quote 0
                        • florentF Offline
                          florent Vates 🪐 XO Team
                          last edited by

                          I am checking it
                          Can you confirm that this a delta backup job and you are not using block.

                          M 1 Reply Last reply Reply Quote 0
                          • M Offline
                            manilx @florent
                            last edited by manilx

                            @florent The original backup is delta. Using NBD+CBT to a NFS storage.

                            Also get these frequently:

                            ScreenShot 2025-05-26 at 09.18.45.png

                            But nothing is running in parallel.

                            1 Reply Last reply Reply Quote 0
                            • florentF Offline
                              florent Vates 🪐 XO Team
                              last edited by

                              this error is known : this when a mirror backup start before the end of the asynchronoux cleaning of previous backup is not done

                              you can check the "merge backup synchronously" on the advanced setting of the first backup

                              M 2 Replies Last reply Reply Quote 1
                              • M Offline
                                manilx @florent
                                last edited by manilx

                                @florent All my backups have this set! And this only happens with the new backup code, never with the old.

                                1 Reply Last reply Reply Quote 0
                                • M Offline
                                  manilx @florent
                                  last edited by

                                  @florent Giving up on mirror backups. Switching to direct backups to remore storage.
                                  At least until this works again as expected.

                                  1 Reply Last reply Reply Quote 0
                                  • florentF Offline
                                    florent Vates 🪐 XO Team
                                    last edited by

                                    At least I reproduce it locally , when going from a non block remote to an encrypted remote

                                    Is your target remote encrypted ?

                                    M 1 Reply Last reply Reply Quote 0
                                    • M Offline
                                      manilx @florent
                                      last edited by

                                      @florent Simple NFS, no encryption.

                                      florentF 1 Reply Last reply Reply Quote 0
                                      • florentF Offline
                                        florent Vates 🪐 XO Team @manilx
                                        last edited by

                                        @manilx said in Our future backup code: test it!:

                                        @florent Simple NFS, no encryption.

                                        I have a potential fix here on the branch fix_backup
                                        ( https://github.com/vatesfr/xen-orchestra/pull/8635/files)

                                        fbeauchamp opened this pull request in vatesfr/xen-orchestra

                                        open fix(disk-transform): backup ominibus fix #8635

                                        M 1 Reply Last reply Reply Quote 0
                                        • M Offline
                                          manilx @florent
                                          last edited by

                                          @florent waiting for final master

                                          florentF 1 Reply Last reply Reply Quote 0
                                          • florentF Offline
                                            florent Vates 🪐 XO Team @manilx
                                            last edited by

                                            @manilx said in Our future backup code: test it!:

                                            @florent waiting for final master

                                            this will reach master today, and probably be released in the may version of XO

                                            F 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post