XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup Failure to BackBlaze

    Scheduled Pinned Locked Moved Backup
    6 Posts 3 Posters 134 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A Online
      acebmxer
      last edited by

      Failure started with one VM no happens with two vms. Local backups are ok error only copying to BackBlase S3.

      XO is built from sources and is latest build

      Screenshot 2025-06-09 065821.png

      Screenshot 2025-06-09 065047.png

      Screenshot 2025-06-09 065510.png

      Screenshot 2025-06-09 065604.png

      {
        "data": {
          "mode": "delta",
          "reportWhen": "always"
        },
        "id": "1749442138367",
        "jobId": "d24e6d91-c3c0-4a5e-a92b-be5f40043e5e",
        "jobName": "Backup Copy to BackBlaze",
        "message": "backup",
        "scheduleId": "940f55c2-0e14-43ce-a050-33a5c25b9729",
        "start": 1749442138367,
        "status": "failure",
        "infos": [
          {
            "data": {
              "vms": [
                "3446772a-4110-7a2c-db35-286c73af4ab4",
                "43263bf7-09dd-e357-235a-213fad1fbb0b",
                "afbef202-af84-7e64-100a-e8a4c40d5130",
                "b25a5709-f1f8-e942-f0cc-f443eb9b9cf3",
                "e0a3093a-52fd-f8dc-1c39-075eeb9d0314"
              ]
            },
            "message": "vms"
          }
        ],
        "tasks": [
          {
            "data": {
              "type": "VM",
              "id": "3446772a-4110-7a2c-db35-286c73af4ab4"
            },
            "id": "1749442139308",
            "message": "backup VM",
            "start": 1749442139308,
            "status": "success",
            "tasks": [
              {
                "id": "1749442139349",
                "message": "clean-vm",
                "start": 1749442139349,
                "status": "success",
                "end": 1749442144492,
                "result": {
                  "merge": false
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442145040",
                "message": "export",
                "start": 1749442145040,
                "status": "success",
                "tasks": [
                  {
                    "id": "1749442145120",
                    "message": "transfer",
                    "start": 1749442145120,
                    "status": "success",
                    "end": 1749442149312,
                    "result": {
                      "size": 188743680
                    }
                  },
                  {
                    "id": "1749442153402",
                    "message": "clean-vm",
                    "start": 1749442153402,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1749442156199",
                        "message": "merge",
                        "start": 1749442156199,
                        "status": "success",
                        "end": 1749442613956
                      }
                    ],
                    "end": 1749442614990,
                    "result": {
                      "merge": true
                    }
                  }
                ],
                "end": 1749442614990
              }
            ],
            "end": 1749442614990
          },
          {
            "data": {
              "type": "VM",
              "id": "43263bf7-09dd-e357-235a-213fad1fbb0b"
            },
            "id": "1749442139348",
            "message": "backup VM",
            "start": 1749442139348,
            "status": "success",
            "tasks": [
              {
                "id": "1749442139470",
                "message": "clean-vm",
                "start": 1749442139470,
                "status": "success",
                "end": 1749442144860,
                "result": {
                  "merge": false
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442146424",
                "message": "export",
                "start": 1749442146424,
                "status": "success",
                "tasks": [
                  {
                    "id": "1749442146839",
                    "message": "transfer",
                    "start": 1749442146839,
                    "status": "success",
                    "end": 1749442152292,
                    "result": {
                      "size": 411041792
                    }
                  },
                  {
                    "id": "1749442156027",
                    "message": "clean-vm",
                    "start": 1749442156027,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1749442159003",
                        "message": "merge",
                        "start": 1749442159003,
                        "status": "success",
                        "end": 1749442696341
                      }
                    ],
                    "end": 1749442697668,
                    "result": {
                      "merge": true
                    }
                  }
                ],
                "end": 1749442697668
              }
            ],
            "end": 1749442697668
          },
          {
            "data": {
              "type": "VM",
              "id": "afbef202-af84-7e64-100a-e8a4c40d5130"
            },
            "id": "1749442615017",
            "message": "backup VM",
            "start": 1749442615017,
            "status": "failure",
            "tasks": [
              {
                "id": "1749442615061",
                "message": "clean-vm",
                "start": 1749442615061,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749442630542",
                    "message": "merge",
                    "start": 1749442630542,
                    "status": "failure",
                    "end": 1749442725411,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "ae90518bbf196efe",
                        "extendedRequestId": "aNyY2MTJXNXQ32GShYsBl8TbXY4g5amY/",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749442725411,
                "result": {
                  "name": "InvalidRequest",
                  "$fault": "client",
                  "$metadata": {
                    "httpStatusCode": 400,
                    "requestId": "ae90518bbf196efe",
                    "extendedRequestId": "aNyY2MTJXNXQ32GShYsBl8TbXY4g5amY/",
                    "attempts": 1,
                    "totalRetryDelay": 0
                  },
                  "Code": "InvalidRequest",
                  "message": "no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390",
                  "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442729548",
                "message": "export",
                "start": 1749442729548,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749442730102",
                    "message": "transfer",
                    "start": 1749442730102,
                    "status": "success",
                    "end": 1749443376739,
                    "result": {
                      "size": 2231369728
                    }
                  },
                  {
                    "id": "1749443379535",
                    "message": "clean-vm",
                    "start": 1749443379535,
                    "status": "failure",
                    "tasks": [
                      {
                        "id": "1749443394670",
                        "message": "merge",
                        "start": 1749443394670,
                        "status": "failure",
                        "end": 1749445284085,
                        "result": {
                          "name": "InvalidRequest",
                          "$fault": "client",
                          "$metadata": {
                            "httpStatusCode": 400,
                            "requestId": "fc8bcda5810aee27",
                            "extendedRequestId": "aN682bDKvNZU3sWQpYh9lZzb5YxQ59GaM",
                            "attempts": 1,
                            "totalRetryDelay": 0
                          },
                          "Code": "InvalidRequest",
                          "message": "no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580",
                          "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                        }
                      }
                    ],
                    "end": 1749445284086,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "fc8bcda5810aee27",
                        "extendedRequestId": "aN682bDKvNZU3sWQpYh9lZzb5YxQ59GaM",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749445284086
              }
            ],
            "end": 1749445284086
          },
          {
            "data": {
              "type": "VM",
              "id": "b25a5709-f1f8-e942-f0cc-f443eb9b9cf3"
            },
            "id": "1749442697741",
            "message": "backup VM",
            "start": 1749442697741,
            "status": "success",
            "tasks": [
              {
                "id": "1749442697768",
                "message": "clean-vm",
                "start": 1749442697768,
                "status": "success",
                "end": 1749442714538,
                "result": {
                  "merge": false
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442717447",
                "message": "export",
                "start": 1749442717447,
                "status": "success",
                "tasks": [
                  {
                    "id": "1749442717512",
                    "message": "transfer",
                    "start": 1749442717512,
                    "status": "success",
                    "end": 1749442774942,
                    "result": {
                      "size": 1967128576
                    }
                  },
                  {
                    "id": "1749442785120",
                    "message": "clean-vm",
                    "start": 1749442785120,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1749442801962",
                        "message": "merge",
                        "start": 1749442801962,
                        "status": "success",
                        "end": 1749445255034
                      }
                    ],
                    "end": 1749445256257,
                    "result": {
                      "merge": true
                    }
                  }
                ],
                "end": 1749445256257
              }
            ],
            "end": 1749445256257
          },
          {
            "data": {
              "type": "VM",
              "id": "e0a3093a-52fd-f8dc-1c39-075eeb9d0314"
            },
            "id": "1749445256312",
            "message": "backup VM",
            "start": 1749445256312,
            "status": "failure",
            "tasks": [
              {
                "id": "1749445256348",
                "message": "clean-vm",
                "start": 1749445256348,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749445262322",
                    "message": "merge",
                    "start": 1749445262322,
                    "status": "failure",
                    "end": 1749445312457,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "1cd85151a7d30275",
                        "extendedRequestId": "aN3Y2RTIiNWQ3WWTeYnFlsTYEY/85xGYR",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749445312457,
                "result": {
                  "name": "InvalidRequest",
                  "$fault": "client",
                  "$metadata": {
                    "httpStatusCode": 400,
                    "requestId": "1cd85151a7d30275",
                    "extendedRequestId": "aN3Y2RTIiNWQ3WWTeYnFlsTYEY/85xGYR",
                    "attempts": 1,
                    "totalRetryDelay": 0
                  },
                  "Code": "InvalidRequest",
                  "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                  "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749445313218",
                "message": "export",
                "start": 1749445313218,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749445313301",
                    "message": "transfer",
                    "start": 1749445313301,
                    "status": "success",
                    "end": 1749445498940,
                    "result": {
                      "size": 1451229184
                    }
                  },
                  {
                    "id": "1749445504096",
                    "message": "clean-vm",
                    "start": 1749445504096,
                    "status": "failure",
                    "tasks": [
                      {
                        "id": "1749445510480",
                        "message": "merge",
                        "start": 1749445510480,
                        "status": "failure",
                        "end": 1749445539561,
                        "result": {
                          "name": "InvalidRequest",
                          "$fault": "client",
                          "$metadata": {
                            "httpStatusCode": 400,
                            "requestId": "16bfab6aac1747e1",
                            "extendedRequestId": "aN2Y23jLzNZw3GGSXYoxlODa3Y0s5WWYr",
                            "attempts": 1,
                            "totalRetryDelay": 0
                          },
                          "Code": "InvalidRequest",
                          "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                          "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                        }
                      }
                    ],
                    "end": 1749445539561,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "16bfab6aac1747e1",
                        "extendedRequestId": "aN2Y23jLzNZw3GGSXYoxlODa3Y0s5WWYr",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749445539561
              }
            ],
            "end": 1749445539561
          }
        ],
        "end": 1749445539561
      }[2025-06-09T04_08_58.367Z - backup NG.json](Invalid file type. Allowed types are: .png, .jpg, .bmp, .txt, .diff, .patch, .jpeg) 
      
      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Hi,

        It's like some files are missing, I'm not ultra confident about Backblaze API/errors.

        @lsouai-vates can you ask someone take a quick look?

        A 1 Reply Last reply Reply Quote 0
        • A Online
          acebmxer @olivierlambert
          last edited by

          @olivierlambert this is related to bug report - https://github.com/vatesfr/xen-orchestra/issues/8678

          Sorry for posting in both locations.

          acebmxer created this issue in vatesfr/xen-orchestra

          open Backup job failed #8678

          florentF 1 Reply Last reply Reply Quote 0
          • florentF Offline
            florent Vates 🪐 XO Team @acebmxer
            last edited by

            @acebmxer backblaze can be quite fickle. I am quite sure the file 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580 is not a file created directly by XO .
            The error is during cleanup. Since this is a mirror backup I would delete the target backup of the relevant VM to ensure they a reuploaded completely from the source

            we are currently reworking the way XO retry a failed command, it should improve resilience , especially on fragile network/storage

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              Yeah, I'm not surprised that Backblaze is always full of surprises 😕

              1 Reply Last reply Reply Quote 0
              • A Online
                acebmxer
                last edited by

                Thank you for the update. Last night before I saw your replys that is exactly what i did. I purged the backup's for the two VMs from BackBlaze. The backup job completed successfully while copying over the previous backup images.

                Screenshot 2025-06-10 055256.png

                1 Reply Last reply Reply Quote 1
                • First post
                  Last post