XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup Failure to BackBlaze

    Scheduled Pinned Locked Moved Backup
    6 Posts 3 Posters 674 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A Offline
      acebmxer
      last edited by

      Failure started with one VM no happens with two vms. Local backups are ok error only copying to BackBlase S3.

      XO is built from sources and is latest build

      Screenshot 2025-06-09 065821.png

      Screenshot 2025-06-09 065047.png

      Screenshot 2025-06-09 065510.png

      Screenshot 2025-06-09 065604.png

      {
        "data": {
          "mode": "delta",
          "reportWhen": "always"
        },
        "id": "1749442138367",
        "jobId": "d24e6d91-c3c0-4a5e-a92b-be5f40043e5e",
        "jobName": "Backup Copy to BackBlaze",
        "message": "backup",
        "scheduleId": "940f55c2-0e14-43ce-a050-33a5c25b9729",
        "start": 1749442138367,
        "status": "failure",
        "infos": [
          {
            "data": {
              "vms": [
                "3446772a-4110-7a2c-db35-286c73af4ab4",
                "43263bf7-09dd-e357-235a-213fad1fbb0b",
                "afbef202-af84-7e64-100a-e8a4c40d5130",
                "b25a5709-f1f8-e942-f0cc-f443eb9b9cf3",
                "e0a3093a-52fd-f8dc-1c39-075eeb9d0314"
              ]
            },
            "message": "vms"
          }
        ],
        "tasks": [
          {
            "data": {
              "type": "VM",
              "id": "3446772a-4110-7a2c-db35-286c73af4ab4"
            },
            "id": "1749442139308",
            "message": "backup VM",
            "start": 1749442139308,
            "status": "success",
            "tasks": [
              {
                "id": "1749442139349",
                "message": "clean-vm",
                "start": 1749442139349,
                "status": "success",
                "end": 1749442144492,
                "result": {
                  "merge": false
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442145040",
                "message": "export",
                "start": 1749442145040,
                "status": "success",
                "tasks": [
                  {
                    "id": "1749442145120",
                    "message": "transfer",
                    "start": 1749442145120,
                    "status": "success",
                    "end": 1749442149312,
                    "result": {
                      "size": 188743680
                    }
                  },
                  {
                    "id": "1749442153402",
                    "message": "clean-vm",
                    "start": 1749442153402,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1749442156199",
                        "message": "merge",
                        "start": 1749442156199,
                        "status": "success",
                        "end": 1749442613956
                      }
                    ],
                    "end": 1749442614990,
                    "result": {
                      "merge": true
                    }
                  }
                ],
                "end": 1749442614990
              }
            ],
            "end": 1749442614990
          },
          {
            "data": {
              "type": "VM",
              "id": "43263bf7-09dd-e357-235a-213fad1fbb0b"
            },
            "id": "1749442139348",
            "message": "backup VM",
            "start": 1749442139348,
            "status": "success",
            "tasks": [
              {
                "id": "1749442139470",
                "message": "clean-vm",
                "start": 1749442139470,
                "status": "success",
                "end": 1749442144860,
                "result": {
                  "merge": false
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442146424",
                "message": "export",
                "start": 1749442146424,
                "status": "success",
                "tasks": [
                  {
                    "id": "1749442146839",
                    "message": "transfer",
                    "start": 1749442146839,
                    "status": "success",
                    "end": 1749442152292,
                    "result": {
                      "size": 411041792
                    }
                  },
                  {
                    "id": "1749442156027",
                    "message": "clean-vm",
                    "start": 1749442156027,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1749442159003",
                        "message": "merge",
                        "start": 1749442159003,
                        "status": "success",
                        "end": 1749442696341
                      }
                    ],
                    "end": 1749442697668,
                    "result": {
                      "merge": true
                    }
                  }
                ],
                "end": 1749442697668
              }
            ],
            "end": 1749442697668
          },
          {
            "data": {
              "type": "VM",
              "id": "afbef202-af84-7e64-100a-e8a4c40d5130"
            },
            "id": "1749442615017",
            "message": "backup VM",
            "start": 1749442615017,
            "status": "failure",
            "tasks": [
              {
                "id": "1749442615061",
                "message": "clean-vm",
                "start": 1749442615061,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749442630542",
                    "message": "merge",
                    "start": 1749442630542,
                    "status": "failure",
                    "end": 1749442725411,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "ae90518bbf196efe",
                        "extendedRequestId": "aNyY2MTJXNXQ32GShYsBl8TbXY4g5amY/",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749442725411,
                "result": {
                  "name": "InvalidRequest",
                  "$fault": "client",
                  "$metadata": {
                    "httpStatusCode": 400,
                    "requestId": "ae90518bbf196efe",
                    "extendedRequestId": "aNyY2MTJXNXQ32GShYsBl8TbXY4g5amY/",
                    "attempts": 1,
                    "totalRetryDelay": 0
                  },
                  "Code": "InvalidRequest",
                  "message": "no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390",
                  "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f475bb85af101ee60_d20250607_m162813_c005_v7007000_t0000_u01749313693390\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442729548",
                "message": "export",
                "start": 1749442729548,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749442730102",
                    "message": "transfer",
                    "start": 1749442730102,
                    "status": "success",
                    "end": 1749443376739,
                    "result": {
                      "size": 2231369728
                    }
                  },
                  {
                    "id": "1749443379535",
                    "message": "clean-vm",
                    "start": 1749443379535,
                    "status": "failure",
                    "tasks": [
                      {
                        "id": "1749443394670",
                        "message": "merge",
                        "start": 1749443394670,
                        "status": "failure",
                        "end": 1749445284085,
                        "result": {
                          "name": "InvalidRequest",
                          "$fault": "client",
                          "$metadata": {
                            "httpStatusCode": 400,
                            "requestId": "fc8bcda5810aee27",
                            "extendedRequestId": "aN682bDKvNZU3sWQpYh9lZzb5YxQ59GaM",
                            "attempts": 1,
                            "totalRetryDelay": 0
                          },
                          "Code": "InvalidRequest",
                          "message": "no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580",
                          "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                        }
                      }
                    ],
                    "end": 1749445284086,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "fc8bcda5810aee27",
                        "extendedRequestId": "aN682bDKvNZU3sWQpYh9lZzb5YxQ59GaM",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749445284086
              }
            ],
            "end": 1749445284086
          },
          {
            "data": {
              "type": "VM",
              "id": "b25a5709-f1f8-e942-f0cc-f443eb9b9cf3"
            },
            "id": "1749442697741",
            "message": "backup VM",
            "start": 1749442697741,
            "status": "success",
            "tasks": [
              {
                "id": "1749442697768",
                "message": "clean-vm",
                "start": 1749442697768,
                "status": "success",
                "end": 1749442714538,
                "result": {
                  "merge": false
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749442717447",
                "message": "export",
                "start": 1749442717447,
                "status": "success",
                "tasks": [
                  {
                    "id": "1749442717512",
                    "message": "transfer",
                    "start": 1749442717512,
                    "status": "success",
                    "end": 1749442774942,
                    "result": {
                      "size": 1967128576
                    }
                  },
                  {
                    "id": "1749442785120",
                    "message": "clean-vm",
                    "start": 1749442785120,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1749442801962",
                        "message": "merge",
                        "start": 1749442801962,
                        "status": "success",
                        "end": 1749445255034
                      }
                    ],
                    "end": 1749445256257,
                    "result": {
                      "merge": true
                    }
                  }
                ],
                "end": 1749445256257
              }
            ],
            "end": 1749445256257
          },
          {
            "data": {
              "type": "VM",
              "id": "e0a3093a-52fd-f8dc-1c39-075eeb9d0314"
            },
            "id": "1749445256312",
            "message": "backup VM",
            "start": 1749445256312,
            "status": "failure",
            "tasks": [
              {
                "id": "1749445256348",
                "message": "clean-vm",
                "start": 1749445256348,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749445262322",
                    "message": "merge",
                    "start": 1749445262322,
                    "status": "failure",
                    "end": 1749445312457,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "1cd85151a7d30275",
                        "extendedRequestId": "aN3Y2RTIiNWQ3WWTeYnFlsTYEY/85xGYR",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749445312457,
                "result": {
                  "name": "InvalidRequest",
                  "$fault": "client",
                  "$metadata": {
                    "httpStatusCode": 400,
                    "requestId": "1cd85151a7d30275",
                    "extendedRequestId": "aN3Y2RTIiNWQ3WWTeYnFlsTYEY/85xGYR",
                    "attempts": 1,
                    "totalRetryDelay": 0
                  },
                  "Code": "InvalidRequest",
                  "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                  "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                }
              },
              {
                "data": {
                  "id": "f4b45591-e0d7-49b7-a4dd-d25472676cc4",
                  "type": "remote"
                },
                "id": "1749445313218",
                "message": "export",
                "start": 1749445313218,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1749445313301",
                    "message": "transfer",
                    "start": 1749445313301,
                    "status": "success",
                    "end": 1749445498940,
                    "result": {
                      "size": 1451229184
                    }
                  },
                  {
                    "id": "1749445504096",
                    "message": "clean-vm",
                    "start": 1749445504096,
                    "status": "failure",
                    "tasks": [
                      {
                        "id": "1749445510480",
                        "message": "merge",
                        "start": 1749445510480,
                        "status": "failure",
                        "end": 1749445539561,
                        "result": {
                          "name": "InvalidRequest",
                          "$fault": "client",
                          "$metadata": {
                            "httpStatusCode": 400,
                            "requestId": "16bfab6aac1747e1",
                            "extendedRequestId": "aN2Y23jLzNZw3GGSXYoxlODa3Y0s5WWYr",
                            "attempts": 1,
                            "totalRetryDelay": 0
                          },
                          "Code": "InvalidRequest",
                          "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                          "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                        }
                      }
                    ],
                    "end": 1749445539561,
                    "result": {
                      "name": "InvalidRequest",
                      "$fault": "client",
                      "$metadata": {
                        "httpStatusCode": 400,
                        "requestId": "16bfab6aac1747e1",
                        "extendedRequestId": "aN2Y23jLzNZw3GGSXYoxlODa3Y0s5WWYr",
                        "attempts": 1,
                        "totalRetryDelay": 0
                      },
                      "Code": "InvalidRequest",
                      "message": "no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621",
                      "stack": "InvalidRequest: no such file: 4_z278662f5871d3b0e966c091f_f4862319f7fbb230f_d20250602_m161947_c005_v7007000_t0000_u01748881187621\n    at de_InvalidRequestRes (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:5005:21)\n    at de_CommandError (/opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4940:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:484:18\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:137:14\n    at async /opt/xo/xo-builds/xen-orchestra-202506062218/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22\n    at async S3Handler._copy (/opt/xo/xo-builds/xen-orchestra-202506062218/@xen-orchestra/fs/dist/s3.js:150:7)"
                    }
                  }
                ],
                "end": 1749445539561
              }
            ],
            "end": 1749445539561
          }
        ],
        "end": 1749445539561
      }[2025-06-09T04_08_58.367Z - backup NG.json](Invalid file type. Allowed types are: .png, .jpg, .bmp, .txt, .diff, .patch, .jpeg) 
      
      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Hi,

        It's like some files are missing, I'm not ultra confident about Backblaze API/errors.

        @lsouai-vates can you ask someone take a quick look?

        A 1 Reply Last reply Reply Quote 0
        • A Offline
          acebmxer @olivierlambert
          last edited by

          @olivierlambert this is related to bug report - https://github.com/vatesfr/xen-orchestra/issues/8678

          Sorry for posting in both locations.

          acebmxer created this issue in vatesfr/xen-orchestra

          closed Backup job failed #8678

          florentF 1 Reply Last reply Reply Quote 0
          • florentF Offline
            florent Vates 🪐 XO Team @acebmxer
            last edited by

            @acebmxer backblaze can be quite fickle. I am quite sure the file 4_z278662f5871d3b0e966c091f_f4696d08d6f580ba7_d20250607_m162813_c005_v7007000_t0000_u01749313693580 is not a file created directly by XO .
            The error is during cleanup. Since this is a mirror backup I would delete the target backup of the relevant VM to ensure they a reuploaded completely from the source

            we are currently reworking the way XO retry a failed command, it should improve resilience , especially on fragile network/storage

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              Yeah, I'm not surprised that Backblaze is always full of surprises 😕

              1 Reply Last reply Reply Quote 0
              • A Offline
                acebmxer
                last edited by

                Thank you for the update. Last night before I saw your replys that is exactly what i did. I purged the backup's for the two VMs from BackBlaze. The backup job completed successfully while copying over the previous backup images.

                Screenshot 2025-06-10 055256.png

                1 Reply Last reply Reply Quote 1

                Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                With your input, this post could be even better 💗

                Register Login
                • First post
                  Last post