XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup Mirror Health Check Error

    Scheduled Pinned Locked Moved Backup
    16 Posts 4 Posters 649 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D Offline
      Delgado
      last edited by

      Hello,

      I am mirroring my backup repository from a local nfs share to a Backblaze bucket. Mirroring the backups themselves seems to be working fine. I run into an issue when I enable Health Check. This is the error I am seeing in the backup job. Cannot read properties of undefined (reading 'healthCheckVmsWithTags'). Any help would be greatly appreciated! Thanks!

      1 Reply Last reply Reply Quote 0
      • DanpD Offline
        Danp Pro Support Team
        last edited by Danp

        Hi,

        We will need more information in order to correctly diagnose this issue.

        • Are you running XOA (the pre-built appliance) or XO from sources?
        • If XOA, which release channel is active?
        • If XO, which commit are you on?

        It would also be beneficial if you showed us the stack trace that is displayed with the error message.

        Regards, Dan

        1 Reply Last reply Reply Quote 0
        • D Offline
          Delgado
          last edited by Delgado

          Hello Danp thank you for the reply. I am running xen orchestra from source commit number eedac here in my home lab per my info page. See below for the debug info.

          {
            "data": {
              "mode": "full",
              "reportWhen": "always"
            },
            "id": "1708142806502",
            "jobId": "d0e1800e-0db5-429a-9a7a-847f71bb4bc4",
            "jobName": "Offline Backup Mirror",
            "message": "backup",
            "scheduleId": "ec0373f5-7292-4768-a3aa-725d3d78aabb",
            "start": 1708142806502,
            "status": "failure",
            "infos": [
              {
                "data": {
                  "vms": [
                    "0666c9d1-7846-560f-66c0-27c1f79c67cc",
                    "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19",
                    "2e700210-a790-d3a2-c5f2-add61872f6c7",
                    "c94557ab-6096-2795-0fd8-e9ceb2b838aa",
                    "f0078138-269e-daee-d893-9c9ef49308bb"
                  ]
                },
                "message": "vms"
              }
            ],
            "tasks": [
              {
                "data": {
                  "type": "VM",
                  "id": "0666c9d1-7846-560f-66c0-27c1f79c67cc"
                },
                "id": "1708142813039",
                "message": "backup VM",
                "start": 1708142813039,
                "status": "failure",
                "tasks": [
                  {
                    "data": {
                      "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                      "type": "remote",
                      "isFull": true
                    },
                    "id": "1708142813294",
                    "message": "export",
                    "start": 1708142813294,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1708142813386",
                        "message": "transfer",
                        "start": 1708142813386,
                        "status": "success",
                        "end": 1708143045473,
                        "result": {
                          "size": 4096174213
                        }
                      }
                    ],
                    "end": 1708143046299
                  },
                  {
                    "id": "1708143046300",
                    "message": "clean-vm",
                    "start": 1708143046300,
                    "status": "success",
                    "end": 1708143046970,
                    "result": {
                      "merge": false
                    }
                  }
                ],
                "end": 1708143046972,
                "result": {
                  "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                  "name": "TypeError",
                  "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                }
              },
              {
                "data": {
                  "type": "VM",
                  "id": "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19"
                },
                "id": "1708142813040",
                "message": "backup VM",
                "start": 1708142813040,
                "status": "failure",
                "tasks": [
                  {
                    "data": {
                      "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                      "type": "remote",
                      "isFull": true
                    },
                    "id": "1708142813243",
                    "message": "export",
                    "start": 1708142813243,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1708142813437",
                        "message": "transfer",
                        "start": 1708142813437,
                        "status": "success",
                        "end": 1708144130009,
                        "result": {
                          "size": 13917374917
                        }
                      }
                    ],
                    "end": 1708144131030
                  },
                  {
                    "id": "1708144131033",
                    "message": "clean-vm",
                    "start": 1708144131033,
                    "status": "success",
                    "end": 1708144132043,
                    "result": {
                      "merge": false
                    }
                  }
                ],
                "end": 1708144132051,
                "result": {
                  "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                  "name": "TypeError",
                  "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                }
              },
              {
                "data": {
                  "type": "VM",
                  "id": "2e700210-a790-d3a2-c5f2-add61872f6c7"
                },
                "id": "1708143046974",
                "message": "backup VM",
                "start": 1708143046974,
                "status": "failure",
                "tasks": [
                  {
                    "data": {
                      "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                      "type": "remote",
                      "isFull": true
                    },
                    "id": "1708143047133",
                    "message": "export",
                    "start": 1708143047133,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1708143047234",
                        "message": "transfer",
                        "start": 1708143047234,
                        "status": "success",
                        "end": 1708143387527,
                        "result": {
                          "size": 2789271786
                        }
                      }
                    ],
                    "end": 1708143388188
                  },
                  {
                    "id": "1708143388191",
                    "message": "clean-vm",
                    "start": 1708143388191,
                    "status": "success",
                    "end": 1708143389489,
                    "result": {
                      "merge": false
                    }
                  }
                ],
                "end": 1708143389493,
                "result": {
                  "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                  "name": "TypeError",
                  "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                }
              },
              {
                "data": {
                  "type": "VM",
                  "id": "c94557ab-6096-2795-0fd8-e9ceb2b838aa"
                },
                "id": "1708143389496",
                "message": "backup VM",
                "start": 1708143389496,
                "status": "failure",
                "tasks": [
                  {
                    "data": {
                      "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                      "type": "remote",
                      "isFull": true
                    },
                    "id": "1708143389680",
                    "message": "export",
                    "start": 1708143389680,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1708143389791",
                        "message": "transfer",
                        "start": 1708143389791,
                        "status": "success",
                        "end": 1708144874427,
                        "result": {
                          "size": 17531378529
                        }
                      }
                    ],
                    "end": 1708144875421
                  },
                  {
                    "id": "1708144875430",
                    "message": "clean-vm",
                    "start": 1708144875430,
                    "status": "success",
                    "end": 1708144875814,
                    "result": {
                      "merge": false
                    }
                  }
                ],
                "end": 1708144875814,
                "result": {
                  "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                  "name": "TypeError",
                  "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                }
              },
              {
                "data": {
                  "type": "VM",
                  "id": "f0078138-269e-daee-d893-9c9ef49308bb"
                },
                "id": "1708144132051:0",
                "message": "backup VM",
                "start": 1708144132051,
                "status": "failure",
                "tasks": [
                  {
                    "data": {
                      "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                      "type": "remote",
                      "isFull": true
                    },
                    "id": "1708144132291",
                    "message": "export",
                    "start": 1708144132291,
                    "status": "success",
                    "tasks": [
                      {
                        "id": "1708144132419",
                        "message": "transfer",
                        "start": 1708144132419,
                        "status": "success",
                        "end": 1708144485755,
                        "result": {
                          "size": 3024678308
                        }
                      }
                    ],
                    "end": 1708144486156
                  },
                  {
                    "id": "1708144486157",
                    "message": "clean-vm",
                    "start": 1708144486157,
                    "status": "success",
                    "end": 1708144486483,
                    "result": {
                      "merge": false
                    }
                  }
                ],
                "end": 1708144486485,
                "result": {
                  "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                  "name": "TypeError",
                  "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402161200/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                }
              }
            ],
            "end": 1708144875814
          }
          

          Thank you again,
          Delgado

          DanpD julien-fJ 2 Replies Last reply Reply Quote 0
          • DanpD Offline
            Danp Pro Support Team @Delgado
            last edited by

            @Delgado Thanks for the additional details. This looks like a bug to me, so I'll notify the dev team so that they can take a closer look.

            D 1 Reply Last reply Reply Quote 0
            • D Offline
              Delgado @Danp
              last edited by

              @Danp thank you very much. If any help is needed with testing I can definitely help out.

              1 Reply Last reply Reply Quote 0
              • julien-fJ Offline
                julien-f Vates 🪐 Co-Founder XO Team @Delgado
                last edited by

                @Delgado Hello, can you please send me the raw job, available in the REST API at the address /rest/v0/abckup/jobs/vm/d0e1800e-0db5-429a-9a7a-847f71bb4bc4?

                D 1 Reply Last reply Reply Quote 0
                • D Offline
                  Delgado @julien-f
                  last edited by

                  @julien-f said in Backup Mirror Health Check Error:

                  /rest/v0/abckup/jobs/vm/d0e1800e-0db5-429a-9a7a-847f71bb4bc4

                  Hello @julien-f got to that url I get a not found result.

                  baseurl/rest/v0/backup/jobs/vm/d0e1800e-0db5-429a-9a7a-847f71bb4bc4

                  Backing up one level I see the following.

                  [
                  "/rest/v0/backup/jobs/vm/3a9574ee-3b8c-4873-814f-ed0cbba2562c",
                  "/rest/v0/backup/jobs/vm/60d2ae7c-be31-42b0-af2b-73318e71b1e8",
                  "/rest/v0/backup/jobs/vm/9a0bd3f3-d487-468e-b005-16992b0573b0",
                  "/rest/v0/backup/jobs/vm/9db041ee-0620-4733-a143-0c2e3bf6e9ed"
                  ]
                  

                  I kicked another backup mirror off again just to try and see if it would show up in that list but I do not see it.
                  Log output from the most recent run:

                  {
                    "data": {
                      "mode": "full",
                      "reportWhen": "always"
                    },
                    "id": "1708353648748",
                    "jobId": "0b6ba93f-eb99-4db2-9439-60d931ae5210",
                    "jobName": "Offsite Mirror",
                    "message": "backup",
                    "scheduleId": "8b109f11-5bcf-4922-b514-197d5aa4bb8a",
                    "start": 1708353648748,
                    "status": "failure",
                    "infos": [
                      {
                        "data": {
                          "vms": [
                            "0666c9d1-7846-560f-66c0-27c1f79c67cc",
                            "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19",
                            "23bb847a-0bab-5bba-3c33-ffcf6d1a905b",
                            "2e700210-a790-d3a2-c5f2-add61872f6c7",
                            "50151510-a3cd-ddfb-4d48-3c56e9b1e023",
                            "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3",
                            "793d1d93-ea37-f71a-8d07-faa396d1a6c5",
                            "c94557ab-6096-2795-0fd8-e9ceb2b838aa",
                            "f0078138-269e-daee-d893-9c9ef49308bb"
                          ]
                        },
                        "message": "vms"
                      }
                    ],
                    "tasks": [
                      {
                        "data": {
                          "type": "VM",
                          "id": "0666c9d1-7846-560f-66c0-27c1f79c67cc"
                        },
                        "id": "1708353651640",
                        "message": "backup VM",
                        "start": 1708353651640,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353651884",
                            "message": "clean-vm",
                            "start": 1708353651884,
                            "status": "success",
                            "end": 1708353653384,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353653385,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19"
                        },
                        "id": "1708353651640:0",
                        "message": "backup VM",
                        "start": 1708353651640,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353652048:0",
                            "message": "clean-vm",
                            "start": 1708353652048,
                            "status": "success",
                            "end": 1708353653498,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353653498,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "23bb847a-0bab-5bba-3c33-ffcf6d1a905b"
                        },
                        "id": "1708353653385:0",
                        "message": "backup VM",
                        "start": 1708353653385,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353653711:0",
                            "message": "clean-vm",
                            "start": 1708353653711,
                            "status": "success",
                            "end": 1708353655200,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353655200,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "2e700210-a790-d3a2-c5f2-add61872f6c7"
                        },
                        "id": "1708353653498:1",
                        "message": "backup VM",
                        "start": 1708353653498,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353653809:0",
                            "message": "clean-vm",
                            "start": 1708353653809,
                            "status": "success",
                            "end": 1708353655200,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353655200,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "50151510-a3cd-ddfb-4d48-3c56e9b1e023"
                        },
                        "id": "1708353655200:1",
                        "message": "backup VM",
                        "start": 1708353655200,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353655907:0",
                            "message": "clean-vm",
                            "start": 1708353655907,
                            "status": "success",
                            "end": 1708353658878,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353658878,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3"
                        },
                        "id": "1708353655200:4",
                        "message": "backup VM",
                        "start": 1708353655200,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353655468:0",
                            "message": "clean-vm",
                            "start": 1708353655468,
                            "status": "success",
                            "end": 1708353656400,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353656400,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "793d1d93-ea37-f71a-8d07-faa396d1a6c5"
                        },
                        "id": "1708353656400:1",
                        "message": "backup VM",
                        "start": 1708353656400,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353656817:0",
                            "message": "clean-vm",
                            "start": 1708353656817,
                            "status": "success",
                            "end": 1708353656916,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353656916,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "c94557ab-6096-2795-0fd8-e9ceb2b838aa"
                        },
                        "id": "1708353656916:1",
                        "message": "backup VM",
                        "start": 1708353656916,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353657149:0",
                            "message": "clean-vm",
                            "start": 1708353657149,
                            "status": "success",
                            "end": 1708353658781,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353658781,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      },
                      {
                        "data": {
                          "type": "VM",
                          "id": "f0078138-269e-daee-d893-9c9ef49308bb"
                        },
                        "id": "1708353658782",
                        "message": "backup VM",
                        "start": 1708353658782,
                        "status": "failure",
                        "infos": [
                          {
                            "message": "No new data to upload for this VM"
                          }
                        ],
                        "tasks": [
                          {
                            "id": "1708353659358:0",
                            "message": "clean-vm",
                            "start": 1708353659358,
                            "status": "success",
                            "end": 1708353660263,
                            "result": {
                              "merge": false
                            }
                          }
                        ],
                        "end": 1708353660265,
                        "result": {
                          "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                          "name": "TypeError",
                          "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                        }
                      }
                    ],
                    "end": 1708353660265
                  }
                  

                  Thanks,
                  Delgado

                  julien-fJ florentF 2 Replies Last reply Reply Quote 0
                  • julien-fJ Offline
                    julien-f Vates 🪐 Co-Founder XO Team @Delgado
                    last edited by

                    @Delgado Oh it's mirror backup, my bad, I did not pay attention to this.

                    The URL should be /rest/v0/abckup/jobs/mirror/d0e1800e-0db5-429a-9a7a-847f71bb4bc4 then!

                    D 1 Reply Last reply Reply Quote 0
                    • D Offline
                      Delgado @julien-f
                      last edited by

                      @julien-f It looks like that one is no longer in the list for me.

                      [
                      "/rest/v0/backup/jobs/mirror/0b6ba93f-eb99-4db2-9439-60d931ae5210"
                      ]
                      

                      Here is the output from that job however which had the same error and is the raw log from the failure in my last post.

                      {
                        "name": "Offsite Mirror",
                        "mode": "full",
                        "sourceRemote": "cfea9e94-4d03-4c33-a3ec-07e4643a23da",
                        "remotes": {
                          "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a"
                        },
                        "settings": {
                          "8b109f11-5bcf-4922-b514-197d5aa4bb8a": {
                            "exportRetention": 7
                          },
                          "": {
                            "reportWhen": "always",
                            "reportRecipients": [
                              "redacted"
                            ]
                          }
                        },
                        "type": "mirrorBackup",
                        "schedules": {
                          "8b109f11-5bcf-4922-b514-197d5aa4bb8a": {
                            "cron": "0 0 * * *",
                            "jobId": "0b6ba93f-eb99-4db2-9439-60d931ae5210",
                            "name": "Offsite Mirror Schedule",
                            "timezone": "America/New_York",
                            "enabled": true
                          }
                        },
                        "id": "0b6ba93f-eb99-4db2-9439-60d931ae5210"
                      }
                      
                      1 Reply Last reply Reply Quote 0
                      • florentF Offline
                        florent Vates 🪐 XO Team @Delgado
                        last edited by

                        @Delgado said in Backup Mirror Health Check Error:

                        @julien-f said in Backup Mirror Health Check Error:

                        /rest/v0/abckup/jobs/vm/d0e1800e-0db5-429a-9a7a-847f71bb4bc4

                        Hello @julien-f got to that url I get a not found result.

                        baseurl/rest/v0/backup/jobs/vm/d0e1800e-0db5-429a-9a7a-847f71bb4bc4

                        Backing up one level I see the following.

                        [
                        "/rest/v0/backup/jobs/vm/3a9574ee-3b8c-4873-814f-ed0cbba2562c",
                        "/rest/v0/backup/jobs/vm/60d2ae7c-be31-42b0-af2b-73318e71b1e8",
                        "/rest/v0/backup/jobs/vm/9a0bd3f3-d487-468e-b005-16992b0573b0",
                        "/rest/v0/backup/jobs/vm/9db041ee-0620-4733-a143-0c2e3bf6e9ed"
                        ]
                        

                        I kicked another backup mirror off again just to try and see if it would show up in that list but I do not see it.
                        Log output from the most recent run:

                        {
                          "data": {
                            "mode": "full",
                            "reportWhen": "always"
                          },
                          "id": "1708353648748",
                          "jobId": "0b6ba93f-eb99-4db2-9439-60d931ae5210",
                          "jobName": "Offsite Mirror",
                          "message": "backup",
                          "scheduleId": "8b109f11-5bcf-4922-b514-197d5aa4bb8a",
                          "start": 1708353648748,
                          "status": "failure",
                          "infos": [
                            {
                              "data": {
                                "vms": [
                                  "0666c9d1-7846-560f-66c0-27c1f79c67cc",
                                  "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19",
                                  "23bb847a-0bab-5bba-3c33-ffcf6d1a905b",
                                  "2e700210-a790-d3a2-c5f2-add61872f6c7",
                                  "50151510-a3cd-ddfb-4d48-3c56e9b1e023",
                                  "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3",
                                  "793d1d93-ea37-f71a-8d07-faa396d1a6c5",
                                  "c94557ab-6096-2795-0fd8-e9ceb2b838aa",
                                  "f0078138-269e-daee-d893-9c9ef49308bb"
                                ]
                              },
                              "message": "vms"
                            }
                          ],
                          "tasks": [
                            {
                              "data": {
                                "type": "VM",
                                "id": "0666c9d1-7846-560f-66c0-27c1f79c67cc"
                              },
                              "id": "1708353651640",
                              "message": "backup VM",
                              "start": 1708353651640,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353651884",
                                  "message": "clean-vm",
                                  "start": 1708353651884,
                                  "status": "success",
                                  "end": 1708353653384,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353653385,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19"
                              },
                              "id": "1708353651640:0",
                              "message": "backup VM",
                              "start": 1708353651640,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353652048:0",
                                  "message": "clean-vm",
                                  "start": 1708353652048,
                                  "status": "success",
                                  "end": 1708353653498,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353653498,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "23bb847a-0bab-5bba-3c33-ffcf6d1a905b"
                              },
                              "id": "1708353653385:0",
                              "message": "backup VM",
                              "start": 1708353653385,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353653711:0",
                                  "message": "clean-vm",
                                  "start": 1708353653711,
                                  "status": "success",
                                  "end": 1708353655200,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353655200,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "2e700210-a790-d3a2-c5f2-add61872f6c7"
                              },
                              "id": "1708353653498:1",
                              "message": "backup VM",
                              "start": 1708353653498,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353653809:0",
                                  "message": "clean-vm",
                                  "start": 1708353653809,
                                  "status": "success",
                                  "end": 1708353655200,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353655200,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "50151510-a3cd-ddfb-4d48-3c56e9b1e023"
                              },
                              "id": "1708353655200:1",
                              "message": "backup VM",
                              "start": 1708353655200,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353655907:0",
                                  "message": "clean-vm",
                                  "start": 1708353655907,
                                  "status": "success",
                                  "end": 1708353658878,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353658878,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3"
                              },
                              "id": "1708353655200:4",
                              "message": "backup VM",
                              "start": 1708353655200,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353655468:0",
                                  "message": "clean-vm",
                                  "start": 1708353655468,
                                  "status": "success",
                                  "end": 1708353656400,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353656400,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "793d1d93-ea37-f71a-8d07-faa396d1a6c5"
                              },
                              "id": "1708353656400:1",
                              "message": "backup VM",
                              "start": 1708353656400,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353656817:0",
                                  "message": "clean-vm",
                                  "start": 1708353656817,
                                  "status": "success",
                                  "end": 1708353656916,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353656916,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "c94557ab-6096-2795-0fd8-e9ceb2b838aa"
                              },
                              "id": "1708353656916:1",
                              "message": "backup VM",
                              "start": 1708353656916,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353657149:0",
                                  "message": "clean-vm",
                                  "start": 1708353657149,
                                  "status": "success",
                                  "end": 1708353658781,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353658781,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            },
                            {
                              "data": {
                                "type": "VM",
                                "id": "f0078138-269e-daee-d893-9c9ef49308bb"
                              },
                              "id": "1708353658782",
                              "message": "backup VM",
                              "start": 1708353658782,
                              "status": "failure",
                              "infos": [
                                {
                                  "message": "No new data to upload for this VM"
                                }
                              ],
                              "tasks": [
                                {
                                  "id": "1708353659358:0",
                                  "message": "clean-vm",
                                  "start": 1708353659358,
                                  "status": "success",
                                  "end": 1708353660263,
                                  "result": {
                                    "merge": false
                                  }
                                }
                              ],
                              "end": 1708353660265,
                              "result": {
                                "message": "Cannot read properties of undefined (reading 'healthCheckVmsWithTags')",
                                "name": "TypeError",
                                "stack": "TypeError: Cannot read properties of undefined (reading 'healthCheckVmsWithTags')\n    at FullRemoteVmBackupRunner._healthCheck (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_Abstract.mjs:77:32)\n    at Disposable.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:96:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async FullRemoteVmBackupRunner.run (file:///opt/xo/xo-builds/xen-orchestra-202402180916/@xen-orchestra/backups/_runners/_vmRunners/_AbstractRemote.mjs:94:5)"
                              }
                            }
                          ],
                          "end": 1708353660265
                        }
                        

                        Thanks,
                        Delgado

                        hi,

                        I have a branch that should fix the issue : https://github.com/vatesfr/xen-orchestra/pull/7396
                        Please tell me if it fixes the issue.

                        Regard

                        fbeauchamp opened this pull request in vatesfr/xen-orchestra

                        closed fix(backup/mirror): don't run healthcheck if nothing as been transferred #7396

                        D 2 Replies Last reply Reply Quote 0
                        • D Offline
                          Delgado @florent
                          last edited by

                          @florent I will give it a shot and let you know.

                          Thanks,
                          Delgado

                          1 Reply Last reply Reply Quote 0
                          • D Offline
                            Delgado @florent
                            last edited by Delgado

                            @florent I am noticing the following so far.

                            • I noticed that my normal backups are now no longer performing health checks when a backup runs.
                            • If there are no changes to the source BR the health check will not run as designed
                            • If I run a backup of some vms to a BR and then run the mirror the health check is not running.
                            • If I run a backup of all the vms to a BR and then run the mirror job the health check is not running.

                            Normal full backup log:

                            {
                              "data": {
                                "mode": "full",
                                "reportWhen": "always"
                              },
                              "id": "1708461785125",
                              "jobId": "9a0bd3f3-d487-468e-b005-16992b0573b0",
                              "jobName": "Online Backup",
                              "message": "backup",
                              "scheduleId": "173edce6-e8bf-4110-be50-a06f06f31730",
                              "start": 1708461785125,
                              "status": "success",
                              "infos": [
                                {
                                  "data": {
                                    "vms": [
                                      "23bb847a-0bab-5bba-3c33-ffcf6d1a905b",
                                      "50151510-a3cd-ddfb-4d48-3c56e9b1e023",
                                      "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3"
                                    ]
                                  },
                                  "message": "vms"
                                }
                              ],
                              "tasks": [
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "23bb847a-0bab-5bba-3c33-ffcf6d1a905b",
                                    "name_label": "AdGuard"
                                  },
                                  "id": "1708462086054",
                                  "message": "backup VM",
                                  "start": 1708462086054,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1708462086161",
                                      "message": "snapshot",
                                      "start": 1708462086161,
                                      "status": "success",
                                      "end": 1708462095587,
                                      "result": "f50fab62-49d8-70ac-f118-377161db2fe8"
                                    },
                                    {
                                      "data": {
                                        "id": "cfea9e94-4d03-4c33-a3ec-07e4643a23da",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708462095785",
                                      "message": "export",
                                      "start": 1708462095785,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708462095811",
                                          "message": "transfer",
                                          "start": 1708462095811,
                                          "status": "success",
                                          "end": 1708462739298,
                                          "result": {
                                            "size": 4886084605
                                          }
                                        }
                                      ],
                                      "end": 1708462739877
                                    },
                                    {
                                      "id": "1708462742143",
                                      "message": "clean-vm",
                                      "start": 1708462742143,
                                      "status": "success",
                                      "end": 1708462742181,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708462742187
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "50151510-a3cd-ddfb-4d48-3c56e9b1e023",
                                    "name_label": "Xen Orchestra"
                                  },
                                  "id": "1708462086063",
                                  "message": "backup VM",
                                  "start": 1708462086063,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1708462086158",
                                      "message": "snapshot",
                                      "start": 1708462086158,
                                      "status": "success",
                                      "end": 1708462110377,
                                      "result": "8b71d94e-ebee-a265-840a-118d5a85bb27"
                                    },
                                    {
                                      "data": {
                                        "id": "cfea9e94-4d03-4c33-a3ec-07e4643a23da",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708462110590",
                                      "message": "export",
                                      "start": 1708462110590,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708462110731",
                                          "message": "transfer",
                                          "start": 1708462110731,
                                          "status": "success",
                                          "end": 1708462917668,
                                          "result": {
                                            "size": 6745019923
                                          }
                                        }
                                      ],
                                      "end": 1708462917756
                                    },
                                    {
                                      "id": "1708462919562",
                                      "message": "clean-vm",
                                      "start": 1708462919562,
                                      "status": "success",
                                      "end": 1708462920672,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708462921204
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3",
                                    "name_label": "Git"
                                  },
                                  "id": "1708462742202",
                                  "message": "backup VM",
                                  "start": 1708462742202,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "id": "1708462742235",
                                      "message": "snapshot",
                                      "start": 1708462742235,
                                      "status": "success",
                                      "end": 1708462754060,
                                      "result": "95907173-3252-48c2-9df5-ba5eaca3f525"
                                    },
                                    {
                                      "data": {
                                        "id": "cfea9e94-4d03-4c33-a3ec-07e4643a23da",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708462754306",
                                      "message": "export",
                                      "start": 1708462754306,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708462754322",
                                          "message": "transfer",
                                          "start": 1708462754322,
                                          "status": "success",
                                          "end": 1708463082422,
                                          "result": {
                                            "size": 2961587877
                                          }
                                        }
                                      ],
                                      "end": 1708463082494
                                    },
                                    {
                                      "id": "1708463098892",
                                      "message": "clean-vm",
                                      "start": 1708463098892,
                                      "status": "success",
                                      "end": 1708463099466,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708463099524
                                }
                              ],
                              "end": 1708463099525
                            }
                            

                            Here is the log for the mirror job. I will update it once it's done.

                            {
                              "data": {
                                "mode": "full",
                                "reportWhen": "always"
                              },
                              "id": "1708463768762",
                              "jobId": "0b6ba93f-eb99-4db2-9439-60d931ae5210",
                              "jobName": "Offsite Mirror",
                              "message": "backup",
                              "scheduleId": "8b109f11-5bcf-4922-b514-197d5aa4bb8a",
                              "start": 1708463768762,
                              "status": "success",
                              "infos": [
                                {
                                  "data": {
                                    "vms": [
                                      "0666c9d1-7846-560f-66c0-27c1f79c67cc",
                                      "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19",
                                      "23bb847a-0bab-5bba-3c33-ffcf6d1a905b",
                                      "2e700210-a790-d3a2-c5f2-add61872f6c7",
                                      "50151510-a3cd-ddfb-4d48-3c56e9b1e023",
                                      "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3",
                                      "793d1d93-ea37-f71a-8d07-faa396d1a6c5",
                                      "c94557ab-6096-2795-0fd8-e9ceb2b838aa",
                                      "f0078138-269e-daee-d893-9c9ef49308bb"
                                    ]
                                  },
                                  "message": "vms"
                                }
                              ],
                              "tasks": [
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "0666c9d1-7846-560f-66c0-27c1f79c67cc"
                                  },
                                  "id": "1708463775484",
                                  "message": "backup VM",
                                  "start": 1708463775484,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708463775744",
                                      "message": "export",
                                      "start": 1708463775744,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708463775826",
                                          "message": "transfer",
                                          "start": 1708463775826,
                                          "status": "success",
                                          "end": 1708463952753,
                                          "result": {
                                            "size": 4127335769
                                          }
                                        }
                                      ],
                                      "end": 1708463953555
                                    },
                                    {
                                      "id": "1708463953556",
                                      "message": "clean-vm",
                                      "start": 1708463953556,
                                      "status": "success",
                                      "end": 1708463955268,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708463955270
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "23bb847a-0bab-5bba-3c33-ffcf6d1a905b"
                                  },
                                  "id": "1708463955271",
                                  "message": "backup VM",
                                  "start": 1708463955271,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708463955523",
                                      "message": "export",
                                      "start": 1708463955523,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708463955590",
                                          "message": "transfer",
                                          "start": 1708463955590,
                                          "status": "success",
                                          "end": 1708464380599,
                                          "result": {
                                            "size": 4886084605
                                          }
                                        }
                                      ],
                                      "end": 1708464381145
                                    },
                                    {
                                      "id": "1708464381146",
                                      "message": "clean-vm",
                                      "start": 1708464381146,
                                      "status": "success",
                                      "end": 1708464383740,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708464383741
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "2e700210-a790-d3a2-c5f2-add61872f6c7"
                                  },
                                  "id": "1708464383742",
                                  "message": "backup VM",
                                  "start": 1708464383742,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708464384216",
                                      "message": "export",
                                      "start": 1708464384216,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708464384330",
                                          "message": "transfer",
                                          "start": 1708464384330,
                                          "status": "success",
                                          "end": 1708464877412,
                                          "result": {
                                            "size": 2771349697
                                          }
                                        }
                                      ],
                                      "end": 1708464878252
                                    },
                                    {
                                      "id": "1708464878252:0",
                                      "message": "clean-vm",
                                      "start": 1708464878252,
                                      "status": "success",
                                      "end": 1708464880074,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708464880075
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "10af754b-59a2-b7fe-1e8d-0d8ebcaffd19"
                                  },
                                  "id": "1708463775484:0",
                                  "message": "backup VM",
                                  "start": 1708463775484,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708463776944",
                                      "message": "export",
                                      "start": 1708463776944,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708463777034",
                                          "message": "transfer",
                                          "start": 1708463777034,
                                          "status": "success",
                                          "end": 1708465107086,
                                          "result": {
                                            "size": 13894182527
                                          }
                                        }
                                      ],
                                      "end": 1708465108060
                                    },
                                    {
                                      "id": "1708465108060:0",
                                      "message": "clean-vm",
                                      "start": 1708465108060,
                                      "status": "success",
                                      "end": 1708465115441,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708465115442
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "5179c39c-6beb-c1af-5f8d-4ce7c58dcbd3"
                                  },
                                  "id": "1708465115443",
                                  "message": "backup VM",
                                  "start": 1708465115443,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708465116001",
                                      "message": "export",
                                      "start": 1708465116001,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708465116073",
                                          "message": "transfer",
                                          "start": 1708465116073,
                                          "status": "success",
                                          "end": 1708465354625,
                                          "result": {
                                            "size": 2961587877
                                          }
                                        }
                                      ],
                                      "end": 1708465355727
                                    },
                                    {
                                      "id": "1708465355728",
                                      "message": "clean-vm",
                                      "start": 1708465355728,
                                      "status": "success",
                                      "end": 1708465357308,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708465357309
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "793d1d93-ea37-f71a-8d07-faa396d1a6c5"
                                  },
                                  "id": "1708465357310",
                                  "message": "backup VM",
                                  "start": 1708465357310,
                                  "status": "success",
                                  "infos": [
                                    {
                                      "message": "No new data to upload for this VM"
                                    }
                                  ],
                                  "tasks": [
                                    {
                                      "id": "1708465357582:0",
                                      "message": "clean-vm",
                                      "start": 1708465357582,
                                      "status": "success",
                                      "end": 1708465357689,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708465357689
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "50151510-a3cd-ddfb-4d48-3c56e9b1e023"
                                  },
                                  "id": "1708464880075:0",
                                  "message": "backup VM",
                                  "start": 1708464880075,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708464880463",
                                      "message": "export",
                                      "start": 1708464880463,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708464880528",
                                          "message": "transfer",
                                          "start": 1708464880528,
                                          "status": "success",
                                          "end": 1708465668003,
                                          "result": {
                                            "size": 6745019923
                                          }
                                        }
                                      ],
                                      "end": 1708465669225
                                    },
                                    {
                                      "id": "1708465669226",
                                      "message": "clean-vm",
                                      "start": 1708465669226,
                                      "status": "success",
                                      "end": 1708465670770,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708465670770
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "f0078138-269e-daee-d893-9c9ef49308bb"
                                  },
                                  "id": "1708465670770:1",
                                  "message": "backup VM",
                                  "start": 1708465670770,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708465671074",
                                      "message": "export",
                                      "start": 1708465671074,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708465671127",
                                          "message": "transfer",
                                          "start": 1708465671127,
                                          "status": "success",
                                          "end": 1708465908023,
                                          "result": {
                                            "size": 3033684472
                                          }
                                        }
                                      ],
                                      "end": 1708465908410
                                    },
                                    {
                                      "id": "1708465908410:0",
                                      "message": "clean-vm",
                                      "start": 1708465908410,
                                      "status": "success",
                                      "end": 1708465910342,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708465910343
                                },
                                {
                                  "data": {
                                    "type": "VM",
                                    "id": "c94557ab-6096-2795-0fd8-e9ceb2b838aa"
                                  },
                                  "id": "1708465357689:1",
                                  "message": "backup VM",
                                  "start": 1708465357689,
                                  "status": "success",
                                  "tasks": [
                                    {
                                      "data": {
                                        "id": "a46a0888-17e1-45aa-8ca7-d6053fc3e55a",
                                        "type": "remote",
                                        "isFull": true
                                      },
                                      "id": "1708465358093",
                                      "message": "export",
                                      "start": 1708465358093,
                                      "status": "success",
                                      "tasks": [
                                        {
                                          "id": "1708465358190",
                                          "message": "transfer",
                                          "start": 1708465358190,
                                          "status": "success",
                                          "end": 1708467293577,
                                          "result": {
                                            "size": 17575842671
                                          }
                                        }
                                      ],
                                      "end": 1708467296188
                                    },
                                    {
                                      "id": "1708467296189",
                                      "message": "clean-vm",
                                      "start": 1708467296189,
                                      "status": "success",
                                      "end": 1708467298196,
                                      "result": {
                                        "merge": false
                                      }
                                    }
                                  ],
                                  "end": 1708467298198
                                }
                              ],
                              "end": 1708467298198
                            }
                            

                            Thanks,
                            Delgado

                            florentF 2 Replies Last reply Reply Quote 0
                            • florentF Offline
                              florent Vates 🪐 XO Team @Delgado
                              last edited by

                              @Delgado
                              That is not the intended result, I will rework the code

                              For Backup Mirror, at least for now, if no data are transferred there won't be a health check , and if some data are transferred, it will only health check the last backup

                              1 Reply Last reply Reply Quote 0
                              • florentF Offline
                                florent Vates 🪐 XO Team @Delgado
                                last edited by

                                @Delgado we pushed a new version of the code. Could you test it ?

                                D 2 Replies Last reply Reply Quote 1
                                • D Offline
                                  Delgado @florent
                                  last edited by

                                  @florent Yes, I will test and let you know.

                                  1 Reply Last reply Reply Quote 1
                                  • D Offline
                                    Delgado @florent
                                    last edited by Delgado

                                    @florent It looks like things are working as expected now. If there are changes that need to be backed up the vms run a health check now. If there are no changes no health check is ran. Thank you!

                                    1 Reply Last reply Reply Quote 1
                                    • First post
                                      Last post