XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup keep failing

    Scheduled Pinned Locked Moved Xen Orchestra
    11 Posts 3 Posters 811 Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • olivierlambertO Offline
      olivierlambert Vates 🪐 Co-Founder CEO
      last edited by

      Ping @nraynaud

      F 1 Reply Last reply Reply Quote 0
      • F Offline
        fred974 @olivierlambert
        last edited by

        @olivierlambert @nraynaud are you able to assist please?

        nraynaudN 1 Reply Last reply Reply Quote 0
        • nraynaudN Offline
          nraynaud XCP-ng Team @fred974
          last edited by

          @fred974 Hello Fred, I'm working on removing the timeout error, in the mean time you can remove it by increasing node memory by at least 2Gb.

          You can not stop the coalesce from happening, the only thing that can be done is planning the backup at a different time.

          I don't know what to do of the VDI_IO_ERROR.

          Nicolas.

          F 1 Reply Last reply Reply Quote 0
          • F Offline
            fred974 @nraynaud
            last edited by

            @nraynaud said in Backup keep failing:

            coalesce

            Hi,

            I have now removed all the backup jobs from XO and a few hours later, I can still see the same coalesce screen above.

            My understanding is that they meant to clear after a while

            F 1 Reply Last reply Reply Quote 0
            • F Offline
              fred974 @fred974
              last edited by fred974

              @fred974 Could anyone please assist me in resolving this Job canceled to protect the VDI chain error ?

              {
                "data": {
                  "mode": "delta",
                  "reportWhen": "failure"
                },
                "id": "1608677679049",
                "jobId": "c2d527c4-6032-4924-9ce5-1d37a564784f",
                "jobName": "daily_delta",
                "message": "backup",
                "scheduleId": "0b87e9de-6052-4e61-88b6-71345faf35c0",
                "start": 1608677679049,
                "status": "skipped",
                "tasks": [
                  {
                    "data": {
                      "type": "VM",
                      "id": "17a93294-e15c-485b-a3af-b7b3452e97c9"
                    },
                    "id": "1608677679081",
                    "message": "Starting backup of Axigen-Mail. (c2d527c4-6032-4924-9ce5-1d37a564784f)",
                    "start": 1608677679081,
                    "status": "skipped",
                    "end": 1608677679196,
                    "result": {
                      "VDI": {
                        "uuid": "28e6a067-db57-4246-bdc5-a5330ef37947",
                        "name_label": "srv-tri-axigen",
                        "name_description": "Axigen-Prod",
                        "allowed_operations": [
                          "snapshot",
                          "clone"
                        ],
                        "current_operations": {},
                        "SR": "OpaqueRef:24decb9d-aa2e-4615-b540-9279a4f2db26",
                        "VBDs": [
                          "OpaqueRef:5adb53e0-7e8e-41ef-b29e-3949c7150c96"
                        ],
                        "crash_dumps": [],
                        "virtual_size": 128849018880,
                        "physical_utilisation": 129109065728,
                        "type": "user",
                        "sharable": false,
                        "read_only": false,
                        "other_config": {},
                        "storage_lock": false,
                        "location": "28e6a067-db57-4246-bdc5-a5330ef37947",
                        "managed": true,
                        "missing": false,
                        "parent": "OpaqueRef:NULL",
                        "xenstore_data": {},
                        "sm_config": {
                          "vdi_type": "vhd",
                          "host_OpaqueRef:feda7d4a-e638-4f8d-b447-af1260c46964": "RW",
                          "read-caching-enabled-on-badb72a0-10a2-4c0a-b18c-832db6900e2c": "false",
                          "vhd-parent": "c65f53c3-2856-48ad-8636-fea72b9725b2",
                          "read-caching-reason-badb72a0-10a2-4c0a-b18c-832db6900e2c": "SR_NOT_SUPPORTED"
                        },
                        "is_a_snapshot": false,
                        "snapshot_of": "OpaqueRef:NULL",
                        "snapshots": [
                          "OpaqueRef:42c52287-3f4f-40b0-9d3a-65b1390c3453"
                        ],
                        "snapshot_time": "19700101T00:00:00Z",
                        "tags": [],
                        "allow_caching": false,
                        "on_boot": "persist",
                        "metadata_of_pool": "",
                        "metadata_latest": false,
                        "is_tools_iso": false,
                        "cbt_enabled": false
                      },
                      "VM": {
                        "uuid": "17a93294-e15c-485b-a3af-b7b3452e97c9",
                        "allowed_operations": [
                          "changing_dynamic_range",
                          "migrate_send",
                          "pool_migrate",
                          "changing_VCPUs_live",
                          "checkpoint",
                          "snapshot"
                        ],
                        "current_operations": {},
                        "name_label": "Axigen-Mail",
                        "name_description": "intIP 172.16.xxx.22 - extIP 193.12.xx.xx - 4vCPU, 8GB RAM, 120GB Disk",
                        "power_state": "Running",
                        "user_version": 1,
                        "is_a_template": false,
                        "is_default_template": false,
                        "suspend_VDI": "OpaqueRef:NULL",
                        "resident_on": "OpaqueRef:feda7d4a-e638-4f8d-b447-af1260c46964",
                        "affinity": "OpaqueRef:NULL",
                        "memory_overhead": 73400320,
                        "memory_target": 4294967296,
                        "memory_static_max": 8589934592,
                        "memory_dynamic_max": 8589934592,
                        "memory_dynamic_min": 4294967296,
                        "memory_static_min": 2147483648,
                        "VCPUs_params": {},
                        "VCPUs_max": 4,
                        "VCPUs_at_startup": 4,
                        "actions_after_shutdown": "destroy",
                        "actions_after_reboot": "restart",
                        "actions_after_crash": "restart",
                        "consoles": [
                          "OpaqueRef:c1bb56ac-0b51-4f5b-9647-74e09b895cbd"
                        ],
                        "VIFs": [
                          "OpaqueRef:f35121db-3295-49d1-9806-3751a083a227"
                        ],
                        "VBDs": [
                          "OpaqueRef:b0171b1e-11cd-4cb2-b749-cdafbcc40658",
                          "OpaqueRef:5adb53e0-7e8e-41ef-b29e-3949c7150c96"
                        ],
                        "VUSBs": [],
                        "crash_dumps": [],
                        "VTPMs": [],
                        "PV_bootloader": "",
                        "PV_kernel": "",
                        "PV_ramdisk": "",
                        "PV_args": "",
                        "PV_bootloader_args": "",
                        "PV_legacy_args": "",
                        "HVM_boot_policy": "BIOS order",
                        "HVM_boot_params": {
                          "firmware": "bios",
                          "order": "cdn"
                        },
                        "HVM_shadow_multiplier": 1,
                        "platform": {
                          "timeoffset": "0",
                          "videoram": "8",
                          "hpet": "true",
                          "secureboot": "false",
                          "device-model": "qemu-upstream-compat",
                          "apic": "true",
                          "device_id": "0001",
                          "vga": "std",
                          "nx": "true",
                          "pae": "true",
                          "viridian": "false",
                          "acpi": "1"
                        },
                        "PCI_bus": "",
                        "other_config": {
                          "auto_poweron": "true",
                          "base_template_name": "CentOS 8",
                          "import_task": "OpaqueRef:52440cd1-8e98-4a5d-9eef-6fc36dbdaff1",
                          "mac_seed": "06639988-0197-4394-b996-84cf1df88515",
                          "install-methods": "cdrom,nfs,http,ftp",
                          "linux_template": "true"
                        },
                        "domid": 2,
                        "domarch": "",
                        "last_boot_CPU_flags": {
                          "vendor": "GenuineIntel",
                          "features": "1fcbfbff-97ba2223-2d93fbff-00000403-00000001-00000000-00000000-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000"
                        },
                        "is_control_domain": false,
                        "metrics": "OpaqueRef:71e4bc16-feda-4c9d-9b40-e913217d2f4e",
                        "guest_metrics": "OpaqueRef:c8b03d4e-d91c-4dce-a72d-7183072c1f02",
                        "last_booted_record": "",
                        "recommendations": "<restrictions><restriction field=\"memory-static-max\" max=\"1649267441664\"/><restriction field=\"vcpus-max\" max=\"32\"/><restriction field=\"has-vendor-device\" value=\"false\"/><restriction field=\"allow-gpu-passthrough\" value=\"1\"/><restriction field=\"allow-vgpu\" value=\"1\"/><restriction field=\"allow-network-sriov\" value=\"1\"/><restriction field=\"supports-bios\" value=\"yes\"/><restriction field=\"supports-uefi\" value=\"no\"/><restriction field=\"supports-secure-boot\" value=\"no\"/><restriction max=\"255\" property=\"number-of-vbds\"/><restriction max=\"7\" property=\"number-of-vifs\"/></restrictions>",
                        "xenstore_data": {
                          "vm-data": "",
                          "vm-data/mmio-hole-size": "268435456"
                        },
                        "ha_always_run": false,
                        "ha_restart_priority": "",
                        "is_a_snapshot": false,
                        "snapshot_of": "OpaqueRef:NULL",
                        "snapshots": [
                          "OpaqueRef:c95fdceb-3295-452e-9012-ebb315165321"
                        ],
                        "snapshot_time": "19700101T00:00:00Z",
                        "transportable_snapshot_id": "",
                        "blobs": {},
                        "tags": [
                          "CentOS",
                          "Production",
                          "SSD"
                        ],
                        "blocked_operations": {
                          "hard_shutdown": "true",
                          "suspend": "true",
                          "pause": "true",
                          "clean_reboot": "true",
                          "shutdown": "true",
                          "hard_reboot": "true",
                          "clean_shutdown": "true",
                          "destroy": "true"
                        },
                        "snapshot_info": {},
                        "snapshot_metadata": "",
                        "parent": "OpaqueRef:c95fdceb-3295-452e-9012-ebb315165321",
                        "children": [],
                        "bios_strings": {
                          "bios-vendor": "Xen",
                          "bios-version": "",
                          "system-manufacturer": "Xen",
                          "system-product-name": "HVM domU",
                          "system-version": "",
                          "system-serial-number": "",
                          "baseboard-manufacturer": "",
                          "baseboard-product-name": "",
                          "baseboard-version": "",
                          "baseboard-serial-number": "",
                          "baseboard-asset-tag": "",
                          "baseboard-location-in-chassis": "",
                          "enclosure-asset-tag": "",
                          "hp-rombios": "",
                          "oem-1": "Xen",
                          "oem-2": "MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d"
                        },
                        "protection_policy": "OpaqueRef:NULL",
                        "is_snapshot_from_vmpp": false,
                        "snapshot_schedule": "OpaqueRef:NULL",
                        "is_vmss_snapshot": false,
                        "appliance": "OpaqueRef:NULL",
                        "start_delay": 0,
                        "shutdown_delay": 0,
                        "order": 0,
                        "VGPUs": [],
                        "attached_PCIs": [],
                        "suspend_SR": "OpaqueRef:NULL",
                        "version": 0,
                        "generation_id": "",
                        "hardware_platform_version": 0,
                        "has_vendor_device": false,
                        "requires_reboot": false,
                        "reference_label": "centos-8",
                        "domain_type": "hvm",
                        "NVRAM": {}
                      },
                      "message": "unhealthy VDI chain",
                      "name": "Error",
                      "stack": "Error: unhealthy VDI chain\n    at Xapi._assertHealthyVdiChain (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xapi/index.js:735:15)\n    at Xapi._assertHealthyVdiChain (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xapi/index.js:739:10)\n    at Xapi._assertHealthyVdiChain (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xapi/index.js:739:10)\n    at Xapi._assertHealthyVdiChain (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xapi/index.js:739:10)\n    at /opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xapi/index.js:746:14\n    at arrayEach (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/_arrayEach.js:15:9)\n    at forEach (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/forEach.js:38:10)\n    at Xapi._assertHealthyVdiChains (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xapi/index.js:744:5)\n    at BackupNg.apply (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xo-mixins/backups-ng/index.js:1119:14)\n    at BackupNg._backupVm (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/golike-defer/src/index.js:57:17)\n    at iteratee (/opt/xo/xo-builds/xen-orchestra-202022122000/packages/xo-server/src/xo-mixins/backups-ng/index.js:693:28)\n    at /opt/xo/xo-builds/xen-orchestra-202022122000/@xen-orchestra/async-map/src/index.js:32:17\n    at Promise._execute (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/bluebird/js/release/debuggability.js:384:9)\n    at Promise._resolveFromExecutor (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/bluebird/js/release/promise.js:518:18)\n    at new Promise (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/bluebird/js/release/promise.js:103:10)\n    at /opt/xo/xo-builds/xen-orchestra-202022122000/@xen-orchestra/async-map/src/index.js:31:7\n    at /opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/_baseMap.js:17:23\n    at /opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/_createBaseFor.js:17:11\n    at baseForOwn (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/_baseForOwn.js:13:20)\n    at /opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/_createBaseEach.js:17:14\n    at baseMap (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/_baseMap.js:16:3)\n    at map (/opt/xo/xo-builds/xen-orchestra-202022122000/node_modules/lodash/map.js:50:10)"
                    }
                  }
                ],
                "end": 1608677679197
              }
              
              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                Protecting the VDI chain is not an error, but… a protection.

                F 1 Reply Last reply Reply Quote 0
                • F Offline
                  fred974 @olivierlambert
                  last edited by

                  @olivierlambert said in Backup keep failing:

                  Protecting the VDI chain is not an error, but… a protection.

                  Hi,
                  Its a protection from what? How do I resolve this issue?
                  I am unable to perform any backup at all

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by olivierlambert

                    Have you looked at the link given when it's happening? This link will send you to our documentation.

                    Everything is explained here: https://xen-orchestra.com/docs/backup_troubleshooting.html#vdi-chain-protection

                    A good reflex is to always have the documentation near you, there's a search bar also. Really, it's the first step you might do always before.

                    F 1 Reply Last reply Reply Quote 0
                    • F Offline
                      fred974 @olivierlambert
                      last edited by

                      @olivierlambert I have read it but struggled with some of the English (my problem). I'll try to read it again.
                      My SR was almost full so I guess it couldn't run it to protect to get any bigger. I am moving VM away to free some space and see if this fix it

                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        Not enough space means not enough space to coalesce, and therefore XO will prevent to continue and fill entirely your storage because of uncoalesced chains.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post