Trouble doing Continuous Replication on pool (XOA)

Slawko

Hello,

I've been having trouble doing Continuous Replication on a second SR for a while now. xoa version latest xo-server 5.110.1.
the error message is shown below. Maybe some of you have already had such a case.
Backup jobs run normally without any problems. The issue relates to Continuous Replication.

{
  "data": {
    "mode": "delta",
    "reportWhen": "failure"
  },
  "id": "1678782947996",
  "jobId": "61b5e584-5192-4cd0-a61f-262dff82bab7",
  "jobName": "TEST",
  "message": "backup",
  "scheduleId": "4d078936-f382-44e2-8f32-4a9e2e12ae84",
  "start": 1678782947996,
  "status": "failure",
  "infos": [
    {
      "data": {
        "vms": [
          "d043653c-7acd-413b-a77b-f3d9ae8d23a1"
        ]
      },
      "message": "vms"
    }
  ],
  "tasks": [
    {
      "data": {
        "type": "VM",
        "id": "d043653c-7acd-413b-a77b-f3d9ae8d23a1"
      },
      "id": "1678782949528:0",
      "message": "backup VM",
      "start": 1678782949528,
      "status": "failure",
      "tasks": [
        {
          "id": "1678782949542",
          "message": "snapshot",
          "start": 1678782949542,
          "status": "success",
          "end": 1678782958398,
          "result": "86d10d13-b2ba-067f-3bfb-e22c217c4a68"
        },
        {
          "data": {
            "id": "b519b009-3a0c-7e26-ad0e-daf6744e4bac",
            "isFull": true,
            "type": "SR"
          },
          "id": "1678782958398:0",
          "message": "export",
          "start": 1678782958398,
          "status": "failure",
          "tasks": [
            {
              "id": "1678782961331",
              "message": "transfer",
              "start": 1678782961331,
              "status": "failure",
              "end": 1678782961338,
              "result": {
                "code": "INTERNAL_ERROR",
                "params": [
                  "(Failure \"Expected float, got 'I(1)'\")"
                ],
                "call": {
                  "method": "VM.create",
                  "params": [
                    {
                      "actions_after_crash": "restart",
                      "actions_after_reboot": "restart",
                      "actions_after_shutdown": "destroy",
                      "affinity": "OpaqueRef:NULL",
                      "HVM_boot_params": {
                        "order": "dc"
                      },
                      "HVM_boot_policy": "BIOS order",
                      "is_a_template": false,
                      "memory_dynamic_max": 4236247040,
                      "memory_dynamic_min": 4236247040,
                      "memory_static_max": 4236247040,
                      "memory_static_min": 134217728,
                      "other_config": {
                        "xo:backup:schedule": "4d078936-f382-44e2-8f32-4a9e2e12ae84",
                        "xo:backup:datetime": "20230314T08:35:58Z",
                        "xo:backup:vm": "d043653c-7acd-413b-a77b-f3d9ae8d23a1",
                        "xo:backup:job": "61b5e584-5192-4cd0-a61f-262dff82bab7",
                        "import_task": "OpaqueRef:ffa5b66e-47ec-401a-4113-0df14b885807",
                        "mac_seed": "b5b54844-68e7-b718-b241-d43ba8f4a53d",
                        "base_template_name": "Other install media",
                        "install-methods": "cdrom",
                        "xo:copy_of": "86d10d13-b2ba-067f-3bfb-e22c217c4a68"
                      },
                      "PCI_bus": "",
                      "platform": {
                        "timeoffset": "1",
                        "nx": "true",
                        "pae": "true",
                        "hpet": "true",
                        "apic": "true",
                        "viridian": "true",
                        "acpi": "1",
                        "cores-per-socket": "2"
                      },
                      "PV_args": "",
                      "PV_bootloader_args": "",
                      "PV_bootloader": "",
                      "PV_kernel": "",
                      "PV_legacy_args": "",
                      "PV_ramdisk": "",
                      "recommendations": "<restrictions><restriction field=\"memory-static-max\" max=\"137438953472\"/><restriction field=\"vcpus-max\" max=\"32\"/><restriction field=\"has-vendor-device\" value=\"false\"/><restriction max=\"255\" property=\"number-of-vbds\"/><restriction max=\"7\" property=\"number-of-vifs\"/></restrictions>",
                      "user_version": 1,
                      "VCPUs_at_startup": 2,
                      "VCPUs_max": 2,
                      "VCPUs_params": {},
                      "appliance": "OpaqueRef:NULL",
                      "blocked_operations": {
                        "start": "Importing…",
                        "start_on": "Importing…"
                      },
                      "generation_id": "",
                      "ha_restart_priority": "best-effort",
                      "hardware_platform_version": 0,
                      "has_vendor_device": false,
                      "HVM_shadow_multiplier": 1,
                      "is_vmss_snapshot": false,
                      "name_description": "Spark Serwer PROD",
                      "name_label": "[Importing…] [XO Backup TEST] SPARK SERWER",
                      "order": 0,
                      "reference_label": "other-install-media",
                      "shutdown_delay": 0,
                      "snapshot_schedule": "OpaqueRef:NULL",
                      "start_delay": 0,
                      "suspend_SR": "OpaqueRef:NULL",
                      "tags": [
                        "SPARK",
                        "Continuous Replication"
                      ],
                      "version": 0,
                      "xenstore_data": {
                        "vm-data": ""
                      },
                      "last_boot_CPU_flags": {
                        "vendor": "GenuineIntel",
                        "features": "17cbfbff-f7fa3223-2d93fbff-00000023-00000001-000007ab-00000000-00000000-00001000-0c000000"
                      },
                      "last_booted_record": "('struct' ('uuid' 'd043653c-7acd-413b-a77b-f3d9ae8d23a1') ('allowed_operations' ('array')) ('current_operations' ('struct' ('OpaqueRef:b5af07d5-88da-7dcd-a5ad-ea836568ab06' 'start'))) ('power_state' 'Halted') ('name_label' 'SPARK SERWER') ('name_description' 'Spark Serwer PROD') ('user_version' '1') ('is_a_template' ('boolean' '0')) ('is_default_template' ('boolean' '0')) ('suspend_VDI' 'OpaqueRef:NULL') ('resident_on' 'OpaqueRef:NULL') ('affinity' 'OpaqueRef:NULL') ('memory_overhead' '37748736') ('memory_target' '0') ('memory_static_max' '4236247040') ('memory_dynamic_max' '4236247040') ('memory_dynamic_min' '4236247040') ('memory_static_min' '134217728') ('VCPUs_params' ('struct')) ('VCPUs_max' '2') ('VCPUs_at_startup' '2') ('actions_after_shutdown' 'destroy') ('actions_after_reboot' 'restart') ('actions_after_crash' 'restart') ('consoles' ('array')) ('VIFs' ('array' 'OpaqueRef:fec4b5ed-718d-e27e-d6e3-73703c0f4606')) ('VBDs' ('array' 'OpaqueRef:9d80aacc-2412-b1c1-1470-77ea703b5930' 'OpaqueRef:c43cb461-0b2f-254e-ef58-2bcf66bfb982')) ('crash_dumps' ('array')) ('VTPMs' ('array')) ('PV_bootloader' '') ('PV_kernel' '') ('PV_ramdisk' '') ('PV_args' '') ('PV_bootloader_args' '') ('PV_legacy_args' '') ('HVM_boot_policy' 'BIOS order') ('HVM_boot_params' ('struct' ('order' 'dc'))) ('HVM_shadow_multiplier' ('double' '1')) ('platform' ('struct' ('timeoffset' '2') ('nx' 'true') ('pae' 'true') ('hpet' 'true') ('apic' 'true') ('viridian' 'true') ('acpi' '1') ('cores-per-socket' '2'))) ('PCI_bus' '') ('other_config' ('struct' ('import_task' 'OpaqueRef:ffa5b66e-47ec-401a-4113-0df14b885807') ('mac_seed' 'b5b54844-68e7-b718-b241-d43ba8f4a53d') ('base_template_name' 'Other install media') ('install-methods' 'cdrom'))) ('domid' '-1') ('domarch' '') ('last_boot_CPU_flags' ('struct' ('vendor' 'GenuineIntel') ('features' '17cbfbff-f7fa3223-2d93fbff-00000023-00000001-000007ab-00000000-00000000-00001000-0c000000'))) ('is_control_domain' ('boolean' '0')) ('metrics' 'OpaqueRef:021e0664-ba81-0a51-a230-97b487acb104') ('guest_metrics' 'OpaqueRef:442eb0d4-aef9-4667-6d28-df7be38a6460') ('last_booted_record' '') ('recommendations' '<restrictions><restriction field=\\\"memory-static-max\\\" max=\\\"137438953472\\\"/><restriction field=\\\"vcpus-max\\\" max=\\\"32\\\"/><restriction field=\\\"has-vendor-device\\\" value=\\\"false\\\"/><restriction max=\\\"255\\\" property=\\\"number-of-vbds\\\"/><restriction max=\\\"7\\\" property=\\\"number-of-vifs\\\"/></restrictions>') ('xenstore_data' ('struct' ('vm-data' ''))) ('ha_always_run' ('boolean' '0')) ('ha_restart_priority' 'best-effort') ('is_a_snapshot' ('boolean' '0')) ('snapshot_of' 'Ref:631') ('snapshots' ('array' 'OpaqueRef:c56fb04e-c6a6-19dd-becf-1c17e34f4bcf')) ('snapshot_time' ('dateTime.iso8601' '19700101T00:00:00Z')) ('transportable_snapshot_id' '') ('blobs' ('struct')) ('tags' ('array' 'SPARK')) ('blocked_operations' ('struct')) ('snapshot_info' ('struct' ('disk-snapshot-type' 'crash_consistent') ('power-state-at-snapshot' 'Running'))) ('snapshot_metadata' '') ('parent' 'OpaqueRef:c56fb04e-c6a6-19dd-becf-1c17e34f4bcf') ('children' ('array')) ('bios_strings' ('struct')) ('protection_policy' 'OpaqueRef:NULL') ('is_snapshot_from_vmpp' ('boolean' '0')) ('snapshot_schedule' 'OpaqueRef:NULL') ('is_vmss_snapshot' ('boolean' '0')) ('appliance' 'OpaqueRef:c40d424e-d17c-6519-08fe-1ce46bc37c0e') ('start_delay' '30') ('shutdown_delay' '0') ('order' '2') ('VGPUs' ('array')) ('attached_PCIs' ('array')) ('suspend_SR' 'OpaqueRef:NULL') ('version' '1') ('generation_id' '') ('hardware_platform_version' '0') ('has_vendor_device' ('boolean' '0')) ('requires_reboot' ('boolean' '0')) ('reference_label' 'other-install-media'))"
                    }
                  ]
                },
                "message": "INTERNAL_ERROR((Failure \"Expected float, got 'I(1)'\"))",
                "name": "XapiError",
                "stack": "XapiError: INTERNAL_ERROR((Failure \"Expected float, got 'I(1)'\"))\n    at XapiError.wrap (/opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/_XapiError.js:21:12)\n    at /opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/transports/json-rpc.js:36:30\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
              }
            }
          ],
          "end": 1678782961339,
          "result": {
            "code": "INTERNAL_ERROR",
            "params": [
              "(Failure \"Expected float, got 'I(1)'\")"
            ],
            "call": {
              "method": "VM.create",
              "params": [
                {
                  "actions_after_crash": "restart",
                  "actions_after_reboot": "restart",
                  "actions_after_shutdown": "destroy",
                  "affinity": "OpaqueRef:NULL",
                  "HVM_boot_params": {
                    "order": "dc"
                  },
                  "HVM_boot_policy": "BIOS order",
                  "is_a_template": false,
                  "memory_dynamic_max": 4236247040,
                  "memory_dynamic_min": 4236247040,
                  "memory_static_max": 4236247040,
                  "memory_static_min": 134217728,
                  "other_config": {
                    "xo:backup:schedule": "4d078936-f382-44e2-8f32-4a9e2e12ae84",
                    "xo:backup:datetime": "20230314T08:35:58Z",
                    "xo:backup:vm": "d043653c-7acd-413b-a77b-f3d9ae8d23a1",
                    "xo:backup:job": "61b5e584-5192-4cd0-a61f-262dff82bab7",
                    "import_task": "OpaqueRef:ffa5b66e-47ec-401a-4113-0df14b885807",
                    "mac_seed": "b5b54844-68e7-b718-b241-d43ba8f4a53d",
                    "base_template_name": "Other install media",
                    "install-methods": "cdrom",
                    "xo:copy_of": "86d10d13-b2ba-067f-3bfb-e22c217c4a68"
                  },
                  "PCI_bus": "",
                  "platform": {
                    "timeoffset": "1",
                    "nx": "true",
                    "pae": "true",
                    "hpet": "true",
                    "apic": "true",
                    "viridian": "true",
                    "acpi": "1",
                    "cores-per-socket": "2"
                  },
                  "PV_args": "",
                  "PV_bootloader_args": "",
                  "PV_bootloader": "",
                  "PV_kernel": "",
                  "PV_legacy_args": "",
                  "PV_ramdisk": "",
                  "recommendations": "<restrictions><restriction field=\"memory-static-max\" max=\"137438953472\"/><restriction field=\"vcpus-max\" max=\"32\"/><restriction field=\"has-vendor-device\" value=\"false\"/><restriction max=\"255\" property=\"number-of-vbds\"/><restriction max=\"7\" property=\"number-of-vifs\"/></restrictions>",
                  "user_version": 1,
                  "VCPUs_at_startup": 2,
                  "VCPUs_max": 2,
                  "VCPUs_params": {},
                  "appliance": "OpaqueRef:NULL",
                  "blocked_operations": {
                    "start": "Importing…",
                    "start_on": "Importing…"
                  },
                  "generation_id": "",
                  "ha_restart_priority": "best-effort",
                  "hardware_platform_version": 0,
                  "has_vendor_device": false,
                  "HVM_shadow_multiplier": 1,
                  "is_vmss_snapshot": false,
                  "name_description": "Spark Serwer PROD",
                  "name_label": "[Importing…] [XO Backup TEST] SPARK SERWER",
                  "order": 0,
                  "reference_label": "other-install-media",
                  "shutdown_delay": 0,
                  "snapshot_schedule": "OpaqueRef:NULL",
                  "start_delay": 0,
                  "suspend_SR": "OpaqueRef:NULL",
                  "tags": [
                    "SPARK",
                    "Continuous Replication"
                  ],
                  "version": 0,
                  "xenstore_data": {
                    "vm-data": ""
                  },
                  "last_boot_CPU_flags": {
                    "vendor": "GenuineIntel",
                    "features": "17cbfbff-f7fa3223-2d93fbff-00000023-00000001-000007ab-00000000-00000000-00001000-0c000000"
                  },
                  "last_booted_record": "('struct' ('uuid' 'd043653c-7acd-413b-a77b-f3d9ae8d23a1') ('allowed_operations' ('array')) ('current_operations' ('struct' ('OpaqueRef:b5af07d5-88da-7dcd-a5ad-ea836568ab06' 'start'))) ('power_state' 'Halted') ('name_label' 'SPARK SERWER') ('name_description' 'Spark Serwer PROD') ('user_version' '1') ('is_a_template' ('boolean' '0')) ('is_default_template' ('boolean' '0')) ('suspend_VDI' 'OpaqueRef:NULL') ('resident_on' 'OpaqueRef:NULL') ('affinity' 'OpaqueRef:NULL') ('memory_overhead' '37748736') ('memory_target' '0') ('memory_static_max' '4236247040') ('memory_dynamic_max' '4236247040') ('memory_dynamic_min' '4236247040') ('memory_static_min' '134217728') ('VCPUs_params' ('struct')) ('VCPUs_max' '2') ('VCPUs_at_startup' '2') ('actions_after_shutdown' 'destroy') ('actions_after_reboot' 'restart') ('actions_after_crash' 'restart') ('consoles' ('array')) ('VIFs' ('array' 'OpaqueRef:fec4b5ed-718d-e27e-d6e3-73703c0f4606')) ('VBDs' ('array' 'OpaqueRef:9d80aacc-2412-b1c1-1470-77ea703b5930' 'OpaqueRef:c43cb461-0b2f-254e-ef58-2bcf66bfb982')) ('crash_dumps' ('array')) ('VTPMs' ('array')) ('PV_bootloader' '') ('PV_kernel' '') ('PV_ramdisk' '') ('PV_args' '') ('PV_bootloader_args' '') ('PV_legacy_args' '') ('HVM_boot_policy' 'BIOS order') ('HVM_boot_params' ('struct' ('order' 'dc'))) ('HVM_shadow_multiplier' ('double' '1')) ('platform' ('struct' ('timeoffset' '2') ('nx' 'true') ('pae' 'true') ('hpet' 'true') ('apic' 'true') ('viridian' 'true') ('acpi' '1') ('cores-per-socket' '2'))) ('PCI_bus' '') ('other_config' ('struct' ('import_task' 'OpaqueRef:ffa5b66e-47ec-401a-4113-0df14b885807') ('mac_seed' 'b5b54844-68e7-b718-b241-d43ba8f4a53d') ('base_template_name' 'Other install media') ('install-methods' 'cdrom'))) ('domid' '-1') ('domarch' '') ('last_boot_CPU_flags' ('struct' ('vendor' 'GenuineIntel') ('features' '17cbfbff-f7fa3223-2d93fbff-00000023-00000001-000007ab-00000000-00000000-00001000-0c000000'))) ('is_control_domain' ('boolean' '0')) ('metrics' 'OpaqueRef:021e0664-ba81-0a51-a230-97b487acb104') ('guest_metrics' 'OpaqueRef:442eb0d4-aef9-4667-6d28-df7be38a6460') ('last_booted_record' '') ('recommendations' '<restrictions><restriction field=\\\"memory-static-max\\\" max=\\\"137438953472\\\"/><restriction field=\\\"vcpus-max\\\" max=\\\"32\\\"/><restriction field=\\\"has-vendor-device\\\" value=\\\"false\\\"/><restriction max=\\\"255\\\" property=\\\"number-of-vbds\\\"/><restriction max=\\\"7\\\" property=\\\"number-of-vifs\\\"/></restrictions>') ('xenstore_data' ('struct' ('vm-data' ''))) ('ha_always_run' ('boolean' '0')) ('ha_restart_priority' 'best-effort') ('is_a_snapshot' ('boolean' '0')) ('snapshot_of' 'Ref:631') ('snapshots' ('array' 'OpaqueRef:c56fb04e-c6a6-19dd-becf-1c17e34f4bcf')) ('snapshot_time' ('dateTime.iso8601' '19700101T00:00:00Z')) ('transportable_snapshot_id' '') ('blobs' ('struct')) ('tags' ('array' 'SPARK')) ('blocked_operations' ('struct')) ('snapshot_info' ('struct' ('disk-snapshot-type' 'crash_consistent') ('power-state-at-snapshot' 'Running'))) ('snapshot_metadata' '') ('parent' 'OpaqueRef:c56fb04e-c6a6-19dd-becf-1c17e34f4bcf') ('children' ('array')) ('bios_strings' ('struct')) ('protection_policy' 'OpaqueRef:NULL') ('is_snapshot_from_vmpp' ('boolean' '0')) ('snapshot_schedule' 'OpaqueRef:NULL') ('is_vmss_snapshot' ('boolean' '0')) ('appliance' 'OpaqueRef:c40d424e-d17c-6519-08fe-1ce46bc37c0e') ('start_delay' '30') ('shutdown_delay' '0') ('order' '2') ('VGPUs' ('array')) ('attached_PCIs' ('array')) ('suspend_SR' 'OpaqueRef:NULL') ('version' '1') ('generation_id' '') ('hardware_platform_version' '0') ('has_vendor_device' ('boolean' '0')) ('requires_reboot' ('boolean' '0')) ('reference_label' 'other-install-media'))"
                }
              ]
            },
            "message": "INTERNAL_ERROR((Failure \"Expected float, got 'I(1)'\"))",
            "name": "XapiError",
            "stack": "XapiError: INTERNAL_ERROR((Failure \"Expected float, got 'I(1)'\"))\n    at XapiError.wrap (/opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/_XapiError.js:21:12)\n    at /opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/transports/json-rpc.js:36:30\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
          }
        }
      ],
      "end": 1678782968567,
      "result": {
        "code": "INTERNAL_ERROR",
        "params": [
          "(Failure \"Expected float, got 'I(1)'\")"
        ],
        "call": {
          "method": "VM.create",
          "params": [
            {
              "actions_after_crash": "restart",
              "actions_after_reboot": "restart",
              "actions_after_shutdown": "destroy",
              "affinity": "OpaqueRef:NULL",
              "HVM_boot_params": {
                "order": "dc"
              },
              "HVM_boot_policy": "BIOS order",
              "is_a_template": false,
              "memory_dynamic_max": 4236247040,
              "memory_dynamic_min": 4236247040,
              "memory_static_max": 4236247040,
              "memory_static_min": 134217728,
              "other_config": {
                "xo:backup:schedule": "4d078936-f382-44e2-8f32-4a9e2e12ae84",
                "xo:backup:datetime": "20230314T08:35:58Z",
                "xo:backup:vm": "d043653c-7acd-413b-a77b-f3d9ae8d23a1",
                "xo:backup:job": "61b5e584-5192-4cd0-a61f-262dff82bab7",
                "import_task": "OpaqueRef:ffa5b66e-47ec-401a-4113-0df14b885807",
                "mac_seed": "b5b54844-68e7-b718-b241-d43ba8f4a53d",
                "base_template_name": "Other install media",
                "install-methods": "cdrom",
                "xo:copy_of": "86d10d13-b2ba-067f-3bfb-e22c217c4a68"
              },
              "PCI_bus": "",
              "platform": {
                "timeoffset": "1",
                "nx": "true",
                "pae": "true",
                "hpet": "true",
                "apic": "true",
                "viridian": "true",
                "acpi": "1",
                "cores-per-socket": "2"
              },
              "PV_args": "",
              "PV_bootloader_args": "",
              "PV_bootloader": "",
              "PV_kernel": "",
              "PV_legacy_args": "",
              "PV_ramdisk": "",
              "recommendations": "<restrictions><restriction field=\"memory-static-max\" max=\"137438953472\"/><restriction field=\"vcpus-max\" max=\"32\"/><restriction field=\"has-vendor-device\" value=\"false\"/><restriction max=\"255\" property=\"number-of-vbds\"/><restriction max=\"7\" property=\"number-of-vifs\"/></restrictions>",
              "user_version": 1,
              "VCPUs_at_startup": 2,
              "VCPUs_max": 2,
              "VCPUs_params": {},
              "appliance": "OpaqueRef:NULL",
              "blocked_operations": {
                "start": "Importing…",
                "start_on": "Importing…"
              },
              "generation_id": "",
              "ha_restart_priority": "best-effort",
              "hardware_platform_version": 0,
              "has_vendor_device": false,
              "HVM_shadow_multiplier": 1,
              "is_vmss_snapshot": false,
              "name_description": "Spark Serwer PROD",
              "name_label": "[Importing…] [XO Backup TEST] SPARK SERWER",
              "order": 0,
              "reference_label": "other-install-media",
              "shutdown_delay": 0,
              "snapshot_schedule": "OpaqueRef:NULL",
              "start_delay": 0,
              "suspend_SR": "OpaqueRef:NULL",
              "tags": [
                "SPARK",
                "Continuous Replication"
              ],
              "version": 0,
              "xenstore_data": {
                "vm-data": ""
              },
              "last_boot_CPU_flags": {
                "vendor": "GenuineIntel",
                "features": "17cbfbff-f7fa3223-2d93fbff-00000023-00000001-000007ab-00000000-00000000-00001000-0c000000"
              },
              "last_booted_record": "('struct' ('uuid' 'd043653c-7acd-413b-a77b-f3d9ae8d23a1') ('allowed_operations' ('array')) ('current_operations' ('struct' ('OpaqueRef:b5af07d5-88da-7dcd-a5ad-ea836568ab06' 'start'))) ('power_state' 'Halted') ('name_label' 'SPARK SERWER') ('name_description' 'Spark Serwer PROD') ('user_version' '1') ('is_a_template' ('boolean' '0')) ('is_default_template' ('boolean' '0')) ('suspend_VDI' 'OpaqueRef:NULL') ('resident_on' 'OpaqueRef:NULL') ('affinity' 'OpaqueRef:NULL') ('memory_overhead' '37748736') ('memory_target' '0') ('memory_static_max' '4236247040') ('memory_dynamic_max' '4236247040') ('memory_dynamic_min' '4236247040') ('memory_static_min' '134217728') ('VCPUs_params' ('struct')) ('VCPUs_max' '2') ('VCPUs_at_startup' '2') ('actions_after_shutdown' 'destroy') ('actions_after_reboot' 'restart') ('actions_after_crash' 'restart') ('consoles' ('array')) ('VIFs' ('array' 'OpaqueRef:fec4b5ed-718d-e27e-d6e3-73703c0f4606')) ('VBDs' ('array' 'OpaqueRef:9d80aacc-2412-b1c1-1470-77ea703b5930' 'OpaqueRef:c43cb461-0b2f-254e-ef58-2bcf66bfb982')) ('crash_dumps' ('array')) ('VTPMs' ('array')) ('PV_bootloader' '') ('PV_kernel' '') ('PV_ramdisk' '') ('PV_args' '') ('PV_bootloader_args' '') ('PV_legacy_args' '') ('HVM_boot_policy' 'BIOS order') ('HVM_boot_params' ('struct' ('order' 'dc'))) ('HVM_shadow_multiplier' ('double' '1')) ('platform' ('struct' ('timeoffset' '2') ('nx' 'true') ('pae' 'true') ('hpet' 'true') ('apic' 'true') ('viridian' 'true') ('acpi' '1') ('cores-per-socket' '2'))) ('PCI_bus' '') ('other_config' ('struct' ('import_task' 'OpaqueRef:ffa5b66e-47ec-401a-4113-0df14b885807') ('mac_seed' 'b5b54844-68e7-b718-b241-d43ba8f4a53d') ('base_template_name' 'Other install media') ('install-methods' 'cdrom'))) ('domid' '-1') ('domarch' '') ('last_boot_CPU_flags' ('struct' ('vendor' 'GenuineIntel') ('features' '17cbfbff-f7fa3223-2d93fbff-00000023-00000001-000007ab-00000000-00000000-00001000-0c000000'))) ('is_control_domain' ('boolean' '0')) ('metrics' 'OpaqueRef:021e0664-ba81-0a51-a230-97b487acb104') ('guest_metrics' 'OpaqueRef:442eb0d4-aef9-4667-6d28-df7be38a6460') ('last_booted_record' '') ('recommendations' '<restrictions><restriction field=\\\"memory-static-max\\\" max=\\\"137438953472\\\"/><restriction field=\\\"vcpus-max\\\" max=\\\"32\\\"/><restriction field=\\\"has-vendor-device\\\" value=\\\"false\\\"/><restriction max=\\\"255\\\" property=\\\"number-of-vbds\\\"/><restriction max=\\\"7\\\" property=\\\"number-of-vifs\\\"/></restrictions>') ('xenstore_data' ('struct' ('vm-data' ''))) ('ha_always_run' ('boolean' '0')) ('ha_restart_priority' 'best-effort') ('is_a_snapshot' ('boolean' '0')) ('snapshot_of' 'Ref:631') ('snapshots' ('array' 'OpaqueRef:c56fb04e-c6a6-19dd-becf-1c17e34f4bcf')) ('snapshot_time' ('dateTime.iso8601' '19700101T00:00:00Z')) ('transportable_snapshot_id' '') ('blobs' ('struct')) ('tags' ('array' 'SPARK')) ('blocked_operations' ('struct')) ('snapshot_info' ('struct' ('disk-snapshot-type' 'crash_consistent') ('power-state-at-snapshot' 'Running'))) ('snapshot_metadata' '') ('parent' 'OpaqueRef:c56fb04e-c6a6-19dd-becf-1c17e34f4bcf') ('children' ('array')) ('bios_strings' ('struct')) ('protection_policy' 'OpaqueRef:NULL') ('is_snapshot_from_vmpp' ('boolean' '0')) ('snapshot_schedule' 'OpaqueRef:NULL') ('is_vmss_snapshot' ('boolean' '0')) ('appliance' 'OpaqueRef:c40d424e-d17c-6519-08fe-1ce46bc37c0e') ('start_delay' '30') ('shutdown_delay' '0') ('order' '2') ('VGPUs' ('array')) ('attached_PCIs' ('array')) ('suspend_SR' 'OpaqueRef:NULL') ('version' '1') ('generation_id' '') ('hardware_platform_version' '0') ('has_vendor_device' ('boolean' '0')) ('requires_reboot' ('boolean' '0')) ('reference_label' 'other-install-media'))"
            }
          ]
        },
        "message": "INTERNAL_ERROR((Failure \"Expected float, got 'I(1)'\"))",
        "name": "XapiError",
        "stack": "XapiError: INTERNAL_ERROR((Failure \"Expected float, got 'I(1)'\"))\n    at XapiError.wrap (/opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/_XapiError.js:21:12)\n    at /opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/transports/json-rpc.js:36:30\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
      }
    }
  ],
  "end": 1678782968568
}

olivierlambert

Hi,

Can you give more details on the host used? XCP-ng or XenServer? Which version?

Slawko

@olivierlambert

Hello

I am currently using xcp-ng 8.2.1 with the latest patches. besides, i have another xen server 7.2 pool where i have the same case noted. Everything was working fine until pool master crashed and I had to restart it manually.

olivierlambert

And you have the error from which host to which host?

Slawko

@Slawko The above error is from xoa from the continuous replication task log.
I actually have three servers in the pool. After the crash (hang), I did nothing with the other two servers. Virtual machines that work on node 2,3 work normally. Backup tasks are performed normally on them.

xoa error 2.jpg

olivierlambert

What I meant: you are replicating from which host (XS/XCP, which version) to which host?

On this host, the transfer seems to fail, but it's interesting to lean where this error is coming from: source or destination, and XS or XCP-ng?

Slawko

@Slawko The XOA server is located on node 3, while the replication task concerns (CR) virtual machines located on node 1,2,3. Node 1 is (pool master). Now i looked every continuous replication task ends with an error as in the attached picture.

olivierlambert

Let me rephrase:

Where XOA is based doesn't matter
What matters is in which way you are replicating a VM: from which XCP-ng/XenServer to which XCP-ng/XenServer. Which one is source, which one is destination
I can assume you have different pools, right? (not everything is within the same pool)

Slawko

@Slawko I've already explained the problem.

As I mentioned, a xen server pool is made of three nodes. Two Storage (SR) on iscsi are connected to this pool. On the first basic SR1 there are virtual machines spread over three nodes. The problem is to set the continuous replication task so that the task executes correctly on the second SR2 (failover storage). I have not had such a problem with the environment so far. To sum up, the CR task is to replicate the machine from SR1 to SR2. :-). There is an attempt to take a snapshot on SR1 and then the task ends with an error.

AtaxyaNetwork

@Slawko Hi !

To help you debug your problem, we need the version of your XCP-ng or XenServer host where the VMs are located. I see on the screen "XEN-7-2", so I suppose it's a XenServer version 7.2, it's right ?

Since you screen says "This is a XenServer/XCP-ng error", you may find something in host's logs, for example /var/log/xensource.log, /var/log/SMlog or dmesg -T

Slawko

@Slawko Hi, This is log on xoaXapiError: INTERNAL_ERROR((Failure "Expected float, got 'I(1)'"))\n at XapiError.wrap (/opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/_XapiError.js:21:12)\n at /opt/xo/xo-builds/xen-orchestra-202303140806/packages/xen-api/dist/transports/json-rpc.js:36:30\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Slawko

Mar 15 10:22:48 XCP-NG-RBS

xapi: [debug||52793 ||storage_access] vdi = {"sm_config":{"read-caching-reason-c9cc0d97-9c63-4cb9-b5b8-9edf3bb8c640":"SR_NOT_SUPPORTED","read-caching-reason-ede532bc-bb60-4b7e-8730-0ccbc9a7fa83":"LICENSE_RESTRICTION","vdi_type":"vhd","vmhint":"6981c560-051c-b63a-5bdb-352ff09ee0a7","vhd-blocks":"eJztV39wFNUd/7zdzd1estztJRAuISWbBPAQZnogLQem5iUEeoMIh3QUNcqR8qvQwvGjDgrk3oUjBKQSiiMIlDmUsWCZ8QRtFZ1hCRRxwJI6FEeg0wNtkaGUgIOJEti+3Q0kMZna6bTTztTvH3v33vv+/vW+D+gByPVU/j2um5fyxdLWlfW73578gU9YNu8vi99ZfGBrkIRGbfLohc9fn0d+2ZL3q6Ua7qW9ofuk6kmv/i1W9uSG95vOVJQX9p1Vj1kzn8ksmnMs/ekTk1YypiKiIEsXi0NCGioETKEDEW4yBWoq0tBIdYPAgL4vxBZFpxbkTUqCsLHxsck8H4toUuVwJg6lFPoAlaApAARkyA5L3ydmwHX3h6c2Y7BhGM0ve186+IubVdTDF0ZNcLLgqwwvPFOe4Ly1eBV1R9YAbldVOdBYrEXKPw/nMGTTg69y0a5ai5/KNYKgE9VVAL6rTpfVrg5S3ZCL0j25zoQ1rVArva0Fjf0nHmY+RKdwW02wlY1AmACvMA9ufeCqKXuoDFG3DgosTziphSYBwaXISSNbg05VJgPyLMbgB+mQq3JagEmQotaaI4lM1RBGbezLdANVRqiDf3PoBCUXCg3jMuWfNg/bo6ZUhZL1Mvo5aBSkk+IB8yMQwT1RAbwaX1BT39ugqUJXQ5mFwNRSM2e4i4i1RnaIvWHYGOLVCBxRDXWdqGRaZJIKZYBtOP96HPdtOnzR9WnJfHr2Ime7vUIe1L9yiYZd0JmmsGGWZjOZN7lDToo+SdOEmVWZFcUIJhSYyupSH9P8wPjafhs+Cgo7D4KVkrdABB9GrcIPwUhKYr3znB7sP6G3TmvCyJBy+v4BavTeJj9ZH1v54vBmbwbY2KaHBLe+8tiH6mvN3x1cz8SaXr0mVAoYufLGIaaFkMV682CwsrQ4Z7Rjc23S8kJYGYgoChKekJ/JAmS9SCILhrgJcuP3ROrlz/fOlxoShCeBjNJ9+eOX77+Zip15f1FUuX+cqMdY+fG3P1qPa0+RZU0uadcfP5maczLz5aKRQ5VK0guj2iRQlZy/1nJ+4sZlZwoIPE0fb42da4UsYep7mwfVeq46r7B5O3d/dsJXX9eQ798DcfcMbZJQqfykpeLjyE/pY48WP/1IqObh/Q639q22FWvmAooyc53uRjbXvh5UkukWnkfhJRjnP/xo+fIpjh0J0Y6VV+oScK3nhBe64TSIESoJPMrn1G7YNsv2XBG+eprsshq8LW9rriTLWHvfBzOZc3uI3Sytao6lx/y55UqapEY10scO/O7ho59dWv7AtfW29EhARcpIg8z5/gvRxovZqIC43YGSV5y8fCyBEjY3wH2k//eEl55rAvPy/GkdNJIMx3EU0KdjmRgBTeacMh7XCsfv/X3b2ThozRCOJRZWxTnlkCR+xntFLuRMsrSkJp2MIDfqTXgur5Cji546I0oZ8bpUTvLSxnwvI2x0oSm2gVqmx3SIzW7m+PaTzuG2hSVjevZpBzh5U3o2cGUzsreZS/Z1+P8OOL/2sPTj43A2OjGs9PnJubiu5RaF115wD4KqyqvyWPCRzbmn76q7LJHiS6xQqQh9hYFAkj/v9SWRyk6/+ThFRq1atcHclrIi5o9DwuJIHUi1uVKJLF0AeifYFo4gwPHGzhRESS5c0rwxnoLdkiEKDmqZruKouotEhDmqelRuLTrZSOwUI516jIrX4aLm5nXewZVneWcgvOtLediQCFENlI5iPA3SVOXN5z5kyXFnsZ9VnoOLK9RPUkwDNCAh/TobqmDIJ6rrtSanZmWuum+BW/Cf44FwF3JWZhprtb6wZioXHj0Z+asOBeSCVC/SGEf5waPrcubAQqpNM1s52VZxHJSMh4jD1FJYBz2iCXarldSGLPgdYnIAKnmwWQ5z6GShb4aDtkpWcam5GtiD8FndmBhnDKMldr4MSps7Eoj7TBbaDuDgBEizF9DVx6S+p/S9mbp77BIueEtccz2QIFWserrOaG/CPWFpQ9vrMKR5Wt5hDrH/0U80Hmm34n+w1lli3khrXoHlZ2IS3CWOXPGmfzqb6hDzqbzB3YwJNPvFYYwzdPLjjHoQ+dAXxVALxVa8t+pgYNKeP2TmZJ8nt/4aUFDTHlJ4bplfZWeReY3JPWcikbYdYXNxMmcbvyZtD4YIpO7YrGf6/yOgoYicxQMpnneyaAiEtAJHQTWKEvO4noizY0LS7LxVmizoUgWQaRHKIOt1n84pMxEm9RlJBBJ8IMt0ZcRH3F13g4UTfG5TCso0Prss9JLnqElUKdHbgp302lhK+pzqwyAmIfhyWZ3E+L7gRoz/6AOKYBdpiS7yfTGR38jZD0DQx0eVDOYgRVuIqmAHrxJeYqrg5ajk8IR27n0TJhGUQWx+dLF86DtcCEczcWCkYN0nGmVkWpq8xiv0LbNGBBFTqvFueYqXdzAicYvrs0wPOILAGHnMBEwzDhVLNi2kOJTXhXePD1XN+UdAmGp8HlWFTeIX+JHMs15zNcAaCpExMm3mGd+M8NngmVZTzxtB3pd0czOim7OM7ZaI5B+zLjOkWxbE6RF+7hOpmBnAnUxn+p3IxViMmfNh4QH4peqOiDqT5eWJ1QGulAaFoUDgbrwzsNnul8bl/VbSO5VDMNsbL2EgzebC3yGjHVrhvEqayi1zyHZerWxOHSgx8cgPbBSXJaLTYEi7srgV7PhvzwkBYrChw2eXdLvN7fu9enW6fdmup5iwypXk2241Iczl9eMtx3wZtMsjaxxdeInGlZpo2eXmbkJsHe/4QLujvdEudzysx4BhsB5pObhifzJajJtnd+0zrhhGjyga6bbVm6xr0zI6W/YvgFSt1nasyJ2JSyOJrvItEe1PCS9v3Z627uMTB8VUxjzQuFd7lNh9JvsnINZg9tiYzruGU7vNwXy6pBO20H8A9SPIXC7V27NjO4PtRv4gSwX4lMqcbWZpVbSHU7VfI6LRAemyTtEi6fax1cmsF8btmJgupT3L87CuayXYI5oNtwV/rRX/M8D+2wp8A9/AfxL+DtAVkR4=","import_task":"OpaqueRef:7fb03bbd-89ef-437b-a108-13d289759a7c","read-caching-enabled-on-ede532bc-bb60-4b7e-8730-0ccbc9a7fa83":"false","read-caching-enabled-on-c9cc0d97-9c63-4cb9-b5b8-9edf3bb8c640":"false","vhd-parent":"edf55f02-4709-461f-b6cc-67bcf8f59fd9","read-caching-enabled-on-ed7c7db9-0a3a-48d4-a5d3-ccf23285d02d":"false","read-caching-reason-ed7c7db9-0a3a-48d4-a5d3-ccf23285d02d":"LICENSE_RESTRICTION"},"sharable":false,"persistent":true,"physical_utilisation":8388608,"virtual_size":80530636800,"cbt_enabled":false,"read_only":false,"snapshot_of":"","snapshot_time":"20230315T09:22:48Z","is_a_snapshot":true,"metadata_of_pool":"OpaqueRef:NULL","ty":"user","name_description":"Win2012R2-RDP-RBSMEDIA","name_label":"Win2012R2-RDP-RBSMEDIA","content_id":"3b7fd1c1-c57c-b968-5216-e23cf227d13a","uuid":"3fcdb648-c842-4044-9299-059645c9e5de","vdi":"3fcdb648-c842-4044-9299-059645c9e5de"}
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52792 |Async.VDI.snapshot R:f39df52b2bcd|message_forwarding] Unmarking SR after VDI.snapshot (task=OpaqueRef:f39df52b-2bcd-46c8-a6eb-0de9347466b3)
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52786 HTTPS 10.10.80.125->|Async.VM.snapshot R:cfd142144a7c|xapi_vm_clone] Finished listening for events relating to tasks OpaqueRef:f39df52b-2bcd-46c8-a6eb-0de9347466b3 and OpaqueRef:cfd14214-4a7c-4398-9d59-364ffa38c3d8
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52897 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:446fd1d9f50e created by task R:cfd142144a7c
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52897 /var/lib/xcp/xapi|session.logout D:0ed95c69b897|xapi_session] Session.destroy trackid=1a280841dee81a1b99989f12f9567373
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52898 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:task.destroy D:95f269f39322 created by task R:cfd142144a7c
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52786 HTTPS 10.10.80.125->|Async.VM.snapshot R:cfd142144a7c|xapi_vif_helpers] VIF.create running
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52786 HTTPS 10.10.80.125->|Async.VM.snapshot R:cfd142144a7c|xapi_vif_helpers] Found mac_seed on VM: supplied MAC parameter = '26:84:3d:be:8b:fe'
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52786 HTTPS 10.10.80.125->|Async.VM.snapshot R:cfd142144a7c|xapi_vif_helpers] VIF ref='OpaqueRef:499dab34-f8ca-406d-864d-9a8a3776a68d' created (VM = 'OpaqueRef:4e27431a-ce9f-4b88-a7eb-8302fd6ccf7d'; MAC address = '26:84:3d:be:8b:fe')
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52899 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.slave_login D:d09a1a974a4a created by task R:cfd142144a7c
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52899 /var/lib/xcp/xapi|session.slave_login D:3cc6910ecc55|xapi_session] Session.create trackid=c699d446b6fd09dc6ee06d5bf5b31763 pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52900 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:10e286f4f51e created by task D:3cc6910ecc55
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52901 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:5eee44303a94 created by task R:cfd142144a7c
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52901 /var/lib/xcp/xapi|session.logout D:27aa9f9cc6cc|xapi_session] Session.destroy trackid=c699d446b6fd09dc6ee06d5bf5b31763
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52902 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:fd98c79705dc created by task R:cfd142144a7c
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52902 /var/lib/xcp/xapi|session.logout D:675762502c4f|xapi_session] Session.destroy trackid=d4510ed7ebb3b1dccf3363ed8bf2f4ed
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52786 HTTPS 10.10.80.125->|Async.VM.snapshot R:cfd142144a7c|xapi_vm_snapshot] Snapshot: end
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||149 ||xenops] Event on VM 67af8ff9-b3db-efad-b961-22ed70ded7c7; resident_here = true
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52903 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:167d7020c152 created by task D:4564f5827955
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52903 /var/lib/xcp/xapi|session.logout D:424cda782a18|xapi_session] Session.destroy trackid=939bdb67b013faa77c0892039093d6b6
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||149 ||xenops] xapi_cache: not updating cache for 67af8ff9-b3db-efad-b961-22ed70ded7c7
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||149 ||dummytaskhelper] task timeboxed_rpc D:23e83957f09b created by task D:e0c8577e4f97
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52904 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:87ed5bc5dbc3 created by task D:e0c8577e4f97
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||151 |watching networks for NBD-related changes D:4564f5827955|network_event_loop] Not updating the firewall, because the set of interfaces to use for NBD did not change: []
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52908 HTTPS 10.10.80.125->:::80|VM.set_is_a_template D:0134a4ae37b1|xapi_vm_helpers] VM.set_is_a_template('false')
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52909 :::80||api_effect] VM.add_to_other_config
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52910 :::80||api_effect] VM.add_to_other_config
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52911 :::80||api_effect] VM.add_to_other_config
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52912 :::80||api_effect] VM.add_to_other_config
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52915 HTTPS 10.10.80.125->:::80|task.create D:8b3e565b0a9b|taskhelper] task [XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33 (uuid:4230f702-c4c6-11bf-5484-6e4ca9e57b3a) created (trackid=9eef03adcd8dff55421459f480b02d0e) by task D:8b3e565b0a9b
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52916 :::80||export_raw_vdi] export_raw_vdi handler
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52917 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.slave_login D:95b6d60ad056 created by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52917 /var/lib/xcp/xapi|session.slave_login D:40cedd4ed96c|xapi_session] Session.create trackid=35144f0276fe901ef61f755f3ec26890 pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52918 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:ac6cc5c3d568 created by task D:40cedd4ed96c
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52916 :::80|VDI.export_raw_vdi D:281f29dbfec1|export_raw_vdi] Checking whether localhost can see SR: OpaqueRef:105b9c29-e9c6-4d34-aafd-560a99f401c5
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52916 :::80||taskhelper] task [XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33 forwarded (trackid=9eef03adcd8dff55421459f480b02d0e)
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|export_raw_vdi] export_raw_vdi task_id = OpaqueRef:317fdffb-9f33-4916-8ba5-bccc2e51a0e4; vdi = OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd; format = vhd; content-type = application/vhd; filename = 3fcdb648-c842-4044-9299-059645c9e5de.vhd
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52919 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VBD.create D:4c6642a8143f created by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52919 /var/lib/xcp/xapi||taskhelper] task VBD.create R:d1c4f8f552eb (uuid:953a8b24-3ec7-2a87-a4f4-8cc4b571d098) created (trackid=35144f0276fe901ef61f755f3ec26890) by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52919 /var/lib/xcp/xapi|VBD.create R:d1c4f8f552eb|audit] VBD.create: VM = 'c73b8859-be4a-4f60-b7b4-113c8fa9ca61 (Control domain on host: XCP-NG-RBS)'; VDI = '3fcdb648-c842-4044-9299-059645c9e5de'
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52919 /var/lib/xcp/xapi|VBD.create R:d1c4f8f552eb|vbdops] Checking whether there's a migrate in progress...
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52919 /var/lib/xcp/xapi|VBD.create R:d1c4f8f552eb|vbdops] VBD.create (device = 1; uuid = 7ebef030-db54-bdab-6c2a-6560f6438436; ref = OpaqueRef:2276a35b-0b97-4d78-a9d8-020462c0b724)
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52920 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VBD.get_uuid D:72394821bf27 created by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52921 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VM.get_uuid D:d9bb724b9eba created by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|attach_helpers] created VBD (uuid 7ebef030-db54-bdab-6c2a-6560f6438436); attempting to hotplug to VM (uuid: c73b8859-be4a-4f60-b7b4-113c8fa9ca61)
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52922 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VBD.plug D:7769dd48c42a created by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52922 /var/lib/xcp/xapi||taskhelper] task VBD.plug R:8c9b9ea1a36d (uuid:0007b8f2-bd84-7ee3-ed5b-1cea514569b4) created (trackid=35144f0276fe901ef61f755f3ec26890) by task D:281f29dbfec1
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|audit] VBD.plug: VBD = '7ebef030-db54-bdab-6c2a-6560f6438436'
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|vbdops] VBD.plug of loopback VBD 'OpaqueRef:2276a35b-0b97-4d78-a9d8-020462c0b724'
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|storage_impl] VDI.attach3 dbg:OpaqueRef:8c9b9ea1-a36d-459e-9bb6-f8223187b4b8 dp:vbd/0/xvdb sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de vm:0 read_write:false
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52924 ||dummytaskhelper] task VDI.attach2 D:c5f5e9d75219 created by task R:8c9b9ea1a36d
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52924 |VDI.attach2 D:c5f5e9d75219|sm] SM lvm vdi_attach sr=OpaqueRef:105b9c29-e9c6-4d34-aafd-560a99f401c5 vdi=OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd writable=false
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52924 |sm_exec D:660e3c471fb5|xapi_session] Session.create trackid=5a46f7566c60d1347339d11446c812d1 pool=false uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52926 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:d8a64c603af1 created by task D:660e3c471fb5
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||149 ||xenops] Event on VM c73b8859-be4a-4f60-b7b4-113c8fa9ca61; resident_here = true
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||149 ||dummytaskhelper] task timeboxed_rpc D:624775404aa5 created by task D:e0c8577e4f97
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52929 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:d0d967e0ffa0 created by task D:e0c8577e4f97
Mar 15 10:22:48 XCP-NG-RBS xapi: [ info||52931 HTTPS 10.10.80.125->:::80|VM.set_is_a_template D:ea69cbd33068|xapi_vm_helpers] VM.set_is_a_template('false')
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52934 HTTPS 10.10.80.125->:::80|VM.destroy R:57d76617b82a|audit] VM.destroy: VM = '547ad3b3-be7f-5379-10fe-37ad2de24e7e ([XO Backup TEST] Win2012R2-RDP-RBSMEDIA)'
Mar 15 10:22:48 XCP-NG-RBS xapi: [debug||52934 HTTPS 10.10.80.125->:::80|VM.destroy R:57d76617b82a|xapi_vm_helpers] VM.destroy: deleting DB records
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||149 ||xenops] Event on VM 67af8ff9-b3db-efad-b961-22ed70ded7c7; resident_here = true
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||149 ||xenops] xapi_cache: not updating cache for 67af8ff9-b3db-efad-b961-22ed70ded7c7
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||149 ||dummytaskhelper] task timeboxed_rpc D:f76d5b71db7e created by task D:e0c8577e4f97
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52941 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:25974f4ba32b created by task D:e0c8577e4f97
Mar 15 10:22:49 XCP-NG-RBS xapi: [ info||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|dispatcher] spawning a new thread to handle the current task (trackid=9eef03adcd8dff55421459f480b02d0e)
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|audit] VDI.destroy: VDI = '3fcdb648-c842-4044-9299-059645c9e5de'
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Marking SR for VDI.destroy (task=OpaqueRef:eba3ae49-d5e6-47f5-bc82-49e8d80a9a7f)
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Caught exception while marking VDI for VDI.destroy in message forwarder: OTHER_OPERATION_IN_PROGRESS: [ VDI; OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd ]
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Unmarking SR after VDI.destroy (task=OpaqueRef:eba3ae49-d5e6-47f5-bc82-49e8d80a9a7f)
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|helpers] VDI.destroy locking failed: caught transient failure OTHER_OPERATION_IN_PROGRESS: [ VDI; OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd ]
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|helpers] Waiting for up to 1.505647 seconds before retrying...
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52943 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:host.get_other_config D:9f4cd139277b created by task D:c5f5e9d75219
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52944 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_other_config D:ac9cb15278ef created by task D:c5f5e9d75219
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52945 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_sm_config D:77e986e04cf6 created by task D:c5f5e9d75219
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52947 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:23c5323163bb created by task D:c5f5e9d75219
Mar 15 10:22:49 XCP-NG-RBS xapi: [ info||52948 /var/lib/xcp/xapi|session.login_with_password D:69e33c42387d|xapi_session] Session.create trackid=a643e794ee41c6d8412aa63b1f385ff9 pool=false uname=root originator=SM is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52949 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:6d45e6f4588a created by task D:69e33c42387d
Mar 15 10:22:49 XCP-NG-RBS xapi: [ info||52957 /var/lib/xcp/xapi|session.logout D:9e4c51147d4b|xapi_session] Session.destroy trackid=a643e794ee41c6d8412aa63b1f385ff9
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52958 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all_records D:1069ed915bfb created by task D:c5f5e9d75219
Mar 15 10:22:49 XCP-NG-RBS xapi: [ info||52924 |sm_exec D:660e3c471fb5|xapi_session] Session.destroy trackid=5a46f7566c60d1347339d11446c812d1
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52924 ||dummytaskhelper] task VDI.attach2 D:332b14768189 created by task R:8c9b9ea1a36d
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|storage_impl] dbg:OpaqueRef:8c9b9ea1-a36d-459e-9bb6-f8223187b4b8 dp:vbd/0/xvdb sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de superstate:attached  RO
Mar 15 10:22:49 XCP-NG-RBS xapi: [ info||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|storage_impl] VDI.activate3 dbg:OpaqueRef:8c9b9ea1-a36d-459e-9bb6-f8223187b4b8 dp:vbd/0/xvdb sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de vm:0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52959 ||dummytaskhelper] task VDI.activate D:0e530d915ea0 created by task R:8c9b9ea1a36d
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52959 |VDI.activate D:0e530d915ea0|dummytaskhelper] task VDI.activate D:90e56dc3860f created by task R:8c9b9ea1a36d
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52959 ||sm] SM lvm vdi_activate sr=OpaqueRef:105b9c29-e9c6-4d34-aafd-560a99f401c5 vdi=OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd
Mar 15 10:22:49 XCP-NG-RBS xapi: [ info||52959 |sm_exec D:7fc75bf7c571|xapi_session] Session.create trackid=705d3be3e7aaf46ae0035564ba6aa518 pool=false uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52960 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:2ba57e7cd496 created by task D:7fc75bf7c571
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52963 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:host.get_other_config D:5be31e87bb56 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52964 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_other_config D:dd148e7a285f created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52965 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_sm_config D:751127d5301e created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52968 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:690e81bf1050 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52969 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_other_config D:ddc721c3687d created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52970 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_by_uuid D:8a1c9d868fdc created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52971 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:host.get_by_uuid D:cca79d7b73ef created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52972 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:c1ffc6fe14c1 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52973 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.add_to_sm_config D:4c3a5210a174 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52973 /var/lib/xcp/xapi||api_effect] VDI.add_to_sm_config
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52974 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:c1e3e1844efe created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52975 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_by_uuid D:e9f528ad0187 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52976 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_SR D:ca947dd03469 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52977 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_uuid D:b9f76e4171e3 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52978 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_by_uuid D:476d3eb78bea created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52979 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_type D:e67c8928194c created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52980 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SM.get_all_records_where D:a336cad5dd18 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52981 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SM.get_driver_filename D:3d7c7f1142f1 created by task D:0e530d915ea0

Slawko

@Slawko

Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52982 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VM.get_all_records_where D:28d7f06f1631 created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52983 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:PBD.get_all_records_where D:36291d19628e created by task D:0e530d915ea0
Mar 15 10:22:49 XCP-NG-RBS xapi: [debug||52984 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:PBD.get_device_config D:7f2ca374a2fa created by task D:0e530d915ea0
Mar 15 10:22:50 XCP-NG-RBS xapi: [ info||52993 /var/lib/xcp/xapi|session.login_with_password D:d460e7b09fb5|xapi_session] Session.create trackid=dce0a16c5bc2d8b9d88b3de7d9b60069 pool=false uname=root originator=SM is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52994 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:a61b8b3a09ce created by task D:d460e7b09fb5
Mar 15 10:22:50 XCP-NG-RBS xapi: [ info||53002 /var/lib/xcp/xapi|session.logout D:59b52e70defe|xapi_session] Session.destroy trackid=dce0a16c5bc2d8b9d88b3de7d9b60069
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Marking SR for VDI.destroy (task=OpaqueRef:eba3ae49-d5e6-47f5-bc82-49e8d80a9a7f)
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Caught exception while marking VDI for VDI.destroy in message forwarder: OTHER_OPERATION_IN_PROGRESS: [ VDI; OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd ]
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Unmarking SR after VDI.destroy (task=OpaqueRef:eba3ae49-d5e6-47f5-bc82-49e8d80a9a7f)
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|helpers] VDI.destroy locking failed: caught transient failure OTHER_OPERATION_IN_PROGRESS: [ VDI; OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd ]
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|helpers] Waiting for up to 14.179822 seconds before retrying...
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||53003 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all_records D:481fb75aa443 created by task D:0e530d915ea0
Mar 15 10:22:50 XCP-NG-RBS xapi: [ info||53004 /var/lib/xcp/xapi|session.login_with_password D:698b7ad2bdde|xapi_session] Session.create trackid=df435b5a03c18286ff950c41dc8a96c5 pool=false uname=root originator=SM is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||53005 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:737f1f4ec6c5 created by task D:698b7ad2bdde
Mar 15 10:22:50 XCP-NG-RBS xapi: [ info||53012 /var/lib/xcp/xapi|session.logout D:ff59ce12044c|xapi_session] Session.destroy trackid=df435b5a03c18286ff950c41dc8a96c5
Mar 15 10:22:50 XCP-NG-RBS xapi: [ info||52959 |sm_exec D:7fc75bf7c571|xapi_session] Session.destroy trackid=705d3be3e7aaf46ae0035564ba6aa518
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|storage_impl] dbg:OpaqueRef:8c9b9ea1-a36d-459e-9bb6-f8223187b4b8 dp:vbd/0/xvdb sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de superstate:activated RO
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52922 /var/lib/xcp/xapi|VBD.plug R:8c9b9ea1a36d|vbdops] device path: sm/backend/f3ca3453-6f62-785e-eed2-35df1844cc64/3fcdb648-c842-4044-9299-059645c9e5de
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Marking SR for VDI.destroy (task=OpaqueRef:eba3ae49-d5e6-47f5-bc82-49e8d80a9a7f)
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Caught exception while marking VDI for VDI.destroy in message forwarder: VDI_IN_USE: [ OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd; destroy ]
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52942 HTTPS 10.10.80.125->|Async.VDI.destroy R:eba3ae49d5e6|message_forwarding] Unmarking SR after VDI.destroy (task=OpaqueRef:eba3ae49-d5e6-47f5-bc82-49e8d80a9a7f)
Mar 15 10:22:50 XCP-NG-RBS xapi: [error||52942 ||backtrace] Async.VDI.destroy R:eba3ae49d5e6 failed with exception Server_error(VDI_IN_USE, [ OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd; destroy ])
Mar 15 10:22:50 XCP-NG-RBS xapi: [error||52942 ||backtrace] Raised Server_error(VDI_IN_USE, [ OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd; destroy ])
Mar 15 10:22:50 XCP-NG-RBS xapi: [error||52942 ||backtrace] 1/1 xapi Raised at file (Thread 52942 has no backtrace table. Was with_backtraces called?, line 0
Mar 15 10:22:50 XCP-NG-RBS xapi: [error||52942 ||backtrace]
Mar 15 10:22:50 XCP-NG-RBS xapi: [debug||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|export_raw_vdi] Copying VDI contents...
Mar 15 10:22:50 XCP-NG-RBS xapi: [ info||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|vhd_tool_wrapper] Executing /bin/vhd-tool stream --source-protocol none --source-format hybrid --source /dev/sm/backend/f3ca3453-6f62-785e-eed2-35df1844cc64/3fcdb648-c842-4044-9299-059645c9e5de:/dev/VG_XenStorage-f3ca3453-6f62-785e-eed2-35df1844cc64/VHD-3fcdb648-c842-4044-9299-059645c9e5de --destination-protocol none --destination-format vhd --destination-fd 94df5fd3-2237-4d5f-b004-6792fcb806a6 --tar-filename-prefix  --progress --machine --direct --path /dev/mapper:.
Mar 15 10:22:51 XCP-NG-RBS xapi: [error||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|vhd_tool_wrapper] vhd-tool failed, returning VDI_IO_ERROR
Mar 15 10:22:51 XCP-NG-RBS xapi: [error||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|vhd_tool_wrapper] vhd-tool output: vhd-tool: internal error, uncaught exception:\x0A          Unix.Unix_error(Unix.EPIPE, "write", "")\x0A          \x0A
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53017 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.slave_login D:17c68132bd5a created by task R:317fdffb9f33
Mar 15 10:22:51 XCP-NG-RBS xapi: [ info||53017 /var/lib/xcp/xapi|session.slave_login D:39e8ffdfab69|xapi_session] Session.create trackid=32a2528aa8c7bfd95775e8dff31666dd pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53018 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:c3e39d74aa51 created by task D:39e8ffdfab69
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VBD.unplug D:2fc0c693dd9e created by task R:317fdffb9f33
Mar 15 10:22:51 XCP-NG-RBS xapi: [ info||53019 /var/lib/xcp/xapi||taskhelper] task VBD.unplug R:e9d3f6caad43 (uuid:5a1ddb67-180f-ee5a-7787-52417a4fd668) created (trackid=32a2528aa8c7bfd95775e8dff31666dd) by task R:317fdffb9f33
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|audit] VBD.unplug: VBD = '7ebef030-db54-bdab-6c2a-6560f6438436'
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|vbdops] VBD.unplug of loopback VBD 'OpaqueRef:2276a35b-0b97-4d78-a9d8-020462c0b724'
Mar 15 10:22:51 XCP-NG-RBS xapi: [ info||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] DP.destroy dbg:OpaqueRef:e9d3f6ca-ad43-4548-9863-a206c3941a2c dp:vbd/0/xvdb allow_leak:false
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] [destroy_sr] Filtered VDI count:0
Mar 15 10:22:51 XCP-NG-RBS xapi: message repeated 2 times: [ [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] [destroy_sr] Filtered VDI count:0]
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] [destroy_sr] Filtered VDI count:1
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] [destroy_sr] VDI found with the dp is 3fcdb648-c842-4044-9299-059645c9e5de
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53022 ||dummytaskhelper] task VDI.deactivate D:c4d166153fe4 created by task R:e9d3f6caad43
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53022 |VDI.deactivate D:c4d166153fe4|dummytaskhelper] task VDI.deactivate D:e897b0623552 created by task R:e9d3f6caad43
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53022 ||sm] SM lvm vdi_deactivate sr=OpaqueRef:105b9c29-e9c6-4d34-aafd-560a99f401c5 vdi=OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd
Mar 15 10:22:51 XCP-NG-RBS xapi: [ info||53022 |sm_exec D:83177b32abbd|xapi_session] Session.create trackid=59bacbd469af054a0582244fe812d46a pool=false uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53024 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:a8ae5d554541 created by task D:83177b32abbd
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||149 ||xenops] Event on VM c73b8859-be4a-4f60-b7b4-113c8fa9ca61; resident_here = true
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||149 ||dummytaskhelper] task timeboxed_rpc D:4aa4c9e5f106 created by task D:e0c8577e4f97
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53025 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:d2cb0005d1ae created by task D:e0c8577e4f97
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53027 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:host.get_other_config D:9b335d23861f created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53028 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_other_config D:efe6a84fe1e8 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53029 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_sm_config D:57279ac4e22a created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53031 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:eff589edda89 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53032 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_by_uuid D:1911ffa38cb8 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53033 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:627200c2bb52 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53036 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_by_uuid D:ce0de9c8473e created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53037 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_SR D:f1c86a4c9995 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53038 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_uuid D:57ecd93af89f created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53039 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_by_uuid D:a2b5210931ce created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53040 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_type D:47c7dafde96c created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53041 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SM.get_all_records_where D:d090d460db73 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53042 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SM.get_driver_filename D:d54f6941ac49 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53043 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VM.get_all_records_where D:d3dd0a49c8a9 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53044 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:PBD.get_all_records_where D:00ac19e65663 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53045 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:PBD.get_device_config D:a7c8773f2f39 created by task D:c4d166153fe4
Mar 15 10:22:51 XCP-NG-RBS xapi: [ info||53050 /var/lib/xcp/xapi|session.login_with_password D:d4738496f337|xapi_session] Session.create trackid=af5a0a4e306bb51db1f7b3a1639d7366 pool=false uname=root originator=SM is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:51 XCP-NG-RBS xapi: [debug||53051 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:ae7ca4f1cd86 created by task D:d4738496f337
Mar 15 10:22:51 XCP-NG-RBS xapi: [ info||53059 /var/lib/xcp/xapi|session.logout D:4365d0f394a0|xapi_session] Session.destroy trackid=af5a0a4e306bb51db1f7b3a1639d7366
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53060 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_by_uuid D:e06dee33ee3b created by task D:c4d166153fe4
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53061 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:host.get_by_uuid D:f31ec4ea7eae created by task D:c4d166153fe4
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53062 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:01a5d0dbba3d created by task D:c4d166153fe4
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53063 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.remove_from_sm_config D:d138c3457e28 created by task D:c4d166153fe4
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53063 /var/lib/xcp/xapi||api_effect] VDI.remove_from_sm_config
Mar 15 10:22:52 XCP-NG-RBS xapi: [ info||53022 |sm_exec D:83177b32abbd|xapi_session] Session.destroy trackid=59bacbd469af054a0582244fe812d46a
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] dbg:OpaqueRef:e9d3f6ca-ad43-4548-9863-a206c3941a2c dp:vbd/0/xvdb sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de superstate:attached  RO
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53064 ||dummytaskhelper] task VDI.detach D:4982a5eeba69 created by task R:e9d3f6caad43
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53064 |VDI.detach D:4982a5eeba69|sm] SM lvm vdi_detach sr=OpaqueRef:105b9c29-e9c6-4d34-aafd-560a99f401c5 vdi=OpaqueRef:d6b35ab6-3a17-478c-bb13-7534dc529cdd
Mar 15 10:22:52 XCP-NG-RBS xapi: [ info||53064 |sm_exec D:39b0de9237d4|xapi_session] Session.create trackid=5b352064d736ef480a9e3be5174625ce pool=false uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53065 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:28b5585f5923 created by task D:39b0de9237d4
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53066 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:host.get_other_config D:5f5190439978 created by task D:4982a5eeba69
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53067 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_other_config D:48d816761a39 created by task D:4982a5eeba69
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53068 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:SR.get_sm_config D:75bbd9a06f9c created by task D:4982a5eeba69
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53071 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VDI.get_sm_config D:968b173f3114 created by task D:4982a5eeba69
Mar 15 10:22:52 XCP-NG-RBS xapi: [ info||53064 |sm_exec D:39b0de9237d4|xapi_session] Session.destroy trackid=5b352064d736ef480a9e3be5174625ce
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53064 ||dummytaskhelper] task VDI.detach D:1c39ca39f0b2 created by task R:e9d3f6caad43
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53019 /var/lib/xcp/xapi|VBD.unplug R:e9d3f6caad43|storage_impl] dbg:OpaqueRef:e9d3f6ca-ad43-4548-9863-a206c3941a2c dp:vbd/0/xvdb sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de superstate:detached
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53072 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:VBD.destroy D:7f79506dac03 created by task R:317fdffb9f33
Mar 15 10:22:52 XCP-NG-RBS xapi: [ info||53072 /var/lib/xcp/xapi||taskhelper] task VBD.destroy R:c31377f2021b (uuid:3e574de5-abad-ac42-2eda-b6385bbcd3ef) created (trackid=32a2528aa8c7bfd95775e8dff31666dd) by task R:317fdffb9f33
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53072 /var/lib/xcp/xapi|VBD.destroy R:c31377f2021b|audit] VBD.destroy: VBD = '7ebef030-db54-bdab-6c2a-6560f6438436'
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53072 /var/lib/xcp/xapi|VBD.destroy R:c31377f2021b|xapi_vbd_helpers] VBD.destroy (uuid = 7ebef030-db54-bdab-6c2a-6560f6438436; ref = OpaqueRef:2276a35b-0b97-4d78-a9d8-020462c0b724)
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53073 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:628fac978911 created by task R:317fdffb9f33
Mar 15 10:22:52 XCP-NG-RBS xapi: [ info||53073 /var/lib/xcp/xapi|session.logout D:f49338c10f1d|xapi_session] Session.destroy trackid=32a2528aa8c7bfd95775e8dff31666dd
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||52916 HTTPS 10.10.80.125->:::80|[XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33|taskhelper] the status of R:317fdffb9f33 is failure; cannot set it to `failure
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] [XO] Exporting content of VDI Win2012R2-RDP-RBSMEDIA R:317fdffb9f33 failed with exception Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] Raised Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 1/13 xapi Raised at file ocaml/xapi/vhd_tool_wrapper.ml, line 77
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 2/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 3/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 4/13 xapi Called from file ocaml/xapi/export_raw_vdi.ml, line 50
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 5/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 6/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 7/13 xapi Called from file ocaml/xapi/export_raw_vdi.ml, line 90
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 8/13 xapi Called from file ocaml/xapi/export_raw_vdi.ml, line 116
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 9/13 xapi Called from file ocaml/xapi/server_helpers.ml, line 101
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 10/13 xapi Called from file ocaml/xapi/server_helpers.ml, line 122
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 11/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 12/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 13/13 xapi Called from file lib/backtrace.ml, line 177
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace]
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53074 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:74e67c4c706c created by task D:281f29dbfec1
Mar 15 10:22:52 XCP-NG-RBS xapi: [ info||53074 /var/lib/xcp/xapi|session.logout D:28dbd0ea817e|xapi_session] Session.destroy trackid=35144f0276fe901ef61f755f3ec26890
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] VDI.export_raw_vdi D:281f29dbfec1 failed with exception Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] Raised Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 1/13 xapi Raised at file lib/debug.ml, line 227
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 2/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 3/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 4/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 5/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 6/13 xapi Called from file ocaml/xapi/xapi_http.ml, line 201
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 7/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 8/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 9/13 xapi Called from file ocaml/xapi/server_helpers.ml, line 101
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 10/13 xapi Called from file ocaml/xapi/server_helpers.ml, line 122
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 11/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 12/13 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 13/13 xapi Called from file lib/backtrace.ml, line 177
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace]
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] VDI.export_raw_vdi D:dfaa02950133 failed with exception Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] Raised Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace] 1/1 xapi Raised at file (Thread 52916 has no backtrace table. Was with_backtraces called?, line 0
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||backtrace]
Mar 15 10:22:52 XCP-NG-RBS xapi: [error||52916 :::80||xapi_http] Unhandled Api_errors.Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||149 ||xenops] Event on VM c73b8859-be4a-4f60-b7b4-113c8fa9ca61; resident_here = true
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||149 ||dummytaskhelper] task timeboxed_rpc D:0956550b71bf created by task D:e0c8577e4f97
Mar 15 10:22:52 XCP-NG-RBS xapi: [debug||53076 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:473d1f9eca42 created by task D:e0c8577e4f97
Mar 15 10:22:53 XCP-NG-RBS xapi: [debug||53083 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.slave_login D:0c01e3134004 created by task D:4564f5827955
Mar 15 10:22:53 XCP-NG-RBS xapi: [ info||53083 /var/lib/xcp/xapi|session.slave_login D:a10421f5d514|xapi_session] Session.create trackid=a6eb1200827471b8f8fb71bdb29fbe25 pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Mar 15 10:22:53 XCP-NG-RBS xapi: [debug||53084 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:680048247abb created by task D:a10421f5d514
Mar 15 10:22:53 XCP-NG-RBS xapi: [debug||53085 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:969f4342d831 created by task D:4564f5827955
Mar 15 10:22:53 XCP-NG-RBS xapi: [debug||53086 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:e3e56581997d created by task D:4564f5827955
Mar 15 10:22:53 XCP-NG-RBS xapi: [ info||53086 /var/lib/xcp/xapi|session.logout D:71d24e012e11|xapi_session] Session.destroy trackid=a6eb1200827471b8f8fb71bdb29fbe25
Mar 15 10:22:53 XCP-NG-RBS xapi: [debug||151 |watching networks for NBD-related changes D:4564f5827955|network_event_loop] Not updating the firewall, because the set of interfaces to use for NBD did not change: []
Mar 15 10:22:55 XCP-NG-RBS xapi: [ info||53088 HTTPS 10.10.80.125->|Async.VDI.destroy R:bdf51a2fe437|dispatcher] spawning a new thread to handle the current task (trackid=9eef03adcd8dff55421459f480b02d0e)
Mar 15 10:22:55 XCP-NG-RBS xapi: [debug||53088 HTTPS 10.10.80.125->|Async.VDI.destroy R:bdf51a2fe437|audit] VDI.destroy: VDI = '3fcdb648-c842-4044-9299-059645c9e5de'
Mar 15 10:22:55 XCP-NG-RBS xapi: [debug||53088 HTTPS 10.10.80.125->|Async.VDI.destroy R:bdf51a2fe437|message_forwarding] Marking SR for VDI.destroy (task=OpaqueRef:bdf51a2f-e437-45fb-9ee2-9a75918789d7)
Mar 15 10:22:56 XCP-NG-RBS xapi: [ info||53088 HTTPS 10.10.80.125->|Async.VDI.destroy R:bdf51a2fe437|storage_impl] VDI.destroy dbg:OpaqueRef:bdf51a2f-e437-45fb-9ee2-9a75918789d7 sr:f3ca3453-6f62-785e-eed2-35df1844cc64 vdi:3fcdb648-c842-4044-9299-059645c9e5de
Mar 15 10:22:56 XCP-NG-RBS xapi: [debug||53089 ||dummytaskhelper] task VDI.destroy D:b60259c8456e created by task R:bdf51a2fe437

AtaxyaNetwork

@Slawko said in Trouble doing Continuous Replication on pool (XOA):

VDI.export_raw_vdi D:281f29dbfec1 failed with exception Server_error(VDI_IO_ERROR, [ Device I/O errors ])

Ok, so that's a start. Your iSCSI seems to have some difficulties to read and/or write. Can you double-check your iSCSI installation ? (I don't really know iSCSI, so maybe others can give advices ?)

Can you do a grep -C10 -i EXCEPTION /var/log/SMlog too on your host ?

olivierlambert

Also a dmesg might be interesting