I'm not positive, but this may be available RSN in xo-lite.

@Greg_E Looks to me like the timeouts are occurring during the Health Check phase of the backup job. Thus the backup was successfully transferred to the remote, but the subsequent restoration and startup of the VMs failed because the VMs didn't respond in time.

Are you performing the Health Check on every backup run?

@andrewperry This script hasn't been designed or tested to work on XCP-ng 8.2.1 because it reached EOL status last year. Installing Python3 on XCP-ng 8.2.1 should be fine, but we aren't able to guarantee the script's functionality on that platform.

Upgrading to XCP-ng 8.3 is your best option here so that you can benefit from product improvements, security updates, etc.

@BrantleyHobbs You may want to provide some additional details on your setup, ie:

• Version of XCP-ng
• Are the host's fully patched?
• Current XO version or commit
• etc

Have you checked /var/crash subdirectory on the crashed host to see if kernel crash logs were captured? https://docs.xcp-ng.org/troubleshooting/log-files/

@limezest said:

ISOs in my NFS ISO SR

The simplest way to correct those is to forget the VDI and then rescan the SR.

Note: The ISO names will be lost if they were uploaded via XOA, which causes them to be renamed to .IMG files.

Nice. I believe that I read elsewhere that our dev team was working on something similar.

FYI, you have a typo (performace.sh vs performance.sh)

@IgorGlock Are you able to open the support tunnel for us to investigate? You can either post the tunnel ID here or open a support ticket.

@IgorGlock Thanks for the report. Can you elaborate on the exact steps required to reproduce this? Also, are you running XOA or XO from sources? Which version or commit?

@acebmxer The documentations says to use Node LTS, which is currently Node v24. There may have been past issues with running on Node v24. However, AFAIK XO should build and operate fine on Node v24 now.

Node v24 should work fine. The reason for your failure is here --

error eslint-visitor-keys@5.0.1: The engine "node" is incompatible with this module. Expected version "^20.19.0 || ^22.13.0 || >=24". Got "23.11.0"

Either upgrade Node to v24 or downgrade it to v22.

@archw Could the results you are observing be due to the fact that some of the files weren't modified in the specific delta backup that you selected as your restore point?

For example:

In another subdirectory there are supposed to be 7 files but in the selection window there are only two.

When were these 5 "missing" files last modified?

@burbilog Check the current vCPU parameters for the affected VM by running the following command --

xe vm-param-list uuid=<VM_UUID> | grep -i vcpu

where you replace <VM_UUID> with the actual UUID of the VM. The output should look similar to this:

# xe vm-param-list uuid=dc7beebf-46e0-ab98-1743-e0324e24xxxx | grep -i vcpu
                          VCPUs-params (MRW):
                             VCPUs-max ( RW): 2
                      VCPUs-at-startup ( RW): 2
                    allowed-operations (SRO): sysprep; metadata_export; changing_VCPUs_live; changing_dynamic_range; migrate_send; pool_migrate; suspend; hard_reboot; hard_shutdown; clean_reboot; clean_shutdown; pause; checkpoint; snapshot
                      last-boot-record ( RO): '('struct' ('uuid' '2c74c10c-325c-8c54-63d9-8c83e54abf63') ('allowed_operations' ('array')) ('current_operations' ('struct' ('OpaqueRef:dd2f6f24-427c-de26-2d1b-744f25f2f001' 'start'))) ('power_state' 'Halted') ('name_label' 'Debian 12 Cloud-init (Hub)') ('name_description' 'Debian 12 Cloud-Init Ready Hub Template') ('user_version' '1') ('is_a_template' ('boolean' '0')) ('suspend_VDI' 'OpaqueRef:NULL') ('resident_on' 'OpaqueRef:NULL')
('affinity' 'OpaqueRef:NULL') ('memory_overhead' '20971520') ('memory_target' '2147483648') ('memory_static_max' '2147483648') ('memory_dynamic_max' '2147483648') ('memory_dynamic_min' '2147483648') ('memory_static_min' '2147483648') ('VCPUs_params' ('struct')) ('VCPUs_max' '2') ('VCPUs_at_startup' '2') ('actions_after_shutdown' 'destroy') ('actions_after_reboot' 'restart') ('actions_after_crash' 'restart') ('consoles' ('array')) ('VIFs' ('array' 'OpaqueRef:d2cbfa49-86ff-3497-2843-ca1932509b95')) ('VBDs' ('array' 'OpaqueRef:3f7aa90c-9f45-f25d-9bce-5936cd010495' 'OpaqueRef:c8b90a71-a76b-bbdc-fd69-9184e51b7255')) ('crash_dumps' ('array')) ('VTPMs' ('array')) ('PV_bootloader' '') ('PV_kernel' '') ('PV_ramdisk' '') ('PV_args' '') ('PV_bootloader_args' '') ('PV_legacy_args' '') ('HVM_boot_policy' 'BIOS order') ('HVM_boot_params' ('struct' ('order' 'cd') ('firmware' 'bios'))) ('HVM_shadow_multiplier' ('double' '1')) ('platform' ('struct' ('pae' 'true') ('apic' 'true') ('timeoffset' '0') ('acpi' '1') ('viridian' 'false') ('nx' 'true'))) ('PCI_bus' '') ('other_config' ('struct' ('base_template_name' 'Debian Jessie 8.0') ('import_task' 'OpaqueRef:5403e9af-b418-bbf8-19f8-f2676b332685') ('mac_seed' '71148202-a0da-6490-164c-1c6311d09bf9') ('install-methods' 'cdrom,nfs,http,ftp') ('linux_template' 'true'))) ('domid' '-1') ('domarch' '') ('last_boot_CPU_flags' ('struct')) ('is_control_domain' ('boolean' '0')) ('metrics' 'OpaqueRef:d2f3433e-2263-ac9a-b45a-b8de124c9373') ('guest_metrics' 'OpaqueRef:NULL') ('last_booted_record' '') ('recommendations' '<restrictions><restriction field=\"memory-static-max\" max=\"1649267441664\"/><restriction field=\"vcpus-max\" max=\"32\"/><restriction field=\"has-vendor-device\" value=\"false\"/><restriction field=\"allow-gpu-passthrough\" value=\"1\"/><restriction field=\"allow-vgpu\" value=\"1\"/><restriction max=\"255\" property=\"number-of-vbds\"/><restriction max=\"7\" property=\"number-of-vifs\"/></restrictions>') ('xenstore_data' ('struct')) ('ha_always_run' ('boolean' '0')) ('ha_restart_priority' '') ('is_a_snapshot' ('boolean' '0')) ('snapshot_of' 'OpaqueRef:NULL') ('snapshots' ('array')) ('snapshot_time' ('dateTime.iso8601' '19700101T00:00:00Z')) ('transportable_snapshot_id' '') ('blobs' ('struct')) ('tags' ('array')) ('blocked_operations' ('struct')) ('snapshot_info' ('struct')) ('snapshot_metadata' '') ('parent' 'OpaqueRef:NULL') ('children' ('array')) ('bios_strings' ('struct')) ('protection_policy' 'OpaqueRef:NULL') ('is_snapshot_from_vmpp' ('boolean' '0')) ('appliance' 'OpaqueRef:NULL') ('start_delay' '0') ('shutdown_delay' '0') ('order' '0') ('VGPUs' ('array')) ('attached_PCIs' ('array')) ('suspend_SR' 'OpaqueRef:NULL') ('version' '0') ('generation_id' '') ('hardware_platform_version' '0') ('has_vendor_device' ('boolean' '0')) ('requires_reboot' ('boolean' '0')) ('reference_label' 'debian-8'))'
                       recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="32"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions>
                          VCPUs-number ( RO): 2
                     VCPUs-utilisation (MRO): 0: 0.006; 1: 0.007
                                 other (MRO): platform-feature-xs_reset_watches: 1; platform-feature-multiprocessor-suspend: 1; has-vendor-device: 0; feature-vcpu-hotplug: 1; feature-suspend: 1; feature-reboot: 1; feature-poweroff: 1; feature-balloon: 1

Hi Jerry,

I believe that we had similar reports when XO6 was first releases, but I don't recall seeing this recently. We can assist you in troubleshooting this issue if you open a support ticket and provide access via the support tunnel.

Regards, Dan

Hi,

Thanks for the report. I'm checking with the XOA team to see if this is a known issue.

Regards, Dan

Yes. In case you missed it, this is from https://docs.xcp-ng.org/installation/upgrade/ :

@User-cxs said:

when I want to shut down a node with VMs on it, the HA will cause the following

We are still waiting for your question.

P.S. The solution is in the GH issue that I previously linked.

@kruess I'm unsure why you are encountering this error. Did you eject all ISOs as shown in the documentation --

Eject CDs from your VMs before upgrading to avoid issues: xe vm-cd-eject --multiple

@kruess I can see numerous instances of VM_HOST_INCOMPATIBLE_VERSION in your xensource log. It is difficult to know the exact cause without more details.

• Was the pool master restarted after the patches were applied?
• Are you trying to start the VM on the master?
• Are you upgrading directly from XS 7.1.2 to XCP 8.3?

@User-cxs You are probably experiencing this issue.

Just a guess because you didn't actually ask a question.

@edsilber said:

you have held broken packages

Have you tried running yum clean and then retrying the install process?

You are running XO from sources, correct? What OS version?

Edit:

Oh, I see you are running a very old deployment of XOA. Deploy a new instance of the appliance and then the package issues will be resolved.