@pilow - I know this has gotten a bit off topic, but in the interests of keeping folks informed... I've managed to migrate ALL VDIs over to remote NFS Storage, except this problem VM. To solve that problem, I: Added a second disk on the remote storage, attached it, and booted into clonezilla. Cloned the disk from INSIDE the VM. Detatched and removed the old disk. The local SR is now void of disks. I decided that instead of rebuild the RAID with a new disk, I'm just going to completely pull the disks and replace with shiny new disks. It's just not worth the uncertainty of what MIGHT happen. All the disks in that array are the same age. EDIT: Backup Succeeded on the new disk, so I think you hit the nail on the head @pilow ... Failing SR.

@ph7 that is a good news thank you for your patience and help

@dinhngtu You rock - thank you!

@florent Thanks for the reply. I will rebuild XO and test with current master.

Ping @Team-XO-Backend

@rzr Hi. Thank you very much. I will be setting up a new server within the next 2 weeks and will definitely test it. Then I will give you feedback! Thank you again for picking it up!

@florent One last update. I reverted to Master branch (6699b) yesterday evening and the backup ran without issues overnight.

@TeddyAstie Attached is the output you requested xen-cpuid -p.txt

@abudef Some quotes from the documentation to clarify the situation: https://docs.xcp-ng.org/compute/#-nested-virtualization

Hi @acebmxer, I've made some tests with a small infrastructure, which helped me understand the behaviour you encounter. With the performance plan, the load balancer can trigger migrations in the following cases: to better satisfy affinity or anti-affinity constraints if a host has a memory or CPU usage exceeds a threshold (85% of the CPU critical threshold, of 1.2 times the free memory critical threshold) with vCPU balancing behaviour, if the vCPU/CPU ratio differs too much from one host to another AND at least one host has more vCPUs than CPUs with preventive behaviour, if CPU usage differs too much from one host to another AND at least one host has more than 25% CPU usage After a host restart, your VMs will be unevenly distributed, but this will not trigger a migration if there are no anti-affinity constraints to satisfy, if no memory or CPU usage thresholds are exceeded, and if no host has more CPUs than vCPUs. If you want migrations to happen after a host restart, you should probably try using the "preventive" behaviour, which can trigger migrations even if thresholds are not reached. However it's based on CPU usage, so if your VMs use a lot of memory but don't use much CPU, this might not be ideal as well. We've received very few feedback about the "preventive" behaviour, so we'd be happy to have yours. As we said before, lowering the critical thresholds might also be a solution, but I think it will make the load balancer less effective if you encounter heavy load a some point.

@dinhngtu @teddyastie Do you have a time line / rough estimated release date regarding Rust-based xen-guest-agent version 1.0? Currently evaluating if I need to switch my Linux guests back to Citrix guest tools in the meantime. Looking forward to a reply. Thank you in advance.

But thank you for your time all the same.

@DustyArmstrong Thanks for responding to the GitHub issue. It’s great that more people want this working; it’s difficult to gain traction otherwise. Regarding your list, it’s correct. A reboot should be on the second place. You need to reboot only to detach your PCI device (video card) from its driver and assign it to the pciback driver instead on the next boot. This effectively creates a reservation for the device and allows you to dynamically assign it to VMs. Once your card is free from other kernel drivers, the rest doesn’t require a reboot.

@Pilow While that could be true only 1 vm was shuffled to another SR. But all VMs now being backup with CBT enabled. Well... i just check again at backups and after the last scheduled backup at 1am all vms fell back to full backup again.

I have two PE R630 and one PE R730xd. The two R630s show the IPMI data in XO but the R730xd does not. If I run ipmitool sensor from an ssh session, I get very similar data back on the 730 and 630. This isn't critical to me, just letting you know that there are cases that functioning ipmitool on a Dell does not seem to be enough to show it. There are some differences in the labels. The R730 has "Fan1 RPM", "Fan2 RPM" while the R630 has "Fan1A", "Fan2A", "Fan1B". Most of the other interesting fields seem to have the same names but the order is a bit different and I didn't do an exhaustive comparison. It's very cool that you are surfacing this data now.

@acebmxer I’m sorry to say that if those Dells are at your workplace, the wrong edition of iDRAC was purchased. You see with at the very least iDRAC 9 access to the full granular IPMI sensor data, was placed behind an edition paywall by Dell Technologies. Outlet temperature is just one of the feeds, missing from the Enterprise or lower edition of iDRAC 9. You’ll get the temperature readings from the Dell iDRAC web browser based interface, but not IPMI with iDRAC 9 Enterprise. To obtain the full IPMI sensor data you need the Datacenter edition.

@olivierlambert Can you provide insights? Is nested virtualization as in running Hyper-V inside a XCP-ng VM expected to work?

@florent That would be great. Thanks again!