@olivierlambert
https://github.com/cert-ee/cuckoo3

you are right.
Seems that they are preparing next version. I don't know what hypervisors will work with this.

Edit: qemu and kvm.

Hello,

Can you with a support client for XCP-ng? (like XO). You can also use xe to see the output.

@stormi Hi, its good to know.

Thanks for the answer i think now its solved and explained.

@Heady

Update - I have been able to make progress.

(1) I have been able to get XO version 5.80.0 appliance installed. For some reason the automated web deployment method didn't work. Couldn't work out why.
(2) Had to reinstall xcp-ng again on the new host.
(3) Using XO appliance - had to add a number of interfaces prior to attempting to add to the pool.
(3) Using XO appliance - have been able to add the new host to the pool.
(4) Took a few reboots and manual powercycles to get the new host integrated into the pool and recognized within XO appliance. Shutdown/reboot would hang and not complete.
(5) The SRs had to be connected / disconnected / connected from the XO Host-Storage tab for them to be online and accessible. Couldn't use connect to all hosts from the Pool-Storage tab.
(6) Then finally success in moving VMs from the old host to the new host.

Phew... half-way there.

@kulmacet Can you recreate this with other VMs, or just this specific oracle linux VM? I would spin up a new debian VM for example and shut this problematic VM off, and see if the issue happens with this VM as well. Outside of that, it's really looking like a hardware issue. Also, double check the ILO and BIOS firmware are at the latest. I can almost guarantee it shipped with ancient versions, and many issues like this have been patched relatively recently.

@jcdick1 I have almost the same servers and everything has been fine (without keyboards)...

HP DL360p G8 with E5-2680 v2 and also some with E5-2680 (not v2). I did have one machine have problems but it was hardware failure (CPU fault), not XCP.

These HP G8 machines are about a decade old now, so hardware issues are not a surprise.

@olivierlambert
Good, that's the answer I needed thanks.

If I understand correctly, PCI pass-through is discouraged by Intel with this family of processors, but there's a way to make it work with kernel patches?

If this someone needs in the Futuere:
I found a way to set the licence server to localhost, so the IP address would no loger appear in the "General Tab":

Run on each host:
xe host-apply-edition edition=XCP-ng Free/Libre Edition license-server-address="localhost"

@thelearner

https://xcp-ng.org/forum/post/20637

fixed! issue was virtualization was not enabled in BIOS settings logs on XOA was not good enough xcp-ng center gave me a better error log message of HVM is Required for this Operation thanks!

@thenorthernlight So I was able to open up the host and remove all PCIE cards and riser cards. At a glace everything appears to have made proper contact, however after reseating the cards and booting again and sure enough it appears to work again. I did not have to run any testing in the bios although the server does do a pre boot inventory check each time for changes. Host is HPE Proliant DL380 G9.

https://access.redhat.com/solutions/5347091

System fails to mount filesystem on multipath partition during boot time in RHEL8
Solution Verified - Updated May 31 2021 at 2:25 PM - English
Environment

Red Hat Enterprise Linux 8 dracut-049-27.git20190906.el8.x86_64

Issue

System fails to mount filesystem created on multipath partition during boot time in RHEL8.

Raw

[ 235.191955] testsystem systemd[1]: Mounting /sysroot...
[ 235.398136] testsystem kernel: SGI XFS with ACLs, security attributes, no debug enabled
[ 235.398913] testsystem mount[2922]: mount: /sysroot: /dev/sdb4 already mounted or mount point busy.
[ 235.399207] testsystem systemd[1]: sysroot.mount: Mount process exited, code=exited status=32
[ 235.399259] testsystem systemd[1]: sysroot.mount: Failed with result 'exit-code'.
[ 235.399480] testsystem systemd[1]: Failed to mount /sysroot.

Resolution

Upgrade dracut package to dracut-049-135.git20210121.el8 as per RHBA-2021:1661 As a workaround, rebuild initramfs with /lib/udev/rules.d/11-dm-parts.rules.

Raw

dracut -f -v --include /lib/udev/rules.d/11-dm-parts.rules /lib/udev/rules.d/11-dm-parts.rules

Or create /etc/dracut.conf.d/part.conf with below lines and rebuild initramfs image.
#vi /etc/dracut.conf.d/part.conf
install_items+=/lib/udev/rules.d/11-dm-parts.rules

Root Cause

dracut doesn't install /lib/udev/rules.d/11-dm-parts.rules, which is what changes the link priority for kpartx partition devices, which should make /dev/disk/by-uuid point to multipath devices instead of the scsi path devices.

Diagnostic Steps

Check boot logs and verify whether root filesystem mounted using sdX device or mpath device.

Raw

[ 235.185721] testsystem systemd[1]: Starting File System Check on /dev/disk/by-uuid/3a9fc23b-e95d-44a2-b95c-9651799f8c77...
[ 235.190597] testsystem systemd-fsck[2920]: fsck.none doesn't exist, not checking file system on /dev/disk/by-uuid/3a9fc23b-e95d-44a2-b95c-9651799f8c77.
[ 235.191192] testsystem systemd[1]: Started File System Check on /dev/disk/by-uuid/3a9fc23b-e95d-44a2-b95c-9651799f8c77.
[ 235.191955] testsystem systemd[1]: Mounting /sysroot...
[ 235.398913] testsystem mount[2922]: mount: /sysroot: /dev/sdb4 already mounted or mount point busy.

Verify /lib/udev/rules.d/11-dm-parts.rules rule available in initramfs image.

Raw

lsinitrd initramfs-4.18.0-193.el8.x86_64.img |grep dm-parts |wc -l

0

Rebuild initramfs image with /usr/lib/udev/rules.d/11-dm-parts.rules.

Raw

#lsinitrd initramfs-4.18.0-193.el8.x86_64.img |grep dm-parts
Arguments: -f -v -a 'multipath' --include '/usr/lib/udev/rules.d/11-dm-parts.rules' '/usr/lib/udev/rules.d/11-dm-parts.rules'
-rw-r--r-- 1 root root 1442 Dec 17 2019 usr/lib/udev/rules.d/11-dm-parts.rules

No worries 🙂 Enjoy!

Great to know 🙂 So it works as "expected". That would be wonderful to have your XCP-ng hosts covered by our support so you get the entire stack supported!

We won't change the default behavior before having a LARGE use base using it without any problem 😄 And if I wonder about the risk on Linux, I have 0 knowledge on Windows based OS.

@danp Thanks!

It's not a failure/problem of XO, but your hosts (as you can see the same issue persists despite using xe, XCP-ng Center or XO).

Have you checked the disk space? Also raise the memory in the dom0.

Short answer: you can't.
Long answer: you can "cheat" with dynamic memory.

But in general, I would avoid it.