After installing updates: 0 bytes free, Control domain memory = 0B
-
@olivierlambert
I can not see any change to the problem.
The host still starts up in maintenance mode. Memory reports no space left despite no VMs running. Not even the host barebone OS itself is shown on the empty memory bar under the "General" tab.
When trying to start a VM:
The full message text:
-1(list index out of range, , Traceback (most recent call last):
File "/etc/xapi.d/plugins/xcpngutils/init.py", line 119, in wrapper
return func(*args, **kwds)
File "/etc/xapi.d/plugins/hyperthreading.py", line 17, in get_hyperthreading
return json.dumps(int(lines[0]) > 1)
IndexError: list index out of rangeWhen I google "AMD HyperThreading" (since this is not an Intel-system but AMD): Google says "Hyper-threading is the Intel technology name, and commonly known name, for the Computing concept known as simultaneous multithreading. AMD have an equivalent technology known as Cluster Multi-threading."
This is enabled by default in BIOS (and I reset BIOS to defaults before upgrading BIOS just an hour ago). -
What's the output of
xl dmesg -
-
You have an issue: I'm pretty sure you are using Xen 4.13 while using the 4.17 tooling around, causing this problem. So the question is "why are you booting on 4.13?". Double check your Grub menu when you boot, to see that you actually boots on Xen 4.17
-
@olivierlambert
https://xcp-ng.org/forum/assets/uploads/files/1714683479351-b73237cc-8f27-4789-a9d3-5e322c2cd244-image.pngAnswer to your question : I don't know
Six servers have been running fine for months. Then i accepted XO's request to install these updates and the problem manifested on all six.
I did nothing to XCP-ng for months before this happened. Did nothing other than reboot one host after, which then made me realize that something was seriously wrong since no VMs came up. I then also saw the other problems shown in screen clippings earlier in this chat.
Luckily the VMs continued running on the other hosts. I am very happy I only rebooted one first. Or all VMs would have been down for days.Lacking in mental capacity to understand what has happened I am therefore hoping you and your eminent staff can "Sherlock Holmes" this :-).
How can the abovementioned updates have triggered it?
Can it happen to others?I may have waited too long between checking if updates were waiting to be installed. Could this have caused the problem? Were they installed in incorrect order?
I can not find Xen 4.17 in the boot menu. How do I fix this? Should I create a new 8.3 installation media and try to upgrade the system? Can I do this safely without damaging VMs?
-
Well, first, you are on XCP-ng 8.3, right? It's still a beta, so by definition this could happen (especially a jump like this from Xen 4.13 to 4.17). If you want guarantee stability, you have to stick to our LTS version.
Anyway, to know which Xen version you are running, you can do a:
cat /sys/hypervisor/version/minor. If it's17, then you are on Xen 4.17 and that's your issue. I don't know how this could happen in the first place, like if the Grub file was never updated or tinkered before.To check a bit more, when on Grub, press "e" key to edit the first entry and make a picture to see what's displayed there. I'm thinking on a way to regen Grub that will take the right Xen version into account (ping @stormi )
-
@olivierlambert
I had to use 8.3 on these newest 6 hosts since 8.2.1 gave me a black screen mid-way in the installation. I guess because of missing drivers. The forum adviced 8.3 and that worked wonderfully. It has been working fine for more than 1/2 year (forgot when I started using it).I did not register well enough that i was running a risk by doing this. Mea culpa.
-
@Dataslak
I am late to a meeting with customers. I would much rather stay in the office to work with this. But when I come back in a couple of hours I will eagerly check for a recipe/howto on what to do next, and then carefully follow those.
I am very optimistic now. Thank you all for assisting and to @olivierlambert for the big development today. -
So now we know the problem, the next step will be to solve it
Let's check what's going on in
/bootwith als -la /boot -
@olivierlambert said in After installing updates: 0 bytes free, Control domain memory = 0B:
You have an issue: I'm pretty sure you are using Xen 4.13 while using the 4.17 tooling around, causing this problem. So the question is "why are you booting on 4.13?". Double check your Grub menu when you boot, to see that you actually boots on Xen 4.17
Nice catch, this is what community is all about
-
@olivierlambert
I love and respect your attitude
Agree wholeheartedly with @nikade
-
So
/boot/xen.gzis pointing toxen-4.17.3-4.gz, which sounds correct. So why you are still running on Xen 4.13?
It's like you did not reboot, but since you showed me the Grub menu, I'm assuming you already did It would be interesting to compare the existing xen file and see if it's the right one from our repo. Something is fishy here
-
Can you
md5sum xen-4.17.3-4.gzFrom the mirror & RPM, I have
f011721be0c7b57563e29ed282558da3 -
Adding @yann in the loop in case I'm missing something obvious
-
@Dataslak what does
lsblk -o name,mountpoint,label,size,uuidshow? -
-
@yann
Hello Yann, thank you for pitching in.
-
@Dataslak can you please request a commandline from GRUB (hit
con the boot menu), and issue the following commands:echo $root search --label --set root root-eqjpzg echo $root -
Also a
cat /proc/mdstatin the Dom0 would help. -
Info: I am mirroring two M.2 SSDs ! Software RAID established by the installation routine of v8.3.
Could the mirror be broken and cause this somehow?
