VM’s Slow booting

bassopt

I haven’t used XCP-ng for a while , but the issues I had were similar. Around 10-12 secs from grub to prompt. Both in Debian or Ubuntu server. For some reason I remember the machines booted way faster using uefi than bios, so I always thought it was some odd issue with your bio’s implementation

I have a machine I can install xcp-ng to test … I’ll give it a go soon.

imaginapix

@olivierlambert said in VM’s Slow booting:

Ubuntu server or desktop?

Sorry, I should have added that. Ubuntu Server it is. A short stop is always on Cloud Init - 2-4 sec.

bassopt

@imaginapix

Well mini was on initramfs. Boot part. I never used cloud init for anything

olivierlambert

Cloudinit is loading many things, so I wouldn't be surprised it makes the boot longer. Can we check with a default/basic Ubuntu server install and give the timing?

imaginapix

@olivierlambert

Installed a new 22.04.2 Server in default installation and removed cloud-init. Hitting enter in grub to login screen 6 sec. on the same host as noted above

olivierlambert

Let see in the community around if they have a similar experience. My impression is 6 seconds is relatively fine.

manilx

@olivierlambert I have one Ubuntu server 22.04, fresh install and it has had this "slow" booting (black screen foir a few seconds) since the install. Doesn't bother me much. While all my Debian 12 boot instantly without this hang.

imaginapix

@manilx

Could this be the grub menu and it's wait timer? I had it once that I did no saw grub but the system bootet fine. In the end grub was the black screen and depending on the wait interval it stayed black.

This might be a thing with how the console handles image grabbing during boot

manilx

@imaginapix Don't know enough of this. I only use debian and know my way around.
The ubuntu VM was just a test....

Here is a video of the VM booting: https://www.dropbox.com/scl/fi/z7urjkcubq8vnjjbcs9nn/ScreenShot-2024-04-19-at-18.01.45.mov?rlkey=iufbqfqy7758pke3hsiv58zpt&dl=0

bassopt

@manilx I can tell you this is extremely dois compare to proxmox for exemple. My Debian VMs with one CORe on a i5-12400 boot in half of that time.

Tristis Oris

Ubuntu 20-22 boot is slow by design. It may take 1-2min.
Debian or any rhel boot in 20sec.

gskger

@olivierlambert Out of curiosity, I carried out a few tests this weekend. My tests are not really scientific and must be taken with a grain of salt. I started with the bare metal installs, followed by the VM installs on the hypervisors. The current server ISOs have been used for OS or hypervisor installs. All OS or hypervisor were updated, but otherwise no tweaking was done (e.g. remove snap from Ubuntu).

The tests were carried out manually by starting the (bare metal/virtualized) OS and taking the time until the login prompt appeared. Where necessary, the grub menu was confirmed manually as quickly as possible. Up to three runs were averaged. The host is a HP ProDesk 400 G6 with a i5-10500T CPU, 32GB RAM and a KIOXIA 512GB M.2 NVMe SSD (KXG70ZNV512G). VMs had 2 CPU, 4 GB RAM and 32 GB storage.

Host	OS	Time to login [BIOS/EUFI in s]
bare metal	Ubuntu 22.04.4	-/39
	Debian 12.5	-/20
XCP-ng 8.2.1	Ubuntu 22.04.4	28/12
	Debian 12.5	18/8
Proxmox 8.1.4	Ubuntu 22.04.4	12/15
	Debian 12.5	10/10

Even if the differences are measurable, they don't have much significance for me in normal everyday operation. The result is still interesting though.

Edit: Added Time to login for EUFI boot on the hypervisors.

olivierlambert

Thanks @gskger Have you tried UEFI vs BIOS?

Forza

@bassopt said in VM’s Slow booting:

@olivierlambert hi, again!

Nah, as told before it’s on initramdisk. After loading grub. A Ubuntu test install I did freezes for almost 20 seconds there.

I do have some warnings when I boot xcp-ng regarding x-273 and x-297 .... would that cripple performance that much. ( consumer hardware )

Sounds like you have the wrong template. If you disable "quiet" option in grub (press e, then go to the kernel command line) you may see it is error out on enabling some hardware feature. I have seen it happen when trying to enumerate the CPUs. Have a look. You might be able to see in dmesg or journalctl too.

bassopt

@Forza I don’t use xcp-ng anymore. I have a machine i can install it to try out but I haven’t had time lately.

gskger

@olivierlambert Just updated my post with the BIOS vs. UEFI Time to login. Both hypervisors are equally fast with EUFI, although Ubuntu 22.04 on proxmox is now slightly slower. But I guess that is not realy relevant.

olivierlambert

That's interesting though