Strange issue with booting XCP-NG

A Former User

I am still curious whether or not this is being worked on as I have an Nvidia quadro P400 which is now laying on my desk doing nothing but gathering dust because when I put it in my Server without video output I cannot even use the server because xcp-ng does not properly boot.

r1

@Appollonius can you edit the grub config and remove console=vga vga=mode-0x0311 from multiboot2 /boot/xen.gz line and report back?

A Former User

@r1 As it looks now, this unfortunately did not solve the issue. Though I will send you the recent kern.log data. Is there something I need to configure in the BIOS to be able to boot properly with XCP-NG with a GPU?

Here is the paste of the recent boot, from timestamp 18:18 is the boot periode with the graphics card:
https://pastebin.com/TDA8FcQW

I can narrow the issue down a bit more, when I have no monitor connected XCP-NG/Xenserver will not boot. But with monitor connected to the GPU then it will boot. So it has something to do with the display I think.

r1

@Appollonius they are still pretty same. So this means that there is no issue for system to boot (with or without GPU) but must be something to do with network config.

Can you check if you have something in /var/log/xensource.log indicating a service failure/network start fail?

A Former User

I dont see any failed services, though I have a question. The nvidia GPU (Quadro P400), do I have to configure Passthrough first? Maybe that could be the problem, as when I look into the host GPU's now it does not detect the GPU at all...

r1

@Appollonius if you are using that GPU in a VM, yes, you need to enable passthrough for that pci id.

A Former User

@r1 Well I will be in the future, but XCP-NG does not detect a GPU at all. Maybe that could be the cause of this issue? Atleast I think, as I see no GPU's listed in XCP-NG Center.

r1

@Appollonius check # lspci -k or # lshw -c display

A Former User

@r1 said in Strange issue with booting XCP-NG:

lshw -c display

lshw -c display
  *-display UNCLAIMED       
       description: VGA compatible controller
       product: GP107GL [Quadro P400]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:07:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list
       configuration: latency=0
       resources: memory:fb000000-fbffffff memory:d0000000-dfffffff memory:e0000000-e1ffffff ioport:e000(size=128) memory:c0000-dffff

This is what I see, so I it does see that its there but for some reason it does not see the card in XCP-NG Center.

Ohh wait, is that the Nvidia 'driver' problem? Which AMD cards dont have?

In XO it does see the graphics card, so happy about that. Now I am still curious why it does not boot properly when the NVidia GPU is plugged in.

r1

@Appollonius no, pci pass through does not need device drivers to be present in dom0. Try XOA to administer.

A Former User

@r1 Well it is now even stranger than. So the host detects the GPU, but without monitor input it will just not boot... hmmm

tuxen

@Appollonius check the BIOS for display emulation or ACPI configuration. Some boards may refuse to boot If the BIOS isn't properly configured to accept the headless mode.

A Former User

@tuxen Thanks for the suggestion, though when I leave out the GPU it still boots up fine. Its only when I install the GPU and dont connect it to a monitor that it will not boot properly. But then again, when I install the GPU and connect it to a monitor it will boot fine. So I think it has something to do with the display. And I dont know if this is a XCP-NG problem or a CentOS/System problem.

tuxen

@Appollonius said in Strange issue with booting XCP-NG:

Its only when I install the GPU and dont connect it to a monitor that it will not boot properly.

Maybe because, when there's a GPU installed but no monitor attached, the motherboard POST fails at EDID probe? As stated, some boards/BIOS require an explicit configuration in order to boot without a monitor/keyboard/mouse plugged, eg.:

https://www.supermicro.com/support/faqs/faq.cfm?faq=11902

A Former User

@tuxen Thanks for your reply, unfortunately I dont have the options ACPI - Display emulation. I use a Gigabyte Aorus X570 Pro for this server.
IOMMU is enabled on the bord.
With the Gigabyte B450 AORUS M I had zero issues, with booting in 'headless'.

A Former User

Unfortunately this issue has not been fixed, I will be waiting if someone has another suggestion for this. Though I do have another question, my Xen Orchestra continually says that the time is out of sync with the XCP-NG host. I have looked into the NTP settings, and they seemed the same. Does anyone know what this could be?

olivierlambert

Double check your NTP settings, if it tells it's not sync, it's not sync.

Note: XCP-ng now uses chrony, but in XO, ntpq -p will give you NTP stats

A Former User

Okay so today I upgraded to XCP-NG 8.2.0 but unfortunately it did not solve the issue with XCP-NG not being able to boot without a display attached to the GPU. Also I am getting the EFI_MEMMAP issue and I don't know if this issue has a relation to the GPU.

If someone has suggestions which I may try, please do so and I will test it

Danp

@appollonius Are you booting on BIOS or UEFI mode? I ran across this issue earlier today that sounds exactly like what you've described.

If this is the same issue, then I wonder why the "fix" hasn't made it into xcp-ng.

olivierlambert

We include all fixes from XS tree into XCP-ng (or those not included are completely analyzed to be sure about it)