Strange issue with booting XCP-NG

tony

@Appollonius there will probably be a fix for this since many people use nVidia GPUs, if they can pinpoint the issue. Since this isn't solved by using gpumon like other cases, they'll need help testing the solution with feedback from people like you who have the problem.

A Former User

@tony Well apparently it has something to do with the input. So when I connect the dongle to my monitor it does properly startup. Though when I disconnect the monitor it does not. So in my opinion it has something to do with input, though I will try to find the service/proces that is handling this.

r1

@Appollonius Your kern.log should be able to tell more. Can you boot without and with, and share that file?

A Former User

Hello @r1 I will send you the kern.log file today, when I finish this document I will try and get the kern.log for you

A Former User

I am still curious whether or not this is being worked on as I have an Nvidia quadro P400 which is now laying on my desk doing nothing but gathering dust because when I put it in my Server without video output I cannot even use the server because xcp-ng does not properly boot.

r1

@Appollonius can you edit the grub config and remove console=vga vga=mode-0x0311 from multiboot2 /boot/xen.gz line and report back?

A Former User

@r1 As it looks now, this unfortunately did not solve the issue. Though I will send you the recent kern.log data. Is there something I need to configure in the BIOS to be able to boot properly with XCP-NG with a GPU?

Here is the paste of the recent boot, from timestamp 18:18 is the boot periode with the graphics card:
https://pastebin.com/TDA8FcQW

I can narrow the issue down a bit more, when I have no monitor connected XCP-NG/Xenserver will not boot. But with monitor connected to the GPU then it will boot. So it has something to do with the display I think.

r1

@Appollonius they are still pretty same. So this means that there is no issue for system to boot (with or without GPU) but must be something to do with network config.

Can you check if you have something in /var/log/xensource.log indicating a service failure/network start fail?

A Former User

I dont see any failed services, though I have a question. The nvidia GPU (Quadro P400), do I have to configure Passthrough first? Maybe that could be the problem, as when I look into the host GPU's now it does not detect the GPU at all...

r1

@Appollonius if you are using that GPU in a VM, yes, you need to enable passthrough for that pci id.

A Former User

@r1 Well I will be in the future, but XCP-NG does not detect a GPU at all. Maybe that could be the cause of this issue? Atleast I think, as I see no GPU's listed in XCP-NG Center.

r1

@Appollonius check # lspci -k or # lshw -c display

A Former User

@r1 said in Strange issue with booting XCP-NG:

lshw -c display

lshw -c display
  *-display UNCLAIMED       
       description: VGA compatible controller
       product: GP107GL [Quadro P400]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:07:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list
       configuration: latency=0
       resources: memory:fb000000-fbffffff memory:d0000000-dfffffff memory:e0000000-e1ffffff ioport:e000(size=128) memory:c0000-dffff

This is what I see, so I it does see that its there but for some reason it does not see the card in XCP-NG Center.

Ohh wait, is that the Nvidia 'driver' problem? Which AMD cards dont have?

In XO it does see the graphics card, so happy about that. Now I am still curious why it does not boot properly when the NVidia GPU is plugged in.

r1

@Appollonius no, pci pass through does not need device drivers to be present in dom0. Try XOA to administer.

A Former User

@r1 Well it is now even stranger than. So the host detects the GPU, but without monitor input it will just not boot... hmmm

tuxen

@Appollonius check the BIOS for display emulation or ACPI configuration. Some boards may refuse to boot If the BIOS isn't properly configured to accept the headless mode.

A Former User

@tuxen Thanks for the suggestion, though when I leave out the GPU it still boots up fine. Its only when I install the GPU and dont connect it to a monitor that it will not boot properly. But then again, when I install the GPU and connect it to a monitor it will boot fine. So I think it has something to do with the display. And I dont know if this is a XCP-NG problem or a CentOS/System problem.

tuxen

@Appollonius said in Strange issue with booting XCP-NG:

Its only when I install the GPU and dont connect it to a monitor that it will not boot properly.

Maybe because, when there's a GPU installed but no monitor attached, the motherboard POST fails at EDID probe? As stated, some boards/BIOS require an explicit configuration in order to boot without a monitor/keyboard/mouse plugged, eg.:

https://www.supermicro.com/support/faqs/faq.cfm?faq=11902

A Former User

@tuxen Thanks for your reply, unfortunately I dont have the options ACPI - Display emulation. I use a Gigabyte Aorus X570 Pro for this server.
IOMMU is enabled on the bord.
With the Gigabyte B450 AORUS M I had zero issues, with booting in 'headless'.

A Former User

Unfortunately this issue has not been fixed, I will be waiting if someone has another suggestion for this. Though I do have another question, my Xen Orchestra continually says that the time is out of sync with the XCP-NG host. I have looked into the NTP settings, and they seemed the same. Does anyone know what this could be?