XCP-ng 8.3 betas and RCs feedback 🚀

[stormi]

@Thunder-Cloud this is not far from accurate, if by "same source" you mean based on. Not everything from Citrix Hypervisor / XenServer is taken identical in XCP-ng, so we don't share 100% of the source code. There are various proprietary components we removed or replaced, and we also have our specific additions.

Also, the name "XenServer 8.0" is not accurate : it's called simply XenServer 8.

[john.c]

@stormi It's currently getting stuck on finishing startup on XCP-ng 8.3 beta 1. I have completed the installation of the 8.3 beta via netinstall media. Configured for dual ipv4 and ipv6 network stack.

However it seems to be taking a really long time to bring up the management network or not bringing it up. Alternatively something else is preventing it from finishing the start up.

If attempt to confirm the ethernet port for the management network, it's currently timing out during the attempt.

Can anyone please help me deduce which part of the finishing start-up is getting stuck?

[gb.123]

@john-c

Can you try installing ipv4 only and alternatively ipv6 only to see if it works ?
(Just wanted to confirm something)

[laurentm]

@john-c Maybe you could add an intel NIC if you have a spare one.
I like DELL servers but not much the Broadcom they provide to save some bucks.

[bc-23]

Hi,

I have a issue starting a PV VM on a fresh installed XCP-ng 8.3 server.
The VM was created from a template I exported from our XCP-ng 8.2 cluster and imported into the new 8.3 server.
The template creates a empty PV VM containing the PV boot information to do a network installation.

The error message I get is:
xenopsd internal error: VM = fb7977de-aa28-273b-7e07-90a8c8639559; domid = 9; Bootloader.Bad_error

In the xensource.log I don't see much more information:

Sep 28 08:20:12 X xapi: [error||26203 |Async.VM.start R:5c82647ea60e|xenops] Re-raising as INTERNAL_ERROR [ xenopsd internal error: VM
 = fb7977de-aa28-273b-7e07-90a8c8639559; domid = 9; Bootloader.Bad_error  ]
Sep 28 08:20:12 X xapi: [error||26203 ||backtrace] Async.VM.start R:5c82647ea60e failed with exception Server_error(INTERNAL_ERROR, [ 
xenopsd internal error: VM = fb7977de-aa28-273b-7e07-90a8c8639559; domid = 9; Bootloader.Bad_error  ])
Sep 28 08:20:12 X xapi: [error||26203 ||backtrace] Raised Server_error(INTERNAL_ERROR, [ xenopsd internal error: VM = fb7977de-aa28-27
3b-7e07-90a8c8639559; domid = 9; Bootloader.Bad_error  ])
Sep 28 08:20:12 X xapi: [error||26203 ||backtrace] 1/39 xenopsd-xc Raised at file ocaml/xenopsd/xc/xenops_server_xen.ml, line 2201
Sep 28 08:20:12 X xapi: [error||26203 ||backtrace] 2/39 xenopsd-xc Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 2
4
...

I skipped the remaining 36 lines from the backtrace, as this only seems to be the ocaml stack trace I it doesn't some to contain any additional relevant information.

When I compare two newly created VMs based on the PV template in the 8.2 and 8.3 environment, the look equal.
The PV elements from vm-param-list on both VMs looks like:

xe vm-param-list uuid=<UUID> | grep PV
                             PV-kernel ( RW): 
                            PV-ramdisk ( RW): 
                               PV-args ( RW): preseed/url=<install specific information>
                        PV-legacy-args ( RW): 
                         PV-bootloader ( RW): eliloader
                    PV-bootloader-args ( RW): 
                    PV-drivers-version (MRO): <not in database>
    PV-drivers-up-to-date ( RO) [DEPRECATED]: <not in database>
                   PV-drivers-detected ( RO): <not in database>

I see a difference on the bios-strings parameter, which is empty in 8.2 but contains the following in 8.3:

bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; baseboard-manufacturer: ; baseboard-product-name: ; baseboard-version: ; baseboard-serial-number: ; baseboard-asset-tag: ; baseboard-location-in-chassis: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d

Do you have a hint what could case this error, or where I could find additional information, as the error message does not contain a lot of information.

Thanks.

[olivierlambert]

PV guests should be avoided as possible, it's not officially supported anymore.

1. Is it a 64 or 32 bits PV guest?
2. Can you convert it to a HVM guest and see the result?

[bc-23]

@olivierlambert

It is a 64bit PV guest.
The reason still using PV guests is that our install environment used to add boot parameters for the installer which was quite a nice way to manage the installation.

Running as HVM guest works without issue.

But when PV guests are not supported anymore then it seems now I should start to migrate our environment to use HVM guests.

May I ask what would be a good way to insert install information into a HVM guest? At the moment I would think about using xenstore to add information like IP to use and other information and create a custom Debian installer reading the information from the xenstore.
I know also about cloudinit, but what I have read so far I don't think it would fit into our environment.

Thanks,

[olivierlambert]

Packer is probably what you need @AtaxyaNetwork worked on it for some VMs here, feel free to ask her if you have question.

[bc-23]

Thanks for the hint, I will take a look into it.

[olivierlambert]

Link to the repo: https://github.com/ddelnano/packer-plugin-xenserver

[john.c]

@gb-123 said in XCP-ng 8.3 beta :

@john-c

Can you try installing ipv4 only and alternatively ipv6 only to see if it works ?
(Just wanted to confirm something)

@stormi @gb-123 When I tried installing as IPv4 only it was OK and came up correctly successfully. About to try IPv6 only will report back.

[john.c]

@john-c said in XCP-ng 8.3 beta :

@gb-123 said in XCP-ng 8.3 beta :

@john-c

Can you try installing ipv4 only and alternatively ipv6 only to see if it works ?
(Just wanted to confirm something)

@stormi When I tried installing as IPv4 only it was OK and came up correctly successfully. About to try IPv6 only will report back.

@stormi @gb-123 When I tried installing as IPv6 only its currently trying to bring up the management network.

EDIT - Update its still trying to bring up the connection with no success.

[john.c]

@stormi @gb-123 Just done a reboot from maintenance mode, to see if this gets it to bring up the IPv6 only management network.

UPDATE: Definitely didn't help still trying to bring up the IPv6 only management network.

[john.c]

@stormi @gb-123 The issue with management network being brought up on XCP-ng 8.3 beta1, dual stack (IPv4 and IPv6). Specifically it taking a really long time or not coming up at all, likely is due to the IPv6 side not completing successfully or in a reasonable timespan.

As shown by the problem on the IPv6 only install.

[stormi]

Thanks for the feedback. Pinging @BenjiReis for the IPv6 topic.

[BenjiReis]

@john-c hi thx for the feedback.

How did you configure the IPv6 at install? Static, DHCP, autoconf?

[john.c]

@BenjiReis said in XCP-ng 8.3 beta :

@john-c hi thx for the feedback.

How did you configure the IPv6 at install? Static, DHCP, autoconf?

It was configured on the IPv6 at install for DHCP as allocates IP addresses from its pool. I can have it done using Autoconf for more data if required.

If requiring static based testing will require a period to figure out what the static IP will be so that it jives successfully with the router's pool.

[BenjiReis]

@john-c yes please try autoconf.

DHCP can be erratic in IPv6 i've found out. And so we might have still issue with in dom0.

[john.c]

@BenjiReis said in XCP-ng 8.3 beta :

@john-c yes please try autoconf.

DHCP can be erratic in IPv6 i've found out. And so we might have still issue with in dom0.

@BenjiReis I'll test IPv6 using Autoconf then, though the DHCP does need some work to better handle its erratic nature. Given your admission in an earlier post, so that way there's less of an issue with the method of obtaining an IPv6 address.

Especially if it's the method of obtaining an IPv6 address for dom0!! Also worth considering whether or not to switch dom0 to obtaining its IPv6 address by Autoconf instead, if its more rock solid.

Anyway it most definitely definitely needs work on DHCP client option as its the method of automatically configuring the DNS servers, when the IP address is obtained using Autoconf.

[john.c]

@BenjiReis Sorry but using Autoconf to obtain IPv6 was no good, it didn't obtain an IP address. Thus it also gets stuck and/or takes a really long time to finish the start up.

xensource.log.txt

I in the end needed to manually mount up a flash drive to copy the log due to not having a network connection to SSH into via.

Hopefully the now finally uploaded log file will give you more than enough information to finally crack this IPv6 management network IP address problem. If you manage to come up with an update or fix please send me an ISO with it on to test.