XCP-ng 8.3 betas and RCs feedback 🚀

stormi

I pushed a batch of updates to XCP-ng 8.3's base repository.

Main highlights:

Ported from XenServer 8:

Updated microcode for AMD and Intel
kernel: bug fixes
xapi: updated to version 23.31.0. Changelogs available at https://github.com/xapi-project/xen-api/releases
blktap: bug fixes
openvswitch: updated to version 2.17.7
qemu: bugfixes for PCI passthrough using multifunction devices
sm: updated to 3.0.12 + various fixes
various python2 scripts and libs ported to python3

Other XCP-ng changes:

edk2: rebuilt to embed a build of ipxe-efi which fixes PXE boot in UEFI
sm: Linstor-related fixes, IPv6-related fixes
updategrub.py script, used for adding or removing a boot entry for the alternate kernel, ported to python3.

We are also about to release the Beta2 ISOs. They won't contain the above, because we froze their contents a few weeks ago, but users who would install using these will just need to update like any other XCP-ng 8.3 tester.

POleszkiewicz

@stormi posted in storage category

Prevok

This post is deleted!

Prevok

I have an atypical host, a Rock Pi X. I can install 8.2.1 fine, but 8.3 beta1 and beta1-test4 do not install. All fresh installs, I haven't tried upgrading 8.2 to 8.3.

They generate different errors (I assume test4 is just more specific).

Using xcp-ng-8.3.0-beta1.iso I get:
Error occured
An unrecoverable error has occured. The error was:
Error installing packages

Using xcp-ng-8.3.0-beta2-test4.iso:
Error occured
An unrecoverable error has occured. The error was:
Signature key import failed

Looking forward trying the official beta2 release to see if it fares better.

Anything I can test to try and pinpoint the source of the issue?

Thanks.

stormi

@Prevok Check the date is correct on your host. A bad date (years off) messes with signature verification.

Otherwise, you can get logs with ALT + RIGHT.

ThierryC01

@stormi said in XCP-ng 8.3 beta :

Everyone: we are about to release XCP-ng 8.3 Beta 2!

It was delayed by one month due to the UEFI issues discussed above, now fixed.

The last thing we need is your feedback on the updated installation ISO: https://updates.xcp-ng.org/tmp/xcp-ng-8.3.0-beta2-test3.iso

I hope I have more success than with the test2 ISO for which I don't remember getting any feedback

Hi,

I have updated my xcp-ng 8.3 beta 1 with this xcp-ng-8.3.0-beta2-test3.iso and while it boots, I have lost all my VMs and no SR are recognized, I am left with a just a server that sits on its own and I cannot connect an oldest XOA to it. If you need more info (logs, anything) tell me now as I really need my VMs back, I think I should be able to revert to the previous version if the backup worked. I only have one homelab server so this is the situation where you kind of update your live prod...

Before upgrading, I applied all the 8.3 patches that were pushed recently.

Prevok

@stormi It was the time. It was not just a bit off, it was 12 years off

Should have thought of that, we had similar issues at work with Rocky 8 servers. I naively thought the NTP servers would have prevented such issues, but I am not sure the NIC works at install time. I have never been able to get the netinstall to work off that box. Meh... this worked, thanks you very much!

stormi

@ThierryC01 Oh, I didn't think someone would upgrade from Beta1 using the ISO. Actually the test4 ISO forbids it to avoid upgrade issues.

Can you share the /var/log/installer/install-log file? (on a separate thread, where you can ping me)

ThierryC01

@stormi Thanks for your help, my VMs are back online but there is no more stats in the tab "stats" for any VM and the host, the log page gives this error multiple times:

vm.stats
{
  "id": "3c807c46-bfdc-f33a-5954-4de92bcf3f08",
  "granularity": "seconds"
}
{
  "message": "Unable to get the true granularity: 5",
  "name": "FaultyGranularity",
  "stack": "FaultyGranularity: Unable to get the true granularity: 5
    at XapiStats._getAndUpdateStats (file:///opt/xen-orchestra/packages/xo-server/src/xapi-stats.mjs:305:13)
    at Api.#callApiMethod (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:445:20)"
}

stormi

This is an error another user reported through a support ticket. Do others confirm?

olivierlambert

I confirm the issue here

Tristis Oris

same issue.

rmaclachlan

Same here (ticket is in).

Anyone having issues with NFS SR performance too? Since the update my VMs are transferring about 140MB/s vs before the update we were seeing speeds around 440MB/s.

I'm using "dd if=/dev/zero of=tmp bs=1G oflag=dsync count=1" to test, if I SSH onto the host I can get full speed to the NFS mount but from within a VM it is much slower, this means backups are also crawling since XOA can't read the VHD very quickly.

iSCSI performance seems unaffected.

flakpyro

Same issue on latest 8.3 release, no stats.

BobTB12

@stormi
OmniOS guest fails to boot in xcp-ng on version 8.3.

Installation is fine: create a new vm, set viridian=false and installation progress smoothly. I've noticed that installation seem to lack PV-drivers.

When booting from freshly installed disk it panics with:
panic[cpu0]/thread=ffffffffffbc490c0: unable to configure /xpvd nexus

I've tried several version of OmniOS (and latest OpenIndiana), I also tried Solaris 11.4, tried disabling apix with apix_enable=0 in /etc/system and all without success.
The same setup in xcp-ng 8.0 works like a charm with working PV-drivers (but no tools).

Anyone with an idea of a workaround or even better, a resolution to this.

olivierlambert

If you have the same problem in 8.2, then it's not 8.3 related.

BobTB12

@olivierlambert oh, sorry, got too carried away and did not notice this is 8.3 beta.

kait

can confirm the stats/true granulrity issue as well

ph7

No stats :o(
but netdata seems to work

stormi

So we are working on this issue. The format of the JSON output changed, which explains why it doesn't work in XO anymore. We'll fix it either in XCP-ng or in XO, depending on what is the best thing to do.