XCP-ng 8.2 updates announcements and testing

cnaumer

@stormi Also installed on our test-lab and booting, migrating of RockyLinux and Windows VMs works as expected.

stormi

@cg I've not heard of it yet on Citrix side. Meanwhile the template for 2019 should be enough. IIRC @Darkbeldin has tested Windows Server 2022 regularly during the pre-release phase.

cg

@stormi sounds good. I was guessing that 2019 might do the job. Tempted to use 2022 on a fresh installation and not use the "old" 2019. (Thanks!)

HeMaN

@stormi Installed on my testlab, no issues

stormi

The security update is now live. Thank you everyone for the prompt feedback in this short timeframe!

https://xcp-ng.org/blog/2021/09/10/september-2021-security-update/

HeMaN

@stormi said in Updates announcements and testing:

There are at least two blocking issues with this update candidate, so we'll retain it until they are fixed:

HTTP 403 errors on port 443. Easily reproduced: just load XCP-ng's web page over HTTPS, most images and scripts don't load. We debugged it and reported it to the XAPI project: https://github.com/xapi-project/xen-api/issues/4517

I am using my own self signed certificate on my servers and did not notice the page looking any different, all pictures and text are there.
I do notice the "deploy xoa" script is not working. The buttons seems to be non-functional and do nothing when clicked.
All other buttons and links are functional.

On a larger screen the alignment of the page content is different (topics and pictures side by side in the old page, but in a single colomn in the new page, just like it is with the old page on smaller windows).

Another thing I think is cosmetical but annoying:
when not typing https explicitly in the browser bar (so using http) I get an "403 forbidden" message instead of being forwarded to https

Edit: browser is firefox version 91.0.2 64 bits on windows 10

gskger

@stormi Nice work and the joint effort by the community for tests gets better every time . Keep up the good work . Updated my (semi production) three host homelab and it works - as usual. Now it is time to tear down my playlab for some 10G testing .

jmccoy555

Hi @stormi is there any easy way to rollback the September updates??? I'm guessing not but I have a strange issue which I think coincides with my last reboot after applying the updates so want to confirm or eliminate as the cause.

Thanks.

stormi

@jmccoy555 Have a look at yum history. You should find the update listed and be able to rollback.

Another way is to use yum downgrade package-1version-release package2-version-release on every package involved in the update (list found in yum history or /var/log/yum.log).

stormi

Note (for you or anyone coming here later): rollback/downgrade is not officially supported, because it's not tested, and it is not always possible to ensure that it really brings the exact previous state. You wouldn't rollback a XAPI update that modified the structure of the XAPI db, for example. However for many packages it's safe to attempt it. The September update, that only contains xen packages IIRC, is one of the updates that should be easy and safe to revert.

jmccoy555

@stormi thanks as always. If I do find the issue I'll let you know..... if I break everything then I'll just get a

jmccoy555

It appears that the rollback worked, but doesn't appear to have an impact on my issue, which is good news in a way.

stormi

Let's test the next train of updates

I have various updates ready and tested internally that are eager to be pushed officially. All they need is a bit of user feedback, and that's why we're all on this thread right?

What changes

Updated ca-certificates removes an expired root certificate that was used by Let's Encrypt, in order to workaround a limitation of the old version of openssl included in XCP-ng when the chain of trust contains an expired certificate, even when another path would allow to verify the certificate. Basically, this just means that wget would fail on most HTTPS URLs that use a Let's Encrypt certificate, and now it won't fail anymore.
Updated kernel (bugfix update, already detailed above and tested by some of you)
Update (2021-10-27): new patches synced from new [Citrix hotfix](https://support.citrix.com/article/CTX330706). Removes spurious kernel warnings and supposedly increases the "resiliency" of the kernel (ie, bugs were fixed).
Updated grub fixes a booting issue with buggy UEFI firmware that only wants to boot from EFI/BOOT/BOOTX64.EFI... Or worse, firmware that doesn't really boot from this file but won't boot if the file doesn't exist...
Updated xcp-featured fixes a bug that made the Pool Secret Rotation feature (something you rarely need - as no one reported the issue - but can be useful sometimes) unavailable.
Updated guest-templates-json* packages add a VM template for Rocky Linux. It's not really different from the template for CentOS 8, but should please Rocky users.
Updated xcp-ng-release* packages bring small fixes to the XOA deploy feature on host landing web pages, and update jquery to fix an XSS vulnerability in this library.

How to update

yum clean metadata --enablerepo=xcp-ng-testing
yum update ca-certificates grub grub-efi grub-tools guest-templates-json guest-templates-json-data-linux guest-templates-json-data-other guest-templates-json-data-windows kernel xcp-featured xcp-ng-release xcp-ng-release-config xcp-ng-release-presets --enablerepo=xcp-ng-testing

Then reboot.

What to test

The same as usual: installation of the update, normal use, check that you find no obvious regressions... This is the most important.

And optionnally the changes described above if you're in a situation that allows it.

Test window before release

A few days.

What's not included in this update train

The XAPI update is not included yet due to a regression found during the tests: our landing web page was completely broken when loaded in HTTPS (which becomes the only way as HTTP is disabled with this update). We identified the issue and contributed a fix to the XAPI project. There's still some work to do internally before we can release it confidently.

ug1556

@jcastang

Hello are you using i40e driver for your network card?

HeMaN

@stormi
I just installed the updates and have not had any issues this far, will have an eye on the systemfor the next few days and report back if I notice something different.

PS you must add --enablerepo=xcp-ng-testing to the yum update command, otherwise it will not install the new packages from the test repo

stormi

@heman Thanks, fixing!

Danp

@stormi Updated and rebooted. I'll report back if I encounter any issues.

JeffBerntsen

@stormi
Working well for me so far.

stormi

So, Citrix published a new kernel hotfix yesterday, so I synced our kernel package to it. As a result, there's a new kernel package in the testing repository, so please update again if you had updated already

yum clean metadata --enablerepo=xcp-ng-testing
yum update kernel --enablerepo=xcp-ng-testing

Anyone else, just follow the instructions of the big post above, you'll get the new kernel with the other updates.

I'll update the big post with details about the new kernel update.

HeMaN

@stormi installed this new kernel as well and rebooted.
Still no regression issues found till now.