XCP-ng 8.0.0 Release Candidate

stormi

Hello to everyone.

The RC (Release Candidate) release of XCP-ng 8.0 is available right now .

For those who already read the beta announcement, this is mostly the same post, updated for the RC.

What's new

Based on Citrix Hypervisor 8.0 (see https://docs.citrix.com/en-us/citrix-hypervisor.html).
New kernel (4.19), updated CentOS packages (7.2 => 7.5)
Xen 4.11
New hardware supported thanks to the new kernel. Some older hardware is not supported anymore. It's still expected to work in most cases but security cannot be guaranteed for those especially against side-channel attacks on legacy Intel CPUs. See Citrix's Hardware Compatibility List. Note: in 7.6 we've started providing alternate drivers for some hardware. That's something we intend to keep doing, so that we can extend the compatibility list whenever possible. See https://github.com/xcp-ng/xcp/wiki/Kernel-modules-policy.
Experimental UEFI support for guests
Quick deploy of the Xen Orchestra Appliance directly from the host's welcome HTML page. Tell us how it works for you and what you think of it!
A new implementation of the infamous emu-manager, rewritten in C. Very good results until now. Test live migrations extensively!
Mirrors: you can offer mirrors and yum now pulls from them, depending on your geographical location: https://github.com/xcp-ng/xcp/wiki/Mirrors
Includes the latest patches for MDS attacks mitigation.
The net-install installer now checks the signature of the downloaded RPMs against our GPG key, which is becoming more important now that we're delegating downloads to mirrors.
Status of our experimental packages:
- ext4 and xfs support for local SR are still considered experimental, although no one reported any issue about it. You still need to install an additional package for it to work: yum install sm-additional-drivers.
- ZFS packages are now available in the main repositories, can be installed with a simple yum install zfs, and have been updated to version 0.8.1 (as of 2019-06-18) which removes the limitations we had with the previous version in XCP-ng 7.6. Reported to work very well! By the way, Xen Orchestra now has support for adding ZFS storage repositories.
- No more modified qemu-dp for Ceph support, due to stalled issue in 7.6 (patches need to be updated). However the newer kernel brings better support for Ceph and there's some documentation in the wiki: https://github.com/xcp-ng/xcp/wiki/Ceph-on-XCP-ng-7.5-or-later.
- Alternate kernel: none available yet for 8.0, but it's likely that we'll provide one later.
Simplified RPM repository structure. updates_testing, extras and extras_testing repositories disappear and there's now simply: base, updates and testing. Additional optional packages are now in the same repos as the main packages, for simpler installation and upgrades.
It's our first release with close to 100% of the packages rebuilt in our own build infrastructure, which is based on Koji: https://koji.xcp-ng.org. Getting to this stage was a long path, so even if it changes nothing for users it's a big step for us. For the curious ones, more about the build process at https://github.com/xcp-ng/xcp/wiki/Development-process-tour.

Main additions since the beta release:

New bash prompt and XSConsole colors
cryptsetup, htop, iftop and yum-utils are now installed by default in dom0
The XAPI now automatically opens the VxLAN default port when a SDN controller is started. This is useful for the new pool-wide private networks feature from Xen Orchestra.
xcp-emu-manager bug fixes
new XAPI plugins for use by Xen Orchestra:
- ZFS pool detection for ZFS SR creation from XO
- HyperThreading detection
Updated linux guest tools: fixed CoreOS support and backported SLES 15 SP1 support from upstream.
The net-install ISO now points at the appropriate online repository for installing XCP-ng 8.0.0.

Documentation

At this stage you should already be aware that our main documentation is in our wiki and you should also know that you can all take part in completing it. Since the previous release, it has improved a lot but there's still a lot to improve.

How to upgrade

Since XCP-ng 8.0.0 is a major release, there will be no support for upgrading using yum only.

The reasons for that:

New kernel. The installer runs on the same kernel as XCP-ng 8.0, so booting the installer ISO is a way to make sure your hardware is still recognized by the new kernel.
The yum-style upgrade does not offer backing-up your system at this stage, so it's too risky for a major upgrade.

If you can't boot your host on an ISO image, you can still use the remote upgrade alternate method.

We initially planned to support it, but changed our mind due to the risks and the amount of extra work this involves. We still plan to support yum-style upgrades from 8.0 to 8.1.

Before upgrading, remember that it is a pre-release, so the risks are higher than with a final release. But if you can take the risk (very small at this stage), please do so and give us feedback! However, we've been testing it internally with success and xcp-ng.org already runs on XCP-ng 8.0 since the beta release (which includes this forum, the main repository and the mirror redirector) !

The Upgrade Howto remains mostly valid.

Standard ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-RC1.iso
Net-install ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-RC1-netinstall.iso
SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS
Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS.asc

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's shoud always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature. See https://github.com/xcp-ng/xcp/wiki/How-to-check-the-authenticity-of-files-downloaded-from-XCP-mirrors.

Update from XCP-ng 8.0.0 beta

Just run yum update and reboot, on pool master first.

Warning: do not run that command from a shell started from XSConsole. The reason for that: when the xsconsole package itself is updated, it restarts XSConsole and thus kills the current shell session, including the currently running yum command (unless you started it in the screen session). This leaves the rpm database in an unclean state.

You can check the cleanliness of yum's database with yum check.

Stay up to date

Run yum update regularly. We'll fix bugs that are detected regularly until the final release. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you need them): we'll announce every important update to the RC here.

What to test

Everything and Anything!

Report or ask anything as replies to this thread.

A community effort to list things to be tested has been started at https://github.com/xcp-ng/xcp/wiki/Test-XCP

XOA quick deploy

Just head towards the IP address of your pool master in a recent browser and try it.

XOA Quick deploy

saimon05

Thanks! Nice!

stormi

https://github.com/xcp-ng/xcp/wiki/Known-Bugs updated. Feel free to complete.

maxcuttins

averell

Just performed the upgrade from beta to RC. Went very smooth with yum upgrade.
All works well, no changes were made to the altered configurations, changes I made to NFS, exports, firewall, containers and passing thru of USB controllers still intact. Very smooth upgrade.
New version of XOA also now handles userid and password correctly.

Good job!

renx999

Is it going to be possible to upgrade from RC to release also through yum when it's final? like from beta to RC?

stormi

@renx999 said in XCP-ng 8.0.0 Release Candidate:

Is it going to be possible to upgrade from RC to release also through yum when it's final? like from beta to RC?

Yes. At worse, there may be an extra manual step but if that happens we'll document it.

FWuerl

Upgrade in legacy mode on the testpool with older IBM x3650 M4 Servers from XCP-ng 7.6 to 8.0 RC runs perfect
All networks, SRs and VMs up und running

Also a fresh installed XCP-ng 8.0 RC in UEFI mode on a Lenovo ThinkSystem SR650 with latest BIOS updates an Mellanox X5 Network Adapters works perfekt out of the box.

Nice work!!!

scpcom

Remote Upgrade from 7.6 to 8.0 RC1 does not seem to work currently (tried on two different hosts)

In dom0 shell:

get UUID of host
xe host-list
Test
xe host-call-plugin plugin=prepare_host_upgrade.py host-uuid=<uuid> fn=testUrl args:url=http://my.example.com/xs/releases/xcp-ng-8.0.0-rc1/
(returns true)
Prepare
xe host-call-plugin plugin=prepare_host_upgrade.py host-uuid=<uuid> fn=main args:url=http://my.example.com/xs/releases/xcp-ng-8.0.0-rc1/
(returns true)
reboot

Server boots , Blue background shows up
Dialog
Attempting to configure networking...
Dialog
Error occured
local variable 'results' referenced before assignment

xcp-ng-8.0-remote-upgrade-error.PNG

rj-dsl

I would be very grateful if anyone would be able to test 8.0 on the following CPUs:

E5520, E5540 and E5649

I unfortunatly dont have access to a test environment and is not able to do it myself..

So if anyone has any experiences with XCP 8.0 on these CPUs, I very interested in hearing your feedback

achim71

@scpcom

Also hit that error there is an modification required in /opt/xensource/installer/answerfile.py

def processAnswerfile(self):
        xelogging.log("Processing XML answerfile for %s." % self.operation)
        if self.operation == 'installation':
            install_type = getStrAttribute(self.top_node, ['mode'], default = 'fresh')
            results['netinstall-gpg-check'] = getBoolAttribute(self.top_node, ['netinstall-gpg-check'], default = True)
            if install_type == "fresh":
                results = self.parseFreshInstall()
            elif install_type == "reinstall":
                results = self.parseReinstall()
            elif install_type == "upgrade":
                results = self.parseUpgrade()
            else:
                raise AnswerfileException, "Unknown mode, %s" % install_type

            results.update(self.parseCommon())
        elif self.operation == 'restore':
            results = self.parseRestore()
        
        return results

Should be

def processAnswerfile(self):
        xelogging.log("Processing XML answerfile for %s." % self.operation)
        if self.operation == 'installation':
            install_type = getStrAttribute(self.top_node, ['mode'], default = 'fresh')
            results = {}
            results['netinstall-gpg-check'] = getBoolAttribute(self.top_node, ['netinstall-gpg-check'], default = True)
            if install_type == "fresh":
                results.update(self.parseFreshInstall())
            elif install_type == "reinstall":
                results.update(self.parseReinstall())
            elif install_type == "upgrade":
                results.update(self.parseUpgrade())
            else:
                raise AnswerfileException, "Unknown mode, %s" % install_type

            results.update(self.parseCommon())
        elif self.operation == 'restore':
            results = self.parseRestore()
        
        return results

In my case i started from the iso and passed an answerfile. Had to boot into manual mode switch to console #2 (ALT+F2) made the modification with vi, killed the process /opt/xensource/installer/init and launched it again using the answerfile and install options

/opt/xensource/installer/init --answerfile=[path to your answer file] --install

In your case you may have to modify the file inside the install.img at your url location.
Hope that modification will make it into the final iso.

scpcom

@achim71
Thank you for the hint. I first tried your changes but it did not work in Remote Upgrade mode (i did not use an extra answer file, I just used the same commands I posted above).
On second try I used these modifications and now it works!
https://github.com/xcp-ng/host-installer/commit/336b2b64fc0f8a6e113ca69dd3053b3124c34426#diff-6c480f72da7e1acb5eab3acf571a27e2
I unpacked the install.img, modified the script, repacked install.img and placed it on our web server.

@rj-dsl
My second test system is with X5670, but I also may have some other servers with E5520

EDIT: Remote Upgrade (with patch) worked on
Asrock FM2A88x Extreme4+ with AMD A8-6500
HP ML350 G6 with two Intel Xeon X5670

0

bdonnahue committed to xcp-ng/host-installer

[Bug fix] - Variable referenced before assignment (#11)

* [Bug fix] - Variable referenced before assignment

The installation will fail while using an answefile_generator script. The results variable is referenced before assignment.

dicode-nl

So far update from XS7.2 to XCP-ng 8.0.0RC1 went fine on Intel E5430.

bnrstnr

@rj-dsl

@bnrstnr said in XCP-ng 8.0.0 Beta now available!:

HP DL360 v7 with (2) Intel E5649 working good so far. Just finished installation and yum updates and everything came right up, as expected.

No issues in the 2 weeks since upgrading to beta and all updates between.

stormi

I made new ISOs meant to fix the unattended installation (either remote upgrade method or installation with answer file). That's the only change so I've not made it an official RC2, just some tests ISOs for the community to confirm that the bug is fixed, so that we are sure that it will not be in the final ISOs.

https://updates.xcp-ng.org/tmp/RC1custom1/

bvitnik

@stormi So is install.img the only thing changed in ISO or is there some changes to rpm packages?

stormi

@bvitnik no other change

bvitnik

Just to confirm. After copying new install.img to my PXE environment, unattended installation went smoothly with my answer file.

achim71

Tried the new ISO here as well and so far it is working with the answerfile now.
Is there an sytnax extension to configure an md root mirror in the answerfile in version 8 ?

stormi

@achim71 I don't think so. See https://github.com/xcp-ng/host-installer/tree/master/doc