Good morning, afternoon, evening or night to everybody.

The beta release of XCP-ng 8.0 is available right now .

What's new

• Based on Citrix Hypervisor 8.0 (see https://xen-orchestra.com/blog/xenserver-7-6/).
• New kernel (4.19), updated CentOS packages (7.2 => 7.5)
• Xen 4.11
• New hardware supported thanks to the new kernel. Some older hardware is not supported anymore. It's still expected work in most cases but security cannot be guaranteed for those especially against side-channel attacks on legacy Intel CPUs. See Citrix's Hardware Compatibility List. Note: in 7.6 we've started providing alternate drivers for some hardware. That's something we intend to keep doing, so that we can extend the compatibility list whenever possible. See https://github.com/xcp-ng/xcp/wiki/Kernel-modules-policy.
• ). Note: in 7.6 we've started providing alternate drivers
• Experimental UEFI support for guests (not tested yet: have fun and report!)
• An updated welcome HTML page with the ability to install Xen Orchestra Appliance directly from there (see below). Tell us how it works for you and what you think of it!
• A new implementation of the infamous emu-manager, rewritten in C. Test live migrations extensively!
• Mirrors: you can offer mirrors and yum now pulls from them: https://github.com/xcp-ng/xcp/wiki/Mirrors
• Already includes the latest patches for MDS attacks.
• The net-install installer now checks the signature of the downloaded RPMs against our GPG key, which is becoming more important now that we're delegating downloads to mirrors.
• Status of our experimental packages:
  • ext4 and xfs support for local SR are still considered experimental, although no one reported any issue about it. You still need to install an additional package for it to work: yum install sm-additional-drivers.
  • ZFS packages are now available in the main repositories, can be installed with a simple yum install zfs, and have been updated to version 0.8.1 (as of 2019-06-18) which removes the limitations we had with the previous version in XCP-ng 7.6.
  • No more modified qemu-dp for Ceph support, due to stalled issue in 7.6 (patches need to be updated). However the newer kernel brings better support for Ceph and there's some documentation in the wiki: https://github.com/xcp-ng/xcp/wiki/Ceph-on-XCP-ng-7.5-or-later.
  • Alternate kernel: none available yet for 8.0, but it's likely that we'll provide one later.
• New repository structure. updates_testing, extras and extras_testing disappear and there's now simply: base,updatesandtesting`. Extra packages are now in the same repos as the main packages, for simpler installation and upgrades.
• It's our first release with close to 100% of the packages rebuilt in our build infrastructure, which is Koji: https://koji.xcp-ng.org. Getting to this stage was a long path, so even if it changes nothing for users it's a big step for us. For the curious ones, more about the build process at https://github.com/xcp-ng/xcp/wiki/Development-process-tour.

Documentation

At this stage you should be aware that our main documentation is in our wiki and you should also know that you can all take part in completing it. Since the previous release, it has improved a lot but there's still a lot to improve.

How to upgrade

As usual, there should be two different upgrade methods: classical upgrade via installation ISO or upgrade using yum.

However, the yum-style update is not ready yet. Since this is a major release, Citrix does not support updating using an update ISO, and the consequence for us is that the RPMs have not been carefully crafted for clean update. So it's all on us, and we plan to have this ready for the RC (release candidate).

Before upgrading, remember that it is a pre-release, so the risks are higher than with a final release. But if you can take the risk, please do and tell us how it went and what method you used! However, we've been testing it internally with success and xcp-ng.org already runs on XCP-ng 8.0 (which includes this forum, the main repository and the mirror redirector) !

Although not updated yet, the Upgrade Howto remains mostly valid (except the part about yum-style upgrade, as I said above).

• Standard ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-beta.iso
• Net-install ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-netinstall-beta.iso
  • Do not use the pre-filled URL which still points at XCP-ng 7.6. Use this instead: http://mirrors.xcp-ng.org/netinstall/8.0
• SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS
• Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS.asc

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's shoud always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature.

Stay up to date

Run yum update regularly. We'll fix bugs that are detected regularly until the release candidate. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you need them): we'll announce every update to the beta here.

What to test

Everything!

Report or ask anything as replies to this thread.

A community effort to list things to be tested has been started at https://github.com/xcp-ng/xcp/wiki/Test-XCP

Hello to everyone.

There's a very important way to prove to the world that your software is good: excellence in documentation. A project without good documentation is not appealing and looks un-professional.

I mentioned it here or there, but never made a proper announcement, so here it is: we are now using the wiki on our github project to store documentation:

https://github.com/xcp-ng/xcp/wiki

Now, as you can see, it's missing a lot of useful information. Every red link leads to a non-existent page that should exist, and I probably forgot to add many of them. Thankfully, our community comprises many experienced users of XS and XCP-ng, so we should be able to fill the gap.

I'm myself totally unable of writing documentation about storage or networking. My field of expertise is packaging, updates, releases, patching, writing some code. So I only write about what I know. But I'm sure you're not thinking that I will write the documentation alone.

So this is the official start of the "let's improve the documentation together" campaign. Any github user has write access to that wiki (it's versioned so we can revert changes from trolls if needed).

Don't ask for permission before adding or modifying something to the documentation. Just do it, and others will review it.

Even the structure of the home page can be modified. I made one to help us start and see where contributions are needed, but it's just a way to help us start.

Who's in? If you were looking for a way to get involved in XCP-ng but did not know how, here's your chance!

Hello everyone.

I'm Samuel Verschelde. I started working full-time on XCP-ng on the 18th or June, so little more than one week ago. It's a pleasure to join your community and to devote my time to this free software project.

Obviously, my first mission targets the release of XCP-ng 7.5. So I figured I'd write a status update post to let you know what's been done, what is being done and what remains to be done. It's easy to get carried away by technical work and to forget to communicate, and I'll try not to fall in this trap. Since I'm still quite new to XCP-ng, please forgive me in advance any approximation or mistake in what I'll write. Just don't hesitate to point any mistake I might be doing, or any omission.

What needs to be done

So, basically the process of releasing the new version of XCP-ng, at the moment (this will evolve), is as follows:

• Get the installation and source ISOs from XenServer 7.5.
• Rebrand to XCP-ng and remove trademarked material (Citrix name, XenServer name, Citrix logo, splash screen...).
• Modify the licensing system to remove the restrictions for features that depend on free software.
• Remove proprietary software components from Citrix that we aren't allowed to redistribute.
• Fix a few known bugs
• Create an update repository for transitioning from 7.4
• Build the installation ISO
• Test (internally at first, but we'll quickly involve the community)
• Release
• (ideally, because we're a free software project after all) Provide source RPMs.

So were are we at this stage?

Rebrand to XCP-ng and remove trademarked material

Mostly done.

We do this for legal reasons, and Citrix obviously doesn't want its name (or XenServer) to be associated with XCP-ng.
Amusingly, though, they added their name in the description of several packages in 7.5. Some of these additions are clarifications, so that's good to help us detect proprietary software or trademarked material among the free software components.

But some other are suprising, such as what they did to the linux kernel package: they added a Citrix logo image in the package (which otherwise contains free software), and then changed the package license from GPLv2 to Portions GPL, Portions Non-Redistributable (See description). And description of the package now contains:

Citrix, the Citrix logo, Xen, XenServer, and certain other marks appearing
herein are proprietary trademarks of Citrix Systems, Inc., and are
registered in the U.S. and other countries. You may not redistribute this
package, nor display or otherwise use any Citrix trademarks or any marks
that incorporate Citrix trademarks without the express prior written
authorization of Citrix. Nothing herein shall restrict your rights, if
any, in the software contained within this package under an applicable
open source license.

Portions of this package are 2017 Citrix Systems, Inc. For other
copyright and licensing information see the relevant source RPM.

A lot of claims for an added logo.

Since the kernel contained by this package is free software, we made a new package without the added claims and without that added logo, so that we can redistribute it.

Modify the licensing system to remove the restrictions for features that depend on free software

We hoped we could simply reuse our package from 7.4.1 but it seems it's not sufficient. Investigations by @johnelse are in progress. Maybe they changed the protocol, maybe we just need a rebuild against latest xapi.

Remove proprietary software components from Citrix that we aren't allowed to redistribute

Some are easy to remove, such as the ParaVirtualization (PV) tools for Windows, which are proprietary. I've been told that Xen Orchestra has support for installing them easily, so we shouldn't miss them too much.

There's one that's giving @johnelse trouble, though: EMU Manager

This closed-source component is tightly coupled to other free software core components. Since I know little about it, I'll let John explain:

emu-manager is a layer between xenopsd (the host-level domain management daemon) and xenguest (which deals with the creation of xen domains) and vgpu (which does the graphics emulation for vGPU)
It is invoked when VMs are suspended, resumed or migrated

To put things in a few words: there's a close-source component that we can't remove without losing suspend, resume and migrate features. So we have to replace it with a free-software component of our own, and that's a hard work: the interfaces are public but not the internals. And we'll lose the vGPU migration ability in the process.

We expect to finish this in two weeks from now, but that may slip. If you think you can help, contact @johnelse or come discuss on the #xcp-ng-dev IRC channel on freenode (we both are on a European timezone).

Fix a few known bugs

• Broken link to XCP-ng center in the HTML welcome page: fixed for 7.5.
• Missing /etc/shadow and other files in install.img: will be fixed when building the new ISO images.
• Fix xcp-python-libs bug mentioned at https://xcp-ng.org/forum/topic/79/configure-dom0-memory-not-work
• Maybe more? I'm not sure this list is exhaustive.

Create an update repository for transitioning from 7.4

Nearly done.

Build the installation ISO, Test, Release

Depends on the above other tasks.

Community involvement

Now that I'm available almost all the time, it should become easier to include contributers from the community. We don't have the target infrastructure for that yet, but it's already possible to help at various stages. So don't hesitate to discuss it on this forum, to contact me ( stormi-xcp@ylix.fr ) or to come to chat on freenode's #xcp-ng-dev IRC channel.

And if you read all of this, thanks!

It's here, finally.

I will not re-detail everything that this release brings, because everything is available here:

• 8.0.0 Release Candidate announcement
• 8.0.0 Official Release announcement

A huge thank you to all of you who have been involved in testing and more.

For you, testers, some information that is not on the blog:

• If you installed XCP-ng 8.0.0 prefinal ISOs from the RC thread, then you actually already have XCP-ng 8.0.0 final! Nothing to do.
• If you installed XCP-ng 8.0.0 beta or RC, just update your hosts like you would do for any regular update and you'll have the same system as if you had installed from the final ISO. Just avoid doing it from the shell console that is started from xsconsole (see why in the updates howto).

Coming. Very. Soon.

Hello to everyone.

The RC (Release Candidate) release of XCP-ng 8.0 is available right now .

For those who already read the beta announcement, this is mostly the same post, updated for the RC.

What's new

• Based on Citrix Hypervisor 8.0 (see https://docs.citrix.com/en-us/citrix-hypervisor.html).
• New kernel (4.19), updated CentOS packages (7.2 => 7.5)
• Xen 4.11
• New hardware supported thanks to the new kernel. Some older hardware is not supported anymore. It's still expected to work in most cases but security cannot be guaranteed for those especially against side-channel attacks on legacy Intel CPUs. See Citrix's Hardware Compatibility List. Note: in 7.6 we've started providing alternate drivers for some hardware. That's something we intend to keep doing, so that we can extend the compatibility list whenever possible. See https://github.com/xcp-ng/xcp/wiki/Kernel-modules-policy.
• Experimental UEFI support for guests
• Quick deploy of the Xen Orchestra Appliance directly from the host's welcome HTML page. Tell us how it works for you and what you think of it!
• A new implementation of the infamous emu-manager, rewritten in C. Very good results until now. Test live migrations extensively!
• Mirrors: you can offer mirrors and yum now pulls from them, depending on your geographical location: https://github.com/xcp-ng/xcp/wiki/Mirrors
• Includes the latest patches for MDS attacks mitigation.
• The net-install installer now checks the signature of the downloaded RPMs against our GPG key, which is becoming more important now that we're delegating downloads to mirrors.
• Status of our experimental packages:
  • ext4 and xfs support for local SR are still considered experimental, although no one reported any issue about it. You still need to install an additional package for it to work: yum install sm-additional-drivers.
  • ZFS packages are now available in the main repositories, can be installed with a simple yum install zfs, and have been updated to version 0.8.1 (as of 2019-06-18) which removes the limitations we had with the previous version in XCP-ng 7.6. Reported to work very well! By the way, Xen Orchestra now has support for adding ZFS storage repositories.
  • No more modified qemu-dp for Ceph support, due to stalled issue in 7.6 (patches need to be updated). However the newer kernel brings better support for Ceph and there's some documentation in the wiki: https://github.com/xcp-ng/xcp/wiki/Ceph-on-XCP-ng-7.5-or-later.
  • Alternate kernel: none available yet for 8.0, but it's likely that we'll provide one later.
• Simplified RPM repository structure. updates_testing, extras and extras_testing repositories disappear and there's now simply: base, updates and testing. Additional optional packages are now in the same repos as the main packages, for simpler installation and upgrades.
• It's our first release with close to 100% of the packages rebuilt in our own build infrastructure, which is based on Koji: https://koji.xcp-ng.org. Getting to this stage was a long path, so even if it changes nothing for users it's a big step for us. For the curious ones, more about the build process at https://github.com/xcp-ng/xcp/wiki/Development-process-tour.

Main additions since the beta release:

• New bash prompt and XSConsole colors
• cryptsetup, htop, iftop and yum-utils are now installed by default in dom0
• The XAPI now automatically opens the VxLAN default port when a SDN controller is started. This is useful for the new pool-wide private networks feature from Xen Orchestra.
• xcp-emu-manager bug fixes
• new XAPI plugins for use by Xen Orchestra:
  • ZFS pool detection for ZFS SR creation from XO
  • HyperThreading detection
• Updated linux guest tools: fixed CoreOS support and backported SLES 15 SP1 support from upstream.
• The net-install ISO now points at the appropriate online repository for installing XCP-ng 8.0.0.

Documentation

At this stage you should already be aware that our main documentation is in our wiki and you should also know that you can all take part in completing it. Since the previous release, it has improved a lot but there's still a lot to improve.

How to upgrade

Since XCP-ng 8.0.0 is a major release, there will be no support for upgrading using yum only.

The reasons for that:

• New kernel. The installer runs on the same kernel as XCP-ng 8.0, so booting the installer ISO is a way to make sure your hardware is still recognized by the new kernel.
• The yum-style upgrade does not offer backing-up your system at this stage, so it's too risky for a major upgrade.

If you can't boot your host on an ISO image, you can still use the remote upgrade alternate method.

We initially planned to support it, but changed our mind due to the risks and the amount of extra work this involves. We still plan to support yum-style upgrades from 8.0 to 8.1.

Before upgrading, remember that it is a pre-release, so the risks are higher than with a final release. But if you can take the risk (very small at this stage), please do so and give us feedback! However, we've been testing it internally with success and xcp-ng.org already runs on XCP-ng 8.0 since the beta release (which includes this forum, the main repository and the mirror redirector) !

The Upgrade Howto remains mostly valid.

• Standard ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-RC1.iso
• Net-install ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-RC1-netinstall.iso
• SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS
• Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS.asc

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's shoud always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature. See https://github.com/xcp-ng/xcp/wiki/How-to-check-the-authenticity-of-files-downloaded-from-XCP-mirrors.

Update from XCP-ng 8.0.0 beta

Just run yum update and reboot, on pool master first.

Warning: do not run that command from a shell started from XSConsole. The reason for that: when the xsconsole package itself is updated, it restarts XSConsole and thus kills the current shell session, including the currently running yum command (unless you started it in the screen session). This leaves the rpm database in an unclean state.

You can check the cleanliness of yum's database with yum check.

Stay up to date

Run yum update regularly. We'll fix bugs that are detected regularly until the final release. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you need them): we'll announce every important update to the RC here.

What to test

Everything and Anything!

Report or ask anything as replies to this thread.

A community effort to list things to be tested has been started at https://github.com/xcp-ng/xcp/wiki/Test-XCP

XOA quick deploy

Just head towards the IP address of your pool master in a recent browser and try it.

Hello everyone.

I am in the process of writing docs in the wiki to document the process of building XCP-ng.

So now there's a Development Process Tour page in the wiki, with lots of information (and still a few TODOs here and there).

Feedback welcome as usual

Hi. Everything is ready except the replacement to EMU-manager, which is still in development. I'll announce it once we have a release candidate for those willing to test it before the broader release.

Installing Xen Orchestra Appliance directly from the browser

The Xen Orchestra team has provided us with a way to install XOA directly from your browser by simply visiting the IP address of your freshly installed host.

Try it!

tl;dr: we need you to test a spin of XCP-ng with updated CentOS packages

As you probably know, XenServer and XCP-ng are built on top of CentOS. Approximately 75% of the dom0 system is made of packages from CentOS 7.

Most of those packages haven't been updated since XenServer 7.0 was released. The reason for this is probably this: packages that don't change don't bring new bugs. And it's understandable: we all like stability and hate regressions.

However, XCP-ng has an objective of being as close as possible from upstream CentOS. We don't know yet how close we can be, but we think we can be closer than what we are right now.

So I spent several days studying the packages from CentOS, looking at their changelogs, and we finally decided to make an experimental repository in which we would update all the packages that we could, shake vigorously, and see that comes out of this. This repository is now ready. We don't know what the outcome of this experiment will be, but it will for sure help us understand the system and probably allow us to get closer to upstream CentOS sooner or later.

Now we need the most adventurous among you to test it. Not in production, of course!

How to test

On top of an existing XCP-ng 7.6:

wget https://updates.xcp-ng.org/tmp/centos-update/xcp-ng-7.6.repo -O /etc/yum.repos.d/xcp-ng.repo
yum clean metadata
yum update
reboot

Then the game starts: if you find a regression, you win! If you help find why, you double win (... our gratitude )!

If you're in, please say so in this thread, tell us what kind of setup you'll be able to test, and make sure to watch the thread and activate mail notifications if needed to stay informed of future findings and updates.

The packages are synced with CentOS 7.5 for now, but CentOS 7.6 has been released so expect another update.

Security Updates released. Intel hardware again. You'll need to choose between safety and performance regarding one of the flaws if you are running untrusted guests. https://xcp-ng.org/blog/2019/11/18/security-updates-for-intel-hardware/

There's a misunderdstanding when I ask for tests. What hardware you have doesn't really matter. Even if you don't have Intel CPUs, your hosts will receive the updates. So they are impacted. If there are regressions in the updates, you'll be impacted.

That's why what's the most important with all the updates I ask feedback for is not what they fix, it's what they don't break. A large number of "I installed them, things seem to still function normally" is what we really need.

Testing the actual changes the updates bring is a plus, but not a requirement to be able to contribute.

End of WE soon, still interested in feedback for release tomorrow.

Testing: November 2019 security update for XCP-ng, 2nd Impact

New security update candidates are available (XCP-ng 7.6 and 8.0). As usual, I will only release them to everyone after enough testing, so if you can please install the update candidates.

Instructions there: https://github.com/xcp-ng/xcp/issues/308

Please use the github issues to provide feedback so this thread here remains clean.

stormi created this issue in xcp-ng/xcp

open Nov 2019 security updates, 2nd impact #308

Get ready for testing. New (previously unforeseen) security updates are coming soon for testing. You can thank hardware bugs in CPUs (and it's only starting)

Updates published: https://xcp-ng.org/blog/2019/11/08/xcp-ng-security-bulletin-november-2019/

Last chance to test before I release them based only on our internal testing.

So don't trust XCP-ng Center on this, I'd say

PV-drivers-detected

Could you also check with the xe vm-param-list uuid=YOUR_VM_UUID command?