Best posts made by stormi

Good morning, afternoon, evening or night to everybody.

The beta release of XCP-ng 8.0 is available right now .

What's new

• Based on Citrix Hypervisor 8.0 (see https://xen-orchestra.com/blog/xenserver-7-6/).
• New kernel (4.19), updated CentOS packages (7.2 => 7.5)
• Xen 4.11
• New hardware supported thanks to the new kernel. Some older hardware is not supported anymore. It's still expected work in most cases but security cannot be guaranteed for those especially against side-channel attacks on legacy Intel CPUs. See Citrix's Hardware Compatibility List. Note: in 7.6 we've started providing alternate drivers for some hardware. That's something we intend to keep doing, so that we can extend the compatibility list whenever possible. See https://github.com/xcp-ng/xcp/wiki/Kernel-modules-policy.
• ). Note: in 7.6 we've started providing alternate drivers
• Experimental UEFI support for guests (not tested yet: have fun and report!)
• An updated welcome HTML page with the ability to install Xen Orchestra Appliance directly from there (see below). Tell us how it works for you and what you think of it!
• A new implementation of the infamous emu-manager, rewritten in C. Test live migrations extensively!
• Mirrors: you can offer mirrors and yum now pulls from them: https://github.com/xcp-ng/xcp/wiki/Mirrors
• Already includes the latest patches for MDS attacks.
• The net-install installer now checks the signature of the downloaded RPMs against our GPG key, which is becoming more important now that we're delegating downloads to mirrors.
• Status of our experimental packages:
  • ext4 and xfs support for local SR are still considered experimental, although no one reported any issue about it. You still need to install an additional package for it to work: yum install sm-additional-drivers.
  • ZFS packages are now available in the main repositories, can be installed with a simple yum install zfs, and have been updated to version 0.8.1 (as of 2019-06-18) which removes the limitations we had with the previous version in XCP-ng 7.6.
  • No more modified qemu-dp for Ceph support, due to stalled issue in 7.6 (patches need to be updated). However the newer kernel brings better support for Ceph and there's some documentation in the wiki: https://github.com/xcp-ng/xcp/wiki/Ceph-on-XCP-ng-7.5-or-later.
  • Alternate kernel: none available yet for 8.0, but it's likely that we'll provide one later.
• New repository structure. updates_testing, extras and extras_testing disappear and there's now simply: base, updatesandtesting`. Extra packages are now in the same repos as the main packages, for simpler installation and upgrades.
• It's our first release with close to 100% of the packages rebuilt in our build infrastructure, which is Koji: https://koji.xcp-ng.org. Getting to this stage was a long path, so even if it changes nothing for users it's a big step for us. For the curious ones, more about the build process at https://github.com/xcp-ng/xcp/wiki/Development-process-tour.

Documentation

At this stage you should be aware that our main documentation is in our wiki and you should also know that you can all take part in completing it. Since the previous release, it has improved a lot but there's still a lot to improve.

How to upgrade

As usual, there should be two different upgrade methods: classical upgrade via installation ISO or upgrade using yum.

However, the yum-style update is not ready yet. Since this is a major release, Citrix does not support updating using an update ISO, and the consequence for us is that the RPMs have not been carefully crafted for clean update. So it's all on us, and we plan to have this ready for the RC (release candidate).

Before upgrading, remember that it is a pre-release, so the risks are higher than with a final release. But if you can take the risk, please do and tell us how it went and what method you used! However, we've been testing it internally with success and xcp-ng.org already runs on XCP-ng 8.0 (which includes this forum, the main repository and the mirror redirector) !

Although not updated yet, the Upgrade Howto remains mostly valid (except the part about yum-style upgrade, as I said above).

• Standard ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-beta.iso
• Net-install ISO: http://mirrors.xcp-ng.org/isos/8.0/xcp-ng-8.0.0-netinstall-beta.iso
  • Do not use the pre-filled URL which still points at XCP-ng 7.6. Use this instead: http://mirrors.xcp-ng.org/netinstall/8.0
• SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS
• Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.0/SHA256SUMS.asc

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's shoud always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature.

Stay up to date

Run yum update regularly. We'll fix bugs that are detected regularly until the release candidate. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you need them): we'll announce every update to the beta here.

What to test

Everything!

Report or ask anything as replies to this thread.

A community effort to list things to be tested has been started at https://github.com/xcp-ng/xcp/wiki/Test-XCP

Hello to everyone.

The Release Candidate (RC) release of XCP-ng 8.1 is available right now .

This post is a mostly a copy-paste from the beta release announcement, with additions regarding the RC release.

Stability status

Good as far as we call tell from our tests and beta feedback. I would personally use it in production (actually we probably will very soon at Vates). The main unknown is how well the new coalesce algorithm will perform. Now is the time for as large testing as possible before the final release.

What changed

Here's what changes from XCP-ng 8.0. Changes since the 8.1 beta are in bold characters.

Non-exhaustive list:

• Based on Citrix Hypervisor 8.1 (https://docs.citrix.com/en-us/citrix-hypervisor/whats-new.html).
• Same base version of CentOS (7.5)
• Same base version of kernel (4.19, with more patches of course)
  • Latest kernel hotfix from CH 8.1 included in the RC release.
• Xen 4.13
• UEFI support for guests is not experimental anymore
• Support for Secure Boot is NOT INCLUDED because it relies on Proprietary packages. We're raising the issue with the Xen community to see how to provide a FOSS implementation.
• Citrix announces:
  • "Improved performance for VM imports and exports that use the XVA format" thanks to the use of a very efficient hash algorithm
  • "Storage performance improvements"
  • "New Windows I/O drivers with improved performance"
  • (From https://bugs.xenserver.org/browse/XSO-951) The next update (apparently not published yet as of 2020-01-27) of the Windows Drivers through Windows Update should at last be compatible with non-english locales!
• chrony replaces ntp
• Support for PV guests is now deprecated
  • templates for creating PV guests have been removed
  • existing guests will still run
  • it is advised to convert them to HVM guests
  • a compatibility layer should be provided in the future for PV guests that really can't be converted. But really anyone who can convert to HVM, should
  • due to how 32-bit PV guests work, keeping it functioning on newer hardware with newer features comes with an increasing performance cost, and the linux kernel is about to drop support for 32-bit PV guests
• Support for AMD EPYC 7xx2(P) added
• Dynamic Memory Control (DMC) is deprecated and will be removed in the future.
• VSS and quiesced snapshots support is removed, because it never worked correctly and caused more harm than good.
  • For backups, we are working on support for backups that include the guest RAM to replace the need for quiesced snapshots.
  • Note that Windows guest tools version 9 (the default for recent versions of Windows if you install through Windows Update) already removed VSS support, even for older versions of CH / XCP-ng
• For new local storage repositories using the EXT filesystem, it now defaults to ext4.
  • The transition from our experimental ext4 storage driver is not automatic.
  • Instructions for transitioning written in specific section below.
  • sm-additional-drivers package updated to refuse to create new experimental ext4 SRs since ext4 is already the default with the ext driver.
• Status of our specific packages:
  • Experimental support for XFS in local storage repository still available through the sm-additional-drivers package.
  • ZFS still available as an additional package and updated to 0.8.3.
  • Alternate kernel now available, version 4.19.102 (probable update to 4.19.108 before final release). Installing it now automatically adds a new boot entry in grub's configuration, to make testing easier. Default entry remains that of the main kernel.
  • netdata RPMs are now ready
    • @r1 contributed a fix to netdata project to bring support for Xen 4.13
    • @stormi made netdata cache be RAM-only to workaround an upstream bug that could make the disk cache grow forever
• Plans are laid out for simpler installation and maintenance of Windows guest tools. Unfortunately, we haven't found resources yet to implement them so the current state remains that of 8.0. If you're a developer on the Windows platforms, we're hiring! (full time or part time, contracts or hires) - Contact us.
• Installer improvements in 8.1 RC: see section below.
• New leaf coalesce logic of VDIs with dynamic limits: see section below.
• Fixed VM Autostart issues in 8.1 beta when updated from 8.0 through yum (upstream bug https://bugs.xenserver.org/browse/XSO-978)
• Fixed netxtreme drivers (bnx2x module) that crashed with some models
• Updated zstd to 1.4.4. Possibly even better perfs according to upstream. Still no import/export comparative benchmarks to 8.0. Does anyone volunteer?

Installer improvements in 8.1 RC

Our installer now offers two new installation options. In BIOS mode, access them with F2 when offered the choice. In UEFI mode, see the added boot menu entries.

• First new option: boot the installer with a 2G RAM limit instead of the 8G default. This is a workaround for installation issues on hardware with Ryzen CPUs. Though those are Desktop-class CPUs and not supported officially in the HCL, we tried to make it easier to workaround the infamous "installer crashes on Ryzen" issue.
• Second new option: boot the installer with our alternate kernel (kernel-alt). That kernel, built and maintained by @r1 for the team, is based on the main kernel, with all upstream kernel.org patches from the LTS 4.19 branch applied. It should be very stable by construction but it receives less testing. That option is there for cases when the main kernel and drivers have issues, so that you can quickly test if kernel.org patches have fixed it already. It will also install the alternate kernel in addition to the main kernel as a convenience. If kernel-alt fixes issues for you, the most important thing to do is to tell us so that we may fix the main kernel!

Announcement about our former experimental ext4 SR driver

It is now deprecated in 8.1. For a good reason: in XCP-ng 8.1 and above, following upstream changes, the ext driver now formats new SRs as EXT4. Existing SRs are untouched (so remain formatted to EXT3).

There is no easy way to convert an existing SR created with our driver, so those using it will need to move the VDIs out (to another SR or to export them), destroy the SR and create an EXT SR instead. Make sure to do this on XCP-ng 8.1.

The sm-additional-drivers package remains available in XCP-ng 8.1 in order to ease the transition. However I've broken the sr-create command on purpose. Any attempt to create a SR of type ext4 will result in an error with a message that explains that you need to use the ext type instead.

Our experimental driver will be completely removed in a later release, possibly XCP-ng 8.2. Unless someone convinces me to delay the removal for a good reason. I will accept reasons such as "I know I shouldn't have used the experimental driver in production, but I did and need more time to convert my SR while at the same time I really need feature xxx from XCP-ng 8.2", but I really would prefer to drop it in 8.2.

Feedback from people doing the transition is welcome to make sure we document the transition in the best way possible.

New leaf coalesce logic with dynamic limits

I have backported patches from sm's master branch, that implement a new, smarter, logic for leaf coalescing.

Those interested in the patches, see https://github.com/xcp-ng-rpms/sm/commit/ed1a55d727846cf5777c8258e6a8f3b068e8a35b (python code).

Feedback wanted.

Documentation

At this stage you should already be aware that our main documentation is in our wiki and you should also know that you can all take part in completing it. It's improving continuously and still has room for improvement.

How to upgrade from XCP-ng 8.1 beta

Just use yum update or Xen Orchestra as if you were installing updates to a stable release.

How to upgrade from previous releases

Since XCP-ng 8.1.0 is a minor release, both upgrade methods are supported:

• From the installation ISO
• From command line using yum (from XCP-ng 8.0 only!)

Refer to the upgrade howto: https://github.com/xcp-ng/xcp/wiki/Upgrade-Howto

Downloads:

• Standard ISO: http://mirrors.xcp-ng.org/isos/8.1/xcp-ng-8.1.0-RC1.iso
• Net-install ISO: http://mirrors.xcp-ng.org/isos/8.1/xcp-ng-8.1.0-RC1-netinstall.iso
• SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.1/SHA256SUMS
• Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.1/SHA256SUMS.asc

If your browser tells you that the page is redirected in a way that prevents from loading the page correctly, this is a known issue. Long story short, it's caused by a wrong security header sent by the forum software, which impacts all non HTTPS requests to the xcp-ng.org domain. Try in Private Navigation mode or use curl or wget.

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's should always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature. See https://github.com/xcp-ng/xcp/wiki/How-to-check-the-authenticity-of-files-downloaded-from-XCP-mirrors.

Stay up to date

Run yum update regularly. We'll fix bugs that are detected regularly until the final release. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you need them): we'll announce every important update to the RC here.

What to test

Everything and Anything!

Report or ask anything as replies to this thread.

A community effort to list things to be tested has been started at https://github.com/xcp-ng/xcp/wiki/Test-XCP

Feedback is welcome both about successes and failures.

Hello to everyone.

There's a very important way to prove to the world that your software is good: excellence in documentation. A project without good documentation is not appealing and looks un-professional.

I mentioned it here or there, but never made a proper announcement, so here it is: we are now using the wiki on our github project to store documentation:

https://github.com/xcp-ng/xcp/wiki

Now, as you can see, it's missing a lot of useful information. Every red link leads to a non-existent page that should exist, and I probably forgot to add many of them. Thankfully, our community comprises many experienced users of XS and XCP-ng, so we should be able to fill the gap.

I'm myself totally unable of writing documentation about storage or networking. My field of expertise is packaging, updates, releases, patching, writing some code. So I only write about what I know. But I'm sure you're not thinking that I will write the documentation alone.

So this is the official start of the "let's improve the documentation together" campaign. Any github user has write access to that wiki (it's versioned so we can revert changes from trolls if needed).

Don't ask for permission before adding or modifying something to the documentation. Just do it, and others will review it.

Even the structure of the home page can be modified. I made one to help us start and see where contributions are needed, but it's just a way to help us start.

Who's in? If you were looking for a way to get involved in XCP-ng but did not know how, here's your chance!

It's now live!

https://xcp-ng.org/blog/2022/11/18/xcp-ng-8-3-alpha/

Hello everyone.

I'm Samuel Verschelde. I started working full-time on XCP-ng on the 18th or June, so little more than one week ago. It's a pleasure to join your community and to devote my time to this free software project.

Obviously, my first mission targets the release of XCP-ng 7.5. So I figured I'd write a status update post to let you know what's been done, what is being done and what remains to be done. It's easy to get carried away by technical work and to forget to communicate, and I'll try not to fall in this trap. Since I'm still quite new to XCP-ng, please forgive me in advance any approximation or mistake in what I'll write. Just don't hesitate to point any mistake I might be doing, or any omission.

What needs to be done

So, basically the process of releasing the new version of XCP-ng, at the moment (this will evolve), is as follows:

• Get the installation and source ISOs from XenServer 7.5.
• Rebrand to XCP-ng and remove trademarked material (Citrix name, XenServer name, Citrix logo, splash screen...).
• Modify the licensing system to remove the restrictions for features that depend on free software.
• Remove proprietary software components from Citrix that we aren't allowed to redistribute.
• Fix a few known bugs
• Create an update repository for transitioning from 7.4
• Build the installation ISO
• Test (internally at first, but we'll quickly involve the community)
• Release
• (ideally, because we're a free software project after all) Provide source RPMs.

So were are we at this stage?

Rebrand to XCP-ng and remove trademarked material

Mostly done.

We do this for legal reasons, and Citrix obviously doesn't want its name (or XenServer) to be associated with XCP-ng.
Amusingly, though, they added their name in the description of several packages in 7.5. Some of these additions are clarifications, so that's good to help us detect proprietary software or trademarked material among the free software components.

But some other are suprising, such as what they did to the linux kernel package: they added a Citrix logo image in the package (which otherwise contains free software), and then changed the package license from GPLv2 to Portions GPL, Portions Non-Redistributable (See description). And description of the package now contains:

Citrix, the Citrix logo, Xen, XenServer, and certain other marks appearing
herein are proprietary trademarks of Citrix Systems, Inc., and are
registered in the U.S. and other countries. You may not redistribute this
package, nor display or otherwise use any Citrix trademarks or any marks
that incorporate Citrix trademarks without the express prior written
authorization of Citrix. Nothing herein shall restrict your rights, if
any, in the software contained within this package under an applicable
open source license.

Portions of this package are 2017 Citrix Systems, Inc. For other
copyright and licensing information see the relevant source RPM.

A lot of claims for an added logo.

Since the kernel contained by this package is free software, we made a new package without the added claims and without that added logo, so that we can redistribute it.

Modify the licensing system to remove the restrictions for features that depend on free software

We hoped we could simply reuse our package from 7.4.1 but it seems it's not sufficient. Investigations by @johnelse are in progress. Maybe they changed the protocol, maybe we just need a rebuild against latest xapi.

Remove proprietary software components from Citrix that we aren't allowed to redistribute

Some are easy to remove, such as the ParaVirtualization (PV) tools for Windows, which are proprietary. I've been told that Xen Orchestra has support for installing them easily, so we shouldn't miss them too much.

There's one that's giving @johnelse trouble, though: EMU Manager

This closed-source component is tightly coupled to other free software core components. Since I know little about it, I'll let John explain:

emu-manager is a layer between xenopsd (the host-level domain management daemon) and xenguest (which deals with the creation of xen domains) and vgpu (which does the graphics emulation for vGPU)
It is invoked when VMs are suspended, resumed or migrated

To put things in a few words: there's a close-source component that we can't remove without losing suspend, resume and migrate features. So we have to replace it with a free-software component of our own, and that's a hard work: the interfaces are public but not the internals. And we'll lose the vGPU migration ability in the process.

We expect to finish this in two weeks from now, but that may slip. If you think you can help, contact @johnelse or come discuss on the #xcp-ng-dev IRC channel on freenode (we both are on a European timezone).

Fix a few known bugs

• Broken link to XCP-ng center in the HTML welcome page: fixed for 7.5.
• Missing /etc/shadow and other files in install.img: will be fixed when building the new ISO images.
• Fix xcp-python-libs bug mentioned at https://xcp-ng.org/forum/topic/79/configure-dom0-memory-not-work
• Maybe more? I'm not sure this list is exhaustive.

Create an update repository for transitioning from 7.4

Nearly done.

Build the installation ISO, Test, Release

Depends on the above other tasks.

Community involvement

Now that I'm available almost all the time, it should become easier to include contributers from the community. We don't have the target infrastructure for that yet, but it's already possible to help at various stages. So don't hesitate to discuss it on this forum, to contact me ( stormi-xcp@ylix.fr ) or to come to chat on freenode's #xcp-ng-dev IRC channel.

And if you read all of this, thanks!

FYI, I have just published security updates today PLUS the fixed ixgbe driver as an official update to XCP-ng 8.1 and 8.2.

We made it. This is the end of this huge thread.

A big thank you to everyone involved in debugging the issue.

And this is not a :D.

Context: various recent devices on the consumer market require network drivers which are not present in XCP-ng 8.2.

Based on the work made with @Andrew for XCP-ng 8.3, I've finally found some time to backport the drivers to XCP-ng 8.2.

The following optional driver packages will soon be updated on the default repositories:

• igc-module (backported from kernel 5.10.150): adds support for Intel I225/I226.
• intel-e1000e-alt (updated to 3.8.7): adds support for Intel I219-V.
• r8125-module (updated to 9.010.01) for RealTek r8125. Quoting @Andrew:

  Note: This 8125 driver has SG/TSO, PTP, RSS enabled by default.
  Note: This 8125 driver has (optional) firmware loading enabled. Please manually download the rtl8125 firmware and install in XCP in /usr/lib/firmware/rtl_nic .
  Note: The 8125 Chipset/PCI-E card/driver has been known to cause system problems and crashes (not an XCP problem).

These are Additional packages, which means only best effort support is provided for them.

Update (2023-05-03): packages now moved to the xcp-ng-updates repository, which means they can be installed with a simple yum install name_of_package, without the need for extra parameters.

I'm publishing new updates to the base repository of XCP-ng 8.3:

• Security fixes for AMD
• Debian 12 VM template
• Removal of the old and unused experimental EXT4 SR driver. Don't jump: the main EXT SR driver still uses EXT4. I'm talking of the old experimental driver we added back then when the default EXT driver would use EXT3 only. This experimental driver has been deprecated since XCP-ng 8.1.
• smartmontools updated to version 7 which brings JSON output
• A fix for live migration support in IPv6-only pools

XCP-ng 8.3 beta 1 is released!

Announcement: https://xcp-ng.org/blog/2023/06/22/xcp-ng-8-3-beta-1/

This thread is the place where we can discuss this release and where all testers can provide feedback.

Update: AND now Beta 2 released too. We're keeping the same thread.

Hello here! I hope you are ready, because we'll have a train of update candidates for you to test shortly

I just pushed a batch of updates to the 8.3/base repository.

You can update your 8.3 testing hosts with yum update, and reboot.

There isn't much on the side of user-visible changes, but this catches up with several months of changes made by both the XenServer team and us. Things are mostly evolving under the hood, so at this stage what matters is that the changes did not cause any regressions. Our internal test suite passed successfully.

Let's mention a few key changes from a very long changelog, though:

• All latest security fixes
• Performance optimizations in XAPI components. This means better performance for dom0 in various situations. This doesn't change the performance of VMs themselves, unless they are in situations where their performance was impacted by XAPI.
• Basic vTPM support (no snapshots, no migration, no HA...). We haven't tested it yet so I can't guide you towards testing it yet.
• Various fixes, as always.
• Work on HTTPS support for all operations (for example storage migration) so that port 80 can be eventually closed. A lot was done, but I'm not sure it's complete yet.
• Beginning of transition from python2 to python3: a few scripts ported to python3, and so python3 is now installed alongside python2 in dom0, and code will be ported progressively.
• Various driver updates

Hello to everyone.

The RC (Release Candidate) release of XCP-ng 8.2 is available right now .

Stability status

Our internal tests have not revealed any major bugs. Community feedback during the beta phase was positive.

We expect the RC phase to be quite short (around two weeks), so don't wait too much or you will not be able to help us test it!

Long Term Support (LTS)

XCP-ng 8.2 will be our first LTS release.

Help us make it rock-solid through your tests!

What changed

Here are the main changes since XCP-ng 8.1. Changes since the beta are in bold characters.

XCP-ng 8.2 is based on the open source components of Citrix Hypervisor 8.2 (https://docs.citrix.com/en-us/citrix-hypervisor/whats-new.html).

Base components:

• Xen 4.13.1 + patches
• Kernel 4.19 + patches

Citrix announces:

• Support for SLES 12 SP5 and Ubuntu 20.04 added
• Support for Windows 7, Windows Server 2008 SP2 and Windows Server 2008 R2 SP1 removed. They may still work, but are not supported officially nor tested anymore by Citrix. Consider upgrading.
• Maximum host RAM raised to 6 TB and maximum number of logical processors per host raised to 448 CPUs.
• Support for installing TLS certificates on hosts (see https://docs.citrix.com/en-us/citrix-hypervisor/hosts-pools.html#install-a-tls-certificate-on-your-server)
• TLS 1.2 protocol enforced for HTTPS traffic, and between XCP-ng components. Support for legacy SSL mode and TLS 1.0/1.1 protocols have been removed.

Other changes:

• Bug fixed for hosts configured with DHCP. xcp-networkd used not to send the hostname along with the DHCP request. We contributed a fix to the xcp-networkd package, that was included in Citrix Hypervisor 8.2 and thus in XCP-ng 8.2 (note: packages with that fix are also available for XCP-ng 8.0 and 8.1 in the xcp-ng-testing repository).
• Backup restore fixed for UEFI hosts. This upstream bug that our community had found and that we reported to Citrix teams has been fixed by them.

Changes specific to XCP-ng:

• Support for Intel IceLake and CometLake CPUs.
• Reimplementation of UEFI VM support : See dedicated section below for details.
• New storage drivers: ZFS, GlusterFS, CephFS... and other changes. See dedicated section below for details.
• The updated Xen brings support for Core Scheduling. We added support for changing the host configuration through the XAPI and contributed it upstream. See dedicated section below.
• Guest tools ISO: Citrix doesn't provide a guest tools ISO anymore in Citrix Hypervisor. They replaced it with downloads from their internet website. We chose to retain the feature and still provide a guest tools ISO that you can mount to your VMs. Many thanks to the XAPI developers who have kept the related source code for us in the XAPI project, rather than delete it.
• OpenFlow support in the SDN controller: support added out of the box for Xen Orchestra to leverage it. The related code is being contributed upstream to the XAPI project.
• We replaced Citrix's gpumon package, not built by us, by a mock build of gpumon sources, without the proprietary nvidia developer kit. See dedicated section below
• Alternate kernel updated to version 4.19.140.
• Intel's e1000e driver updated to version 3.8.4 in order to support more devices
• rsyslog synced from latest CentOS 7.8 security and bugfix update because several memory leaks have been patched. Please give special attention to the logging system during your tests..
• Additional packages:
  • ZFS updated to 0.8.5
  • zstd updated to 1.4.5
  • glusterfs 8.1 added to the repositories
  • New additional driver package: r8125-module
  • Alternate driver package intel-igb-alt updated to version 5.4.6

Reimplementation of UEFI VM support

The component that handles UEFI variables for VMs, varstored, was redistributable but not open source. So we had to implement our own, named uefistored.

Since the component was entirely rewritten, we ask the community to give special attention to UEFI VM tests.

Support for Secure Boot in VMs will also arrive soon.

New storage drivers: ZFS, GlusterFS, CephFS... and other changes

We added three new experimental storage drivers: zfs, glusterfs and cephfs.

We also decided to remove the sm-additional-drivers package (that only contained the xfs SR driver) and include all SR drivers by default, including experimental ones. However, we do not install all the dependencies on dom0 by default: xfsprogs, gluster-server, ceph-common, zfs... They need to be installed using yum for you to use the related SR drivers.

zfs

We already provided zfs packages in our repositories, but there was no dedicated SR driver. Users would use the file driver, which has a major drawback: if the zpool is not active, that driver may believe that the SR suddenly became empty, and forget all VDI metadata.

So we developed a dedicated zfs SR driver that checks whether zfs is present before drawing such conclusions.

• Required dependency: zfs
• Documentation: https://xcp-ng.org/docs/storage.html#zfs

glusterfs

Use this driver to connect to an existing Gluster storage as a shared SR.

• Required dependency: glusterfs-server
• Documentation: https://xcp-ng.org/docs/storage.html#glusterfs

cephfs

Use this driver to connect to an existing Ceph storage through the CephFS storage interface.

• Required dependency: ceph-common (not provided in XCP-ng repositories. Check the docs.)
• Documentation: https://xcp-ng.org/docs/storage.html#cephfs

xfs

Use this driver to create and use an XFS SR.

• Required dependency: xfsprogs
• Documentation : to be written. There's nothing special about it, it works exactly like the ext driver, but uses xfs instead of ext4 as a filesystem.

Core Scheduling

The updated Xen in XCP-ng 8.2 brings support for Core Scheduling.

As you probably know, Hyper Threading defeats all mitigations of CPU vulnerabilities related to side-channel attacks. That's why it was required to disable it as part of the mitigations. The reason is that with Hyper Threading enabled you can't protect a VM's vCPUs from attacks originating from other VMs that have workloads scheduled on the same physical core.

With Core Scheduling, you now have another solution: you can choose to leave Hyper Threading enabled and ask the scheduler to always group vCPUs of a given VM together on the same physical core(s). This will remove the vulnerability to a class of attacks from other VMs, but will leave the VM processes vulnerables to attacks from malevolent processes from within itself. To be usedonly with entirely trusted workloads.

How to enable it:

• The latest version of Xen Orchestra, released at the same time as XCP-ng 8.2 RC, now offers a way to change the scheduling mode for a host.
• You can also enable it manually by adding sched-gran=core to Xen's command line parameters in grub.cfg and reboot. See Xen's documentation.

gpumon's mock build (tests with nVIDIA GPUs required)

gpumon is an opensource component that can't be built without the appropriate proprietary nvidia toolkit. It is supposed to be only used for nvidia vgpus, which we don't support (proprietary stuff too). We already tried to remove it from XCP-ng in 8.1, however you may remember than we discovered, after the release of XCP-ng 8.1, that the XAPI would not manage to start without it on systems that have nVIDIA GPUs.

This time, we chose another approach: we built a mock version of gpumon (working, but giving fake statistics about the GPUs), so the binary is present on XCP-ng and the XAPI is happy enough with it to start in presence of an nVIDIA GPU.

However, there still is a risk that the XAPI would try to use gpumon later, not only during the startup process, so we need users with nVIDIA GPUs to install XCP-ng 8.2 and test it in various situations.

How to upgrade from previous releases

Since XCP-ng 8.2.0 is a minor release, both upgrade methods are supported:

• From the installation ISO
• From command line using yum (from XCP-ng 8.0, 8.1 or 8.2 beta only!)

Refer to the upgrade howto.

Downloads:

• Standard ISO: http://mirrors.xcp-ng.org/isos/8.2/xcp-ng-8.2.0-RC1.iso
• Net-install ISO: http://mirrors.xcp-ng.org/isos/8.2/xcp-ng-8.2.0-RC1-netinstall.iso
• SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.2/SHA256SUMS
• Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.2/SHA256SUMS.asc

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's should always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature. See https://xcp-ng.org/docs/mirrors.html#security.

After the installation, stay updated

Run yum update regularly. We'll fix bugs that are detected regularly until the final release. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you want them): we'll announce every important update to the RC here.

What to test

Everything and anything! A community effort to list things to be tested has been started at https://xcp-ng.org/docs/develprocess.html#tests.

In addition, we ask you to give special attention to:

• UEFI VM support
• Logging (rsyslog update)
• The new storage drivers
• Hosts with nVIDIA GPUs

Report or ask anything as replies to this thread.

Feedback is always useful, both about successes and failures.

This thread is now read-only. Testing feedback now happens on the thread dedicated to the 8.1 RC release.

Hello to everyone.

The beta release of XCP-ng 8.1 is available right now .

What changed

Non-exhaustive list:

• Based on Citrix Hypervisor 8.1 (https://docs.citrix.com/en-us/citrix-hypervisor/whats-new.html).
• Same base version of CentOS (7.5), same base version of kernel (4.19, with more patches of course)
• Xen 4.13
• UEFI support for guests is not experimental anymore
• Support for Secure Boot is NOT INCLUDED because it relies on Proprietary packages. We're raising the issue with the Xen community to see how to provide a FOSS implementation.
• Citrix announces:
  • "Improved performance for VM imports and exports that use the XVA format" thanks to the use of a very efficient hash algorithm
  • "Storage performance improvements"
  • "New Windows I/O drivers with improved performance"
  • (From https://bugs.xenserver.org/browse/XSO-951) The next update (apparently not published yet as of 2020-01-27) of the Windows Drivers through Windows Update should at last be compatible with non-english locales!
• chrony replaces ntp
• Support for PV guests is now deprecated
  • templates for creating PV guests have been removed
  • existing guests will still run
  • it is advised to convert them to HVM guests
  • a compatibility layer should be provided in the future for PV guests that really can't be converted. But really anyone who can convert to HVM, should
  • due to how 32-bit PV guests work, keeping it functioning on newer hardware with newer features comes with an increasing performance cost, and the linux kernel is about to drop support for 32-bit PV guests
• Support for AMD EPYC 7xx2(P) added
• Dynamic Memory Control (DMC) is deprecated and will be removed in the future.
• VSS and quiesced snapshots support is removed, because it never worked correctly and caused more harm than good.
  • For backups, we are working on support for backups that include the guest RAM to replace the need for quiesced snapshots.
  • Note that Windows guest tools version 9 (the default for recent versions of Windows if you install through Windows Update) already removed VSS support, even for older versions of CH / XCP-ng
• For new local storage repositories using the EXT filesystem, it now defaults to ext4.
  • The transition from our experimental ext4 storage driver is not automatic so you still need the sm-additional-drivers package if you have such a local storage repository.
  • We will try to provide an automated conversion from the ext4 experimental driver to the ext driver for the RC (release candidate) release of XCP-ng. If we can't, we'll document the transition.
• Status of our specific packages:
  • Support for XFS in local storage repository still available through the sm-additional-drivers package.
  • ZFS still available as an additional package and updated to 0.8.2. We may update to 0.8.3 for the RC release of XCP-ng.
  • Alternate kernel: none available yet for 8.1, work in progress
  • netdata RPMs are not ready yet: waiting for Xen 4.13 support added by @r1, needs testing: https://github.com/netdata/netdata/issues/7803

2020-02-20 update

Announcement about our former experimental ext4 SR driver:

It is now deprecated in 8.1. For a good reason: in XCP-ng 8.1 and above, following upstream changes, the ext driver now formats new SRs as EXT4.

There is no easy way to convert an existing SR created with our driver, so those using it will need to move the VDIs out (to another SR or to export them), destroy the SR and create an EXT SR instead. Make sure to do this on XCP-ng 8.1.

The sm-additional-drivers package remains available in XCP-ng 8.1 in order to ease the transition. However I've broken the sr-create command on purpose. Any attempt to create a SR of type ext4 will result in an error with a message that explains that you need to use the ext type instead.

Our experimental driver will be completely removed in a later release, possibly XCP-ng 8.2. Unless someone convinces me to delay the removal for a good reason. I will accept reasons such as "I know I shouldn't have used the experimental driver in production, but I did and need more time to convert my SR while at the same time I really need feature xxx from XCP-ng 8.2", but I really would prefer to drop it in 8.2.

Feedback from people doing the transition is welcome to make sure we document the transition in the best way possible.

Documentation

At this stage you should already be aware that our main documentation is in our wiki and you should also know that you can all take part in completing it. It's improving continuously and still has room for improvement.

How to upgrade

Since XCP-ng 8.1.0 is a minor release, both upgrade methods are supported:

• From the installation ISO
• From command line using yum

Refer to the upgrade howto: https://github.com/xcp-ng/xcp/wiki/Upgrade-Howto

Downloads:

• Standard ISO: http://mirrors.xcp-ng.org/isos/8.1/xcp-ng-8.1.0-beta.iso
• Net-install ISO: http://mirrors.xcp-ng.org/isos/8.1/xcp-ng-8.1.0-beta-netinstall.iso
• SHA256SUMS: http://mirrors.xcp-ng.org/isos/8.1/SHA256SUMS
• Signatures of the sums: http://mirrors.xcp-ng.org/isos/8.1/SHA256SUMS.asc

If your browser tells you that the page is redirected in a way that prevents from loading the page correctly, this is a know issue. Long story short, it's caused by a wrong security header sent by the forum software, which impacts all non HTTPS requests to the xcp-ng.org domain. Try in Private Navigation mode or use curl or wget.

It's a good habit to check your downloaded ISO against the SHA256 sum and for better security also check the signature of those sums. Although our mirror redirector does try to detect file changes on mirrors, it's should always be envisioned that a mirror (or in the worst case our source mirror) gets hacked and managed to provide both fake ISOs and SHA256 sums. But they can't fake the signature. See https://github.com/xcp-ng/xcp/wiki/How-to-check-the-authenticity-of-files-downloaded-from-XCP-mirrors.

Stay up to date

Run yum update regularly. We'll fix bugs that are detected regularly until the final release. Subscribe to this thread (and make sure to activate mail notifications in the forum parameters if you need them): we'll announce every important update to the RC here.

What to test

Everything and Anything!

Report or ask anything as replies to this thread.

A community effort to list things to be tested has been started at https://github.com/xcp-ng/xcp/wiki/Test-XCP

Feedback is welcome both about successes and failures.

It's here, finally.

I will not re-detail everything that this release brings, because everything is available here:

• 8.0.0 Release Candidate announcement
• 8.0.0 Official Release announcement

A huge thank you to all of you who have been involved in testing and more.

For you, testers, some information that is not on the blog:

• If you installed XCP-ng 8.0.0 prefinal ISOs from the RC thread, then you actually already have XCP-ng 8.0.0 final! Nothing to do.
• If you installed XCP-ng 8.0.0 beta or RC, just update your hosts like you would do for any regular update and you'll have the same system as if you had installed from the final ISO. Just avoid doing it from the shell console that is started from xsconsole (see why in the updates howto).

Coming. Very. Soon.

Problem solved on chat. Our last update doesn't like it at all when a certificate was manually removed from the system without letting XAPI know.

So we had to:

• revert the update and restart the toolstack - XAPI works again
• touch /etc/stunnel/certs/sdn-controller-ca.pem to create back a dummy file for the missing cert, so that xe will accept to remove it from database (it fails if it can't find the file for which you're trying to remove the database entry )
• xe pool-certificate-uninstall name=sdn-controller-ca.pem
• update again
• perform the update of the rest of the pool hosts

I added a comment in the issue related to the patches we included in this update to fix certificate fingerprints: https://github.com/xapi-project/xen-api/issues/5955

I just published the last big update before the release of the XCP-ng 8.3 Release Candidate. The last pre-release before the final release.

Now is the best time to test thoroughly.

The changes contain both the result of the work of XCP-ng developers, and a rebase on the latest updates of the XenServer 8 release when appropriate.

Main highlights:

• Rebased on XenServer 8 + updates published since the release
• We enriched the API in order to let Xen Orchestra give you feedback about the status of Secure Boot in your UEFI VMs and updated the Secure Boot documentation
  • Xen Orchestra feature 1: Warn before UEFI VM creation if Secure Boot is on but the pool is not setup for SB
  • Xen Orchestra feature 2: Display accurate Secure Boot status and allow to fix a VM's UEFI certs
  • Both will be included in the next Xen Orchestra release, which is currently planned for the end of this week.
• Security updates: curl, Intel and AMD microcode, OpenSSL, openvswitch, sudo, xen
• Installer: support for upgrading from 8.3 alpha/betas is back.
• More python tools ported to python3.
• Alternate "debugging" kernel updated to version 4.19.316.
• Optional netdata package updated to versionn 1.44.3.
• New optional driver package mlx4-modules-alt which provides the LTS 4.9 version of the MLX4 drivers, used by older ConnectX cards.
• XO Lite updated to version 0.2.3:
  • [Pool/Dashboard] In the Storage usage card, DVDs are no longer taken into account
  • [Header] Add link to XOA when XOA is detected on the pool
  • Add german translation (based on the contribution made by @borzel.

Packages details (detailed changelog available by following the links)

• amd-microcode
• blktap
• Various drivers updated:
  • cisco-enic-4.5.0.7-1.xcpng8.3
  • cisco-fnic-2.0.0.90-1.1.xcpng8.3
  • intel-igc-5.10.214-3.1.xcpng8.3
  • microsemi-smartpqi-2.1.28_025-1.xcpng8.3
  • mlx4-modules-alt-4.9_7.1.0.0-1.xcpng8.3
  • vendor-drivers-2.0.3-1.1.xcpng8.3
• curl-8.6.0-2.1.xcpng8.3
• edk2-20220801-1.7.5.1.xcpng8.3
• gdisk-1.0.10-1.xcpng8.3
• gpumon-24.0.0-4.1.xcpng8.3
• guest-templates-json-2.0.10-1.1.xcpng8.3
• host-installer-10.10.16.xcpng.2-2.xcpng8.3
• intel-microcode
• interface-rename-2.0.5-1.xcpng8.3
• kernel-4.19.19-8.0.34.2.xcpng8.3
• kernel-alt-4.19.316+1-1.xcpng8.3
• kexec-tools-2.0.15-20.xcpng8.3
• logrotate-3.8.6-21.xcpng8.3
• lvm2-2.02.180-16.1.xcpng8.3
• ncurses-6.4-3.xcpng8.3
• net-snmp-5.7.2-51.1.xcpng8.3
• netdata-1.44.3-1.1.xcpng8.3
• nspr-4.35.0-1.el7_9
• nss-3.90.0-2.el7_9
• nss-softokn-3.90.0-6.el7_9
• nss-util-3.90.0-1.el7_9
• openssl-1.0.2k-26.1.xcpng8.3
• openvswitch-2.17.7-1.3.xcpng8.3
• python-pam-1.8.4-1.xcpng8.3
• qemu-4.2.1-5.2.9.xcpng8.3
• setup-2.8.71-9.1.xcpng8.3
• sm (SMAPIv1 storage manager)
• sudo-1.9.15-3.1.xcpng8.3
• swtpm-0.7.3-8.xcpng8.3
• xapi-24.16.0-1.1.xcpng8.3
• xcp-featured-1.1.7-2.xcpng8.3
• xcp-ng-deps-8.3-8
• xcp-ng-plymouth-theme-1.1.0-2.xcpng8.3
• xcp-ng-release-8.3.0-24
• xcp-python-libs-3.0.4-1.1.xcpng8.3
• xcp-python-libs-compat-2.3.5-6.xcpng8.3
• xen-4.17.4-3.xcpng8.3
• xenserver-hwdata-20240411-1.xcpng8.3
• xo-lite-0.2.3-1.xcpng8.3
• xs-openssl-1.1.1k-12.1.xcpng8.3
• xsconsole-11.0.2-1.1.xcpng8.3

Reboot after update

And XCP-ng 8.3 Beta 2 finally released!

https://xcp-ng.org/blog/2024/02/15/xcp-ng-8-3-beta-2/

Fixes are on the way for the stats issue. It involves both XCP-ng and Xen-Orchestra, so this will take a few more days.

New batch of updates for XCP-ng 8.3 beta

So, as several already found out, I pushed many updated packages to XCP-ng 8.3's RPM repositories at the same time. This is the result of the last six months of work on XCP-ng 8.3.

Changes coming from XenServer

This update includes everything published by XenServer between the release of XCP-ng 8.3 Beta1 and October 2023. Newer changes will be brought in after Beta2.

I am a bit late and sadly don't have time now to detail all their changes, but the most important one is maybe completion of vTPM support (and thus compatibility with Windows 11). Previously, it would work, but many features were still missing. Now, snapshots, VM export/import and live migration all are supported for UEFI VMs with a vTPM. A Windows 11 template is available, which will transparently create the required vTPM when you create a new VM.

Changes coming from XCP-ng

On XCP-ng's side, in no particular order:

• We rebased all packages on XenServer 8 preview + all updates up to early October.
• Installer
  • rebased on latest upstream release.
  • better granularity in error messages displayed to users when install fails due to wrong system date, signature issues, etc.
  • re-allow interactive installation of driver disks on host during installation.
  • Avoid failed services in the installer (there were benign "failed" messages for services the installer doesn't actually use)
• IPv6 support continued
  • IPv6 testing / automated tests. Not run automatically on our internal CI yet due to setup difficulties.
  • Fixed live migration failure due to upstream bug in ocaml-uri. Fix contributed upstream to ocaml-uri.
  • Allow to use a CIDR for VIFs IPv4 and IPv6 allowed IPs. Feature contributed upstream to XAPI project.
  • xsconsole : allow to configure IPv6 via autoconf for the management interface. Feature contributed upstream to XAPI project.
• Making XOSTOR available in XCP-ng 8.3. Another update is pending, so better wait for it.
• Installer image generation: fixes and improvements.
• VLAN display in xsconsole. Feature contributed upstream to XAPI project.
• smartmontools updated to version 7.
• Plugin added to use the data from smartmontools 7 in JSON format (initially contributed by Cécile, then improved by one of our developers)
• Debian 12 template added.
• Security fixes. However, no security fix was made to XCP-ng 8.3 in the last weeks, while we were busy testing and stabilizing.
• Added new tests to our test suite. For example this new test which exercises the vTPM features. Windows 11 is now also tested in our CI/CD.
• Deleted the old, unsupported since XCP-ng 8.1, experimental EXT4 driver. We're talking about an old experimental driver that you never used unless you installed the experimental packages in the XCP-ng 7.x era.
• As in 8.2.1, lsscsi added to our repositories.
• As in 8.2.1, more alternate drivers packaged.
• As in 8.2.1, drivers updated and added to default installation: mpi3mr, igc, r8125 (+firmware)
• xo-lite RPM added and installed by default.
• UEFI and SecureBoot support:
  • The version of XAPI included in XCP-ng 8.3 after Beta1 provides a new way to handle UEFI certificates at the pool level, implemented by XenServer developers while trying to take both their and our needs into account.
  • We then adapted the automated tests for XCP-ng 8.3, based on new XAPI behaviour.
  • Running the adapted tests revealed issues (one bug, but also more importantly that we did not fully understand each other regarding XCP-ng's needs). So we proposed a new design to the XAPI project, then implemented it. This hopefully concludes more than a year of intermittent work on this very topic.
  • From a user standpoint, despite all the work behind the scene, there aren't many changes. Mostly retaining the ability we had in XCP-ng 8.2.1 for users to download and install the default certificates from Microsoft (must run secureboot-certs install at least once on the pool after this update), or install their own.
• Various diagnoses and fixes.

Known issues

Several users reported issues with UEFI support. UEFI, again! Just when we thought we were done with it. According to what we found out together, the issue is not related to the changes I was mentioning above, which are limited to certificate management. Here, it's apparently a piece of software emulating a UEFI firmware for VMs, edk2, which regressed in some situations.

So, the current situation, as I know it today:

• For various users, it works, but it looks like pfSense (and maybe FreeBSD) in a UEFI VM has trouble starting.
• In our internal tests, UEFI works for all the OSes we tested. But currently FreeBSD is only tested in BIOS mode, so we wouldn't have seen it. We'll improve our test coverage.
• For one user, it manifested in a way that no UEFI VM would ever start, whatever the OS.

For now, the workaround is to downgrade the component which seems faulty, on every host: yum downgrade edk2-20180522git4b8552d-1.5.1.xcpng8.3. Note that Xen Orchestra, or yum update, will attempt to reinstall the more recent version of the package, so better avoid updating again until we've solved these issues.

Updated installer (XCP-ng 8.3 Beta2)

As soon as any remaining blockers are lifted, this batch of updates will also be released in a XCP-ng 8.3 Beta2 installation ISO image.

You may download a test build of the installation ISOs: https://mirrors.xcp-ng.org/tmp/xcp-ng-8.3.0-beta2-test2.iso

SHA256SUM: 11ed1ca5d757177e0edab6c7a7a371306e3ddcdfab7df62b6f076e527a7a5015

/!\ The netinstall repository won't be updated until Beta2 is released, so any network installation using the main mirrors will actually install XCP-ng Beta1, not Beta2.

A new batch of updates for XCP-ng 8.3 alpha arrived.

It's been a while: one specific update I wanted to push together with everything that was ready gave us more trouble than expected, but now it's solved and so I pushed all the updates at once.

Main changes

• xcp-ng-release*: updated repository definitions in /etc/yum.repos.d/xcpng.repo, to add more testing repositories (disabled by default. More about this here - look for "About the new testing repositories"). Warning: If you have any local changes to this file, it won't be overwritten. In this case, look for /etc/yum.repos.d/xcpng.repo.rpmnew after applying the update, and move it over xcpng.repo. This won't change much for you as testers of XCP-ng 8.3, because we're still pushing the updates directly to xcp-ng-base when they are ready for your testing, but there might be the occasional experimental package in some other location - and the updated .repo file with make it easier to pull from there if needed.
• AMD and Intel microcode updates (not the very latest yet, though)
• sm (Storage Manager) ported to python 3 by XenServer, and we rebased our additional storage drivers on it
• XAPI and related components
  • HTTP to HTTPS redirection on the host's web page.
  • lots of changes from upstream. Full changelog.
• IPv6 support added to xha and xsconsole. You'll need to wait for an updated installer in order to test this and the rest of IPv6 support for the management interface.
• memtest86+ updated to 6.20 to support EFI hosts and more hardware. Will be available in the installer.
• xen: better hardware support (Ryzen...), security fixes, bug fixes...
• Other updates synced from the preview release of XenServer 8.3.
• zstd 1.5.4 (1.5.5 soon)
• removed corosync and dlm from installed systems. They were unneeded dependencies for XenServer's clustering support, which is proprietary and thus unsupported on XCP-ng. Let's trim dom0 down slightly.

We also continued our work on the installer and on automating the build of installer ISO images. We'll have new builds for you shortly.

How to update

From an installed XCP-ng 8.3 alpha, just run yum update, and reboot. The usual update rules apply: master host first, etc. Check https://xcp-ng.org/docs/updates.html. I haven't tested it on this update, but XOA's Rolling Pool Update should work on this update.

What to test

Depending on the time available, it goes from "just update and check things still work as expected" to "test everything you can think of", with an intermediate step of "test specific features which are important to you".