XCP-ng 8.2 updates announcements and testing

stormi

Many thanks to all testers. The updates have been published on Friday, along with the blog post: https://xcp-ng.org/blog/2021/02/26/february-2021-security-updates/

jmccoy555

Updating my pool now.... quite a substantial speed increase on FreeBSD

root@FILE001:~ # iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 10.10.1.125 port 5001 connected with 10.10.1.126 port 42952
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.1 sec   694 MBytes   576 Mbits/sec
^Croot@FILE001:~ # iperf -c 10.10.1.126
------------------------------------------------------------
Client connecting to 10.10.1.126, TCP port 5001
TCP window size:  105 KByte (default)
------------------------------------------------------------
[  3] local 10.10.1.125 port 50605 connected with 10.10.1.126 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   202 MBytes   169 Mbits/sec
root@FILE001:~ # iperf -c 10.10.1.126
------------------------------------------------------------
Client connecting to 10.10.1.126, TCP port 5001
TCP window size: 80.8 KByte (default)
------------------------------------------------------------
[  3] local 10.10.1.125 port 45072 connected with 10.10.1.126 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  4.06 GBytes  3.49 Gbits/sec
root@FILE001:~ # iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 10.10.1.125 port 5001 connected with 10.10.1.126 port 47101
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  2.17 GBytes  1.86 Gbits/sec
^Croot@FILE001:~ #

stormi

It's testing time again!

We changed the way we build the guest tools ISO and so we need as much feedback as we can get regarding the new guest tools. The main goal is to look for regressions. Details and feedback on https://xcp-ng.org/forum/topic/4378/new-guest-tools-iso-for-linux-and-freebsd-can-you-help-with-the-tests.

stormi

New security updates to test for 8.1 and 8.2

New update candidates are available for testing and due to be released as official updates very soon, as is usually the case for security updates.

kernel security and bugfix update, prevents DoS attacks from the guests and brings fixes to event handling.
openvswitch security update, prevents malicious network traffic to cause packets to be dropped.
Fixed ixgbe driver to avoid the memory leaks discussed at https://xcp-ng.org/forum/topic/2507/alert-control-domain-memory-usage

Test on XCP-ng 8.2

yum clean metadata --enablerepo=xcp-ng-testing
yum update kernel openvswitch openvswitch-ipsec intel-ixgbe --enablerepo=xcp-ng-testing
reboot

Note: this won't update openvswitch-ipsec if not present on the system, that is, unless you're using XO's private networks.

Test on XCP-ng 8.1

This will be the last security update for XCP-ng 8.1, whose end of life is officially today, March 31st.

yum clean metadata --enablerepo=xcp-ng-testing
yum update kernel openvswitch openvswitch-ipsec intel-ixgbe --enablerepo=xcp-ng-testing
reboot

What to test

The main goal is to avoid obvious regressions, so test whatever you want. The closer to your actual use of XCP-ng, the better.

Note: as the the code for event handling was changed, pay special attention to network performance in VMs, especially FreeBSD VMs since they are outside the scope of Citrix testing.

Test window before official release of the updates

Between 24h and 48h.

gskger

@stormi Successfully updated my two host playlab (8.2.0 full patched, but with the new guest tools ISO). Rebooted both hosts and ran my usual test program (create, live migrate (without (only Debian) and with guest-tools installed (Debian 10 and Windows 10) avoiding the VM_LACKS_FEATURE error on Windows 10 - more sleep this time ), copy and delete as well as create / revert to snapshot (with/without ram) and (live/halted=shutdown) storage migration remote to local SR and back). Also restored a Debian 10 VM from a pre-update backup with no problem. Looks good . Will see how backup runs tonight, but I am confident that this will work as well.

Edit #1: typos

Edit #2: Thanks to Xen Orchestra, out-of-band management even for Optiplex 9010s, a notebook and a very nice evening on the balcony, I extended my tests to Ubuntu 20 LTS. And you might guess it - works as well

Edit #3: Deleted a subsequent post on an asyncMap is not defined error because it is not related to the security update.

gskger

This post is deleted!

tzerod

Tested new security patches on 8.2.

Network performance seems just fine. Have 4 Windows server 2019 and 2 Fedora Server VMs recording video feed. Haven't observed any difference in performance.

Supermicro SYS-1029U-TR4 with 768gig ram.
Network usage: about 400mbps constant.
ZFS

jmccoy555

Only updated one host in my pool so far, but it has my TrueNAS VM and network performance looks comparable

edit:

Actually quite an improvement in comparison to my results above

root@FILE001:~ # iperf -c 10.10.1.126
------------------------------------------------------------
Client connecting to 10.10.1.126, TCP port 5001
TCP window size: 80.8 KByte (default)
------------------------------------------------------------
[  3] local 10.10.1.125 port 35576 connected with 10.10.1.126 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.1 sec  6.30 GBytes  5.36 Gbits/sec
root@FILE001:~ # iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 10.10.1.125 port 5001 connected with 10.10.1.126 port 54773
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  3.57 GBytes  3.06 Gbits/sec

Also done a VM reboot now and no passthrough issues

stormi

Just in time

Updates now published: https://xcp-ng.org/blog/2021/04/01/april-2021-1st-security-bugfixe-update/

jmccoy555

@stormi Security updates on April 1st

stormi

@jmccoy555 Yeah. Install them today and they'll turn your hosts into VMware.

jmccoy555

@stormi Isn't that a virus, not a bug fix????

olivierlambert

BenjiReis

New xsconsole fix to test for 8.2

New update candidate are available for testing and due to be released as official updates.

Original topic:

https://xcp-ng.org/forum/topic/3886/xcp-ng-8-2-dns-settings-gone-after-reboot

https://xcp-ng.org/forum/topic/3277/dns-config-on-8-1-hosts-disappearing

yum clean metadata --enablerepo=xcp-ng-testing
yum update xsconsole --enablerepo=xcp-ng-testing
systemctl restart xsconsole.service

What to test

Changing the DNS settings from the XSConsole and the change is retain after a reboot.

stormi

@benjireis said in Updates announcements and testing:

What to test

Changing the DNS settings from the XSConsole and the change is retain after a reboot.

And, of course, as usual, that you don't notice any obvious regression in XSConsole.

gskger

@stormi Did not even know the problem existed . Anyway, added a new (second) DNS server (9.9.9.9) to the DNS server list via xsconsole and rebooted the host (XCP-ng 8.2.0 fully patched).

Before update: DNS 9.9.9.9 did not persist, only the previous settings are shown
After update: DNS 9.9.9.9 did persist the reboot and is listed together with the previous settings

Deleting DNS 9.9.9.9 worked as well, so the xsconsole update worked for me.

olivierlambert

IIRC, it's on old problem reported a long time ago to Citrix. But they never fixed it.

edit: and thanks again @gskger for your tests, it matters a lot!

gskger

@olivierlambert as always, it is a pleasure to help and also easy to do and to contribute. Oh and @BenjiReis - sorry for not replying to your initial post.

BenjiReis

@gskger Thanks for the report!

BenjiReis

Experimental feature: select a network to evacuate an host
A new feature is available in our testing repo: select a network for host evacuation, this would allow to evacuate an host on any (faster) given network instead of the management one.

To access the feature, on all your hosts (always starting with master when in a pool):
yum update --enablerepo=xcp-ng-testing xapi-core-1.249.5-1.1.0.evacnet.1.xcpng8.2.x86_64 xapi-xe-1.249.5-1.1.0.evacnet.1.xcpng8.2.x86_64 xapi-tests-1.249.5-1.1.0.evacnet.1.xcpng8.2.x86_64
And restart your hosts.

WHAT TO TEST
Host evacuation not on the management network (probably a 10G storage network to go faster!)
You can run xe host-evacuate host=<host_uuid> network-uuid=<network_uuid>
Or a XAPI client can call host.evacuate with a network ref parameter.
Host evacuation without the optionnal new parameter should behave as before the update.

Please report here if anything goes wrong (or right hopefully ) and if you spot a regression as well.

Thanks!

Edit: there are no plans for now to add this feature in 8.2 LTS, the package will probably stay in the testing repo for 8.2 and will be available in 8.3. It means that the package would be erased at next xapi update.