-
@stormi what is the best strategy to revert a host with testing updates installed back to the standard, non-testing status? Just wondering, if there is a more simple approach than doing a clean install / revert to backup.
-
@gskger the
yum historycommand can be handy to rollback to earlier versions of RPMs that are still available on repositories.yum downgrade same-list-of-RPMs-that-you-updatedcan also work most of the time.Be aware though that RPM transactions are not always meant to be reversible. Replacing files is one thing, but the scriptlets that run after an update are almost never tested backwards. I don't foresee any specific issue, it's a general warning.
-
@stormi
Did some Testing over the Weekend too.
Setup with 2 Hosts in a Pool and shared iSCSI-LMV Storage with multipath 8 paths per LUN.
Anything seems to work fine (migrate/import/cross-pool-migrate/snapshots/backups).Even our longtime Problem (snapshots taking much too long) is getting much better (still not good, but much better).
-
The update was released yesterday: https://xcp-ng.org/blog/2021/06/28/summer-security-and-bugfix-updates/
Again, a lot of thanks for the feedback.
As I said earlier, the update for
uefistored, which brings guest secure boot support, was not included yet. -
New installation ISO for XCP-ng 8.2
I opened a dedicated thread. Meet you there for the tests.
-
A bugfix kernel update available for testing
Based on Citrix's hotfix XS82E030, here's a bugfix kernel update. I don't think it will change much for most hosts, except in some specific cases.
What changed
- Previous kernel updates (that fixed network performance issues for FreeBSD and sometimes other VMs), may have reduced the performance in some situation according to Citrix. Based on the patches, it looks like it's related to IRQ affinity and cross-domain networking. Here's the patch: https://github.com/xcp-ng-rpms/kernel/blob/master/SOURCES/0001-xen-events-fix-setting-irq-affinity.patch
- Tools that need to make the
iopermsyscall were crashing on dom0. For example Supermicro Update Manager (SUM). This should fix it. - An additional dependency was added to the
perfRPM (not installed by default) to make it able to do backtraces when you try to run it on binaries in dom0. - A patch fixes CVE-2021-29154 was added. It's not considered a security update because it does not fix an exploitable vulnerability. It's extra defence in depth.
How to update (XCP-ng 8.2 only)
yum update kernel --enablerepo=xcp-ng-testingVersion that should be installed:
4.19.19-7.0.12.1.xcpng8.2What to test
Installation of the update, normal use, no obvious regressions...
Plus the changes described above if you're in a situation that allows it.
Test window before release
None defined at the moment. As it's not a security update, I'll wait for more updates to be ready before I push the next train officially. But feedback is always useful as soon as it can be provided.
-
@stormi A lazy sunday morning, some coffee, a notebook and time for testing. Updated my two host playlab (DELL Optiplex 9010, 8.2 fully patched) and tested Debian VMs and Windows 10 VMs (create, live migrate with/-out guest tools, start/stop/reboot, snapshot with/-out RAM and revert, online-/offline storage migrate from/to shared and local SR). Restored a Windows 10 and a Debian VM from backup. All good so far for this regression test.
-
Hello, @stormi I did a quick test because we encounter in some cases "bad' network perfs on our pools (with Intel 10GB network cards) such as :
- VXLAN network are getting max 2.5Gbps of bandwidth
- VM to VM network with a BSD firewall in the middle getting max 2.5Gbps of bandwidth
But VM to VM (on same LAN) are getting near 10Gbps.
Testing this new kernel does not change anything on BP for theses cases.
BP still stuck at 2,5Gbps. -
@jcastang AFAIK there's still room for improvement regarding network perfs in *BSD VMs, but I doubt the update was meant to address this. If you want to discuss it further, please create a new thread.
-
A bugfix update for USB passthrough
A fix was contributed by @jeremfg to the XAPI project, so that when XAPI calls
usb_reset.pywith the-rswitch it does not fail anymore. This fixes passing through both a PCI device and an USB device to a VM, and could also fix other USB passthrough issues that raised the same error:"usage: usb_reset.py attach [-h] -d DOMID -p PID [-r RESET_ONLY] device\nusb_reset.py attach: error: argument -r: expected one argument\n".Related thread: https://xcp-ng.org/forum/topic/3594/pci-passthrough-usb-passthrough-does-not-work-together
Related issues:
I have built patched XAPI packages for XCP-ng 8.2 that are available for testing.
How to update (XCP-ng 8.2 only)
yum update xapi-core xapi-tests xapi-xe --enablerepo=xcp-ng-testing xe-toolstack-restartYou should get version-release
1.249.9-1.2.xcpng8.2What to test
USB passthrough:
- What used to work should still work
- What did not work, if the error message in the logs is the one quoted above, should now work... Or maybe fail further in cases that haven't been tested yet?
Test window before release
None defined at the moment. As it's not a security update, I'll wait for more updates to be ready before I push the next train officially. But feedback is always useful as soon as it can be provided
-
I created a dedicated thread for the testing of Guest Secure Boot support: https://xcp-ng.org/forum/post/41541
See you there.
And a reminder about the current update candidates that are in testing in the current thread:
... with my interest being geared mainly towards regression testing.
-
New toolstack (XAPI) update candidate for 8.2
Based on Citrix's hotfix XS82E031. Also includes the USB passthrough bugfix mentioned above in this thread.
What changed
Fixes
- USB passthrough fix
- Reduced log verbosity for stunnel, to prevent
xapi-ssl.logfrom growing so fast that it could fill the/var/logpartition entirely in some cases (users from HA-lizard had reported such behaviour and we were waiting for this upstream fix from the XAPI project to be included in a hotfix). - Fix
xe vm-reset-powerstatefor VMs with GPU passthrough that are stuck in paused state. - Other fixes for rare issues (see hotfix description).
Features
The hotfix from Citrix also comes with a few improvements. Quoting (the highlight in bold characters is mine):
- Adds a default boot order for VMs.
- Improved error messaging for when the graphics card is not configured for SRIOV.
- Remove HTTP access to the management network static web page. This web page can now only be accessed through HTTPS.
- Adds additional snapshots of glocktop data to the bugtool output.
How to update (XCP-ng 8.2 only)
yum clean metadata --enablerepo=xcp-ng-testing yum update forkexecd message-switch xapi-core xapi-tests xapi-xe xcp-rrdd xenopsd xenopsd-cli xenopsd-xc --enablerepo=xcp-ng-testing xe-toolstack-restart # or reboot to be 100% sure everything restartedWhat to test
No obvious regressions, and if possible the changes described above.
Test window before release
A few weeks, but the quicker the better, as this kind of message tends to be forgotten over time.
Other update candidates still in testing phase
The kernel bugfix update is still in testing phase. Many thanks to those who already tested it, and I'm still eager for feedback from others: https://xcp-ng.org/forum/post/41241
-
@stormi
Got these applied too fast.
Http does seem disabled
stunnel logging does seem drastically reduced well know more as time goes on.
I do see a boot order section for VM properties and tested this out be booting into a livecd on a VM.
I ran the bugtool but without knowing what was added I can't confirm or deny this one. -
@stormi said in Updates announcements and testing:
Have a installation without any of the fixed bugs, but Installed the updates on my system and did the toolstack restart.
No issues to report this far.
-
There are at least two blocking issues with this update candidate, so we'll retain it until they are fixed:
- HTTP 403 errors on port 443. Easily reproduced: just load XCP-ng's web page over HTTPS, most images and scripts don't load. We debugged it and reported it to the XAPI project: https://github.com/xapi-project/xen-api/issues/4517
- HA Lizard users reported issues connecting to XAPI. I don't know yet whether the fix belongs in XAPI or in HA Lizard itself. Wait and see.
-
New security updates (xen)
Citrix security bulletin: https://support.citrix.com/article/CTX325319
Impact: privileged code in a guest VM may crash or compromise a host.
Test on XCP-ng 8.2
yum clean metadata --enablerepo=xcp-ng-testing yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing rebootVersion for xen packages: 4.13.1-9.12.1.xcpng8.2
What to test
The main goal is to avoid obvious regressions, so test whatever you want. The closer to your actual use of XCP-ng, the better.
Test window before official release of the updates
24h.
-
@stormi Running well for me. I've tested startup, shutdown, and migration of Windows and Linux VMs with no obvious regressions.
-
@stormi No issue with updating my two host playlab. Just run some simple tests with Debian VMs (create, live migrate with/-out 7.20.0-9 guest tools, start/stop/reboot, snapshot with/-out RAM and revert, online-/offline storage migrate from/to shared and local SR, restore from backup). Restored a Windows 10 VM from backup as well and moved it around a bit. Everything works as expected.
-
Any info about support of Windows Server 2022. RTM (final) version has been released ~2 weeks ago.
-
@stormi Also installed on our test-lab and booting, migrating of RockyLinux and Windows VMs works as expected.
