-
@gskger I think we might have the same. Do you have an error log?
-
@olivierlambert I haven't looked for error logs (yet), but if you tell me which logs are interesting or what to look for, I'll be happy to give it a try.
-
@gskger mainly xensource.log, possibly daemon.log.
-
I thought about XO Logs (in Settings/Logs).
-
@olivierlambert @stormi I will send you what I have tomorrow if it is not that urgent, ok?
-
No rush at all Thanks a lot already for giving us that valuable feedback.
-
@olivierlambert Install patches from from XO source (current master) and 8.2.1. It works, but I an error.
Host install:
pool.installPatches { "hosts": [ "64b9bf4a-a991-438d-973f-60179f3d0868" ] } { "message": "", "name": "Error", "stack": "Error: at Xapi._xcpUpdate (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xapi/mixins/patching.mjs:323:15) at Object.installPatches (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/api/pool.mjs:115:3) at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xo-mixins/api.mjs:307:20)" }
Pool install:
pool.installPatches { "pool": "98ab0129-f9d1-ba08-c1c0-aa93c38d7fec" } { "message": "", "name": "Error", "stack": "Error: at Xapi._xcpUpdate (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xapi/mixins/patching.mjs:323:15) at runMicrotasks (<anonymous>) at runNextTicks (node:internal/process/task_queues:61:5) at processImmediate (node:internal/timers:437:9) at process.topLevelDomainCallback (node:domain:152:15) at process.callbackTrampoline (node:internal/async_hooks:128:24) at Object.installPatches (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/api/pool.mjs:115:3) at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xo-mixins/api.mjs:307:20)" }
-
@olivierlambert @stormi I started the rolling update from XO and left it running alone since it usually takes some time to complete. This
xenopsd internal error
is from the XO log.2022-02-28T15_21_32.891Z - XO.log.txt
An hour later I realized that the updates had been applied, but the hosts did not reboot. I tried to reboot the master from XO first which might have resulted in this XO log.
2022-02-28T16_35_04.612Z - XO.log.txt
After that all VMs including XO became unresponsive and I continued to investigate with XCP-ng Center. In the end, I decided to reboot the hosts from CLI (master first and than all slaves). That resolved the issue and XO came up without any further problems.
I have copies of the logs @stormi suggested from the master and one slave if they are needed. XO is XO from source by 3rd party script. I did an XO update after resolving the issue, so if you need the XO version, I can restore from backup and check.
Hope that helps.
-
-
@olivierlambert Yes, restarting the toolstack first would have been a better approach. Is this something that needs additional actions on the hosts?
-
No, that should be enough
-
We added a warning on the blog post https://xcp-ng.org/blog/2022/02/28/xcp-ng-8-2-1-update/ and will work on a fix in XO.
-
@stormi
now realise I had the same behaviour after applying the latest 8.2.0 security updates a few weeks ago.
At that time thought it was because I patched the master manually already (test repo release) and updated the entire pool only after the official release 2 days later (roling pool update).So it is not (only) related to the 8.2.1 release
-
@HeMaN I think it's more likely to happen with the 8.2.1 update because more components are updated. Else I agree.
-
New security updates (xen, Intel microcode)
Impact: newly disclosed security vulnerabilities may allow attackers from within a guest to access memory of the host or other VMs. Only AMD CPUs are affected. To know if your specific model is affected, consult the vendor publications. Xen has been patched to mitigate the hardware issue.
Additionnally but unrelated to the security issue, firmware is updated for some models.
Citrix' security bulletin: https://support.citrix.com/article/CTX341586
Test on XCP-ng 8.2
From an up to date host:
yum clean metadata --enablerepo=xcp-ng-testing yum update linux-firmware microcode_ctl xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing reboot
Versions:
- xen-*: 4.13.4-9.20.1.xcpng8.2
- microcode_ctl: 2.1-26.xs20.xcpng8.2
- linux-firmware-20190314-3.xcpng8.2
What to test
Normal use and anything else you want test. The closer to your actual use of XCP-ng, the better.
Test window before official release of the updates
~2 days. Maybe less.
-
@stormi Manual updates done on all my 8.2.1 machines and things are working correctly so far.
-
Same thing on my EPYC machine.
-
Thanks!
I see there's also another XSA available; XSA-396 (https://xenbits.xen.org/xsa/ )
Will these be combined with the microcode patches ?Cheers!
-
@NielsH XSA-396 does not affect XCP-ng as the device back-ends are in dom0 directly, not in driver domains.
-
Security update released. Thanks @Andrew for testing!
https://xcp-ng.org/blog/2022/03/11/march-2022-security-update/