XCP-ng 8.2 updates announcements and testing

stormi

@heman said in Updates announcements and testing:

yum clean metadata --enablerepo=xcp-ng-testing

Looks like there's a small delay during which mirrorbits hasn't noticed the changed metadata files, and thus redirects you to old filenames. I had taken measures against this (I make it rescan the local repository just after it is updated), but there seems to be situations where it doesn't work.

I restarted the service and now it's good.

HeMaN

@stormi

Installation now went well, no issues noted after the reboot.
Will keep an eye on my system and report back if anything unusual is happening

Andrew

@stormi Are these updates going to staging and be part of 8.2.1 or is this a February 8.2.0 security update before 8.2.1 ?

stormi

@andrew There will be a security update before 8.2.1.

stormi

Security update published. Thanks for testing! https://xcp-ng.org/blog/2022/02/11/february-2022-security-update/

stormi

The update that we named XCP-ng 8.2.1 is now published!

https://xcp-ng.org/blog/2022/02/28/xcp-ng-8-2-1-update/

gskger

@stormi Updating my homelab (3x R210II, 2x R720) to 8.2.1 worked without a problem (apart from the fact that XO produced some hiccups with the rolling update). Nice

olivierlambert

@gskger I think we might have the same. Do you have an error log?

gskger

@olivierlambert I haven't looked for error logs (yet), but if you tell me which logs are interesting or what to look for, I'll be happy to give it a try.

stormi

@gskger mainly xensource.log, possibly daemon.log.

olivierlambert

I thought about XO Logs (in Settings/Logs).

gskger

@olivierlambert @stormi I will send you what I have tomorrow if it is not that urgent, ok?

olivierlambert

No rush at all Thanks a lot already for giving us that valuable feedback.

Andrew

@olivierlambert Install patches from from XO source (current master) and 8.2.1. It works, but I an error.

Host install:

pool.installPatches
{
  "hosts": [
    "64b9bf4a-a991-438d-973f-60179f3d0868"
  ]
}
{
  "message": "",
  "name": "Error",
  "stack": "Error: 
    at Xapi._xcpUpdate (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xapi/mixins/patching.mjs:323:15)
    at Object.installPatches (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/api/pool.mjs:115:3)
    at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xo-mixins/api.mjs:307:20)"
}

Pool install:

pool.installPatches
{
  "pool": "98ab0129-f9d1-ba08-c1c0-aa93c38d7fec"
}
{
  "message": "",
  "name": "Error",
  "stack": "Error: 
    at Xapi._xcpUpdate (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xapi/mixins/patching.mjs:323:15)
    at runMicrotasks (<anonymous>)
    at runNextTicks (node:internal/process/task_queues:61:5)
    at processImmediate (node:internal/timers:437:9)
    at process.topLevelDomainCallback (node:domain:152:15)
    at process.callbackTrampoline (node:internal/async_hooks:128:24)
    at Object.installPatches (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/api/pool.mjs:115:3)
    at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202202281818/packages/xo-server/src/xo-mixins/api.mjs:307:20)"
}

gskger

@olivierlambert @stormi I started the rolling update from XO and left it running alone since it usually takes some time to complete. This xenopsd internal error is from the XO log.

2022-02-28T15_21_32.891Z - XO.log.txt

An hour later I realized that the updates had been applied, but the hosts did not reboot. I tried to reboot the master from XO first which might have resulted in this XO log.

2022-02-28T16_35_04.612Z - XO.log.txt

After that all VMs including XO became unresponsive and I continued to investigate with XCP-ng Center. In the end, I decided to reboot the hosts from CLI (master first and than all slaves). That resolved the issue and XO came up without any further problems.

I have copies of the logs @stormi suggested from the master and one slave if they are needed. XO is XO from source by 3rd party script. I did an XO update after resolving the issue, so if you need the XO version, I can restore from backup and check.

Hope that helps.

olivierlambert

@Andrew I believe this is a problem on XO, unrelated to the XCP-ng update itself.

@gskger so we had the same issue. However, I did something differently: after restarting the toolstack, I could migrate VMs around without any problem, then reboot manually each evacuated host.

gskger

@olivierlambert Yes, restarting the toolstack first would have been a better approach. Is this something that needs additional actions on the hosts?

olivierlambert

No, that should be enough

stormi

We added a warning on the blog post https://xcp-ng.org/blog/2022/02/28/xcp-ng-8-2-1-update/ and will work on a fix in XO.

HeMaN

@stormi
now realise I had the same behaviour after applying the latest 8.2.0 security updates a few weeks ago.
At that time thought it was because I patched the master manually already (test repo release) and updated the entire pool only after the official release 2 days later (roling pool update).

So it is not (only) related to the 8.2.1 release