XCP-ng 8.2 updates announcements and testing

Ultra2D

@stormi Thanks. Installing the update candidate on all hosts and restarting the toolstack works, but only after power cycling the VM once.
xcp-emu-manager is version 0.0.9-1

Is it advisable to stay on the testing repo until the next version? There are some more non-Windows HVM VM's.

stormi

@Ultra2D so you mean that installing the updated packages would "break" the first migration of such a VM unless it's been rebooted once? If that is so, then I'd advise to revert to the previous version (or to make sure not to attempt a migration without power cycling the VMs once). Else, your choice. If the updated packages bring a benefit to you, you can keep them, else revert to the previous ones.

Ultra2D

@stormi I only tested with one VM. It crashed a couple of times when the pool master had updates from xcp-ng-updates_testing and the slaves had the updates that were released yesterday. After updating the last remaining slave to updates from xcp-ng-updates_testing, moving the VM resulted in a stuck VM. So I don't think you can draw any conclusions from this, except maybe that you should install the same version on master and slaves.

stormi

Xen security updates pushed to everyone (7.6 yesterday, 7.5 today).
Blog post: https://xcp-ng.org/blog/2019/03/12/xcp-ng-security-bulletin-vulnerabilities-pv-guests/

codedmind

I'm getting an error


Running transaction test


Transaction check error:
  installing package xen-dom0-tools-4.7.6-6.4.1.xcpng.x86_64 needs 1 inodes on the /var/log filesystem

stormi

@codedmind This is probably related to a bug in the openvswitch package and log rotation. We sent a newsletter about it a few days ago: https://mailchi.mp/7ed52f9a2151/important-noticeopenvswitch-issue

codedmind

Humm ok... but i cannot update... yes i have that version.. but as i cannot up date how can i solve it?

stormi

@codedmind when it got to the point that you ran out of inodes in /var/log, you need to remove some files before you can update. find /var/log -name ovsdb-server.log.*.gz -delete should work (untested, try without -delete first). Else look at the other threads that covered the subject.

codedmind

Ok thanks!

rechotech

Does the openvswitch bug cause the local storage to fill up, or just the log file?
Would this openvwswitch potential lead our XCP server to crash?

stormi

Local storage is not touched. Only the /var/log partition, or the / partition if you have no /var/log partition. In the latter case, I suppose this could hang or crash the server. In the former case, it prevents some new tasks to be performed, but we haven't had reports of crashing servers at this stage.

rechotech

since this is unlikely the cause of our crash, where would you suggest we look to determine why our xcp servers crashed a few days ago?

stormi

I suggest opening a dedicated thread with as much information as possible.

stormi

As a starting point, see https://github.com/xcp-ng/xcp/wiki/Logfiles and look for whatever happened by the time of the crashes, if logging was still operational at that moment.

stormi

For XCP-ng 7.6 only

New security update for the latest Intel CPU vulnerabilities:

http://xenbits.xen.org/xsa/advisory-297.html

Install it with:

yum update --enablerepo='xcp-ng-updates_testing' microcode_ctl-2.1-26.xs5.x86_64 xen-dom0-libs-4.7.6-6.5.1.xcpng.x86_64 xen-dom0-tools-4.7.6-6.5.1.xcpng.x86_64 xen-hypervisor-4.7.6-6.5.1.xcpng.x86_64 xen-libs-4.7.6-6.5.1.xcpng.x86_64 xen-tools-4.7.6-6.5.1.xcpng.x86_64

There's nothing fancy in the way the updates were built so no breakage is expected, but we still need feedback from early-adopters before we push them to everyone.

Reboot required.

Downgrade if needed with:

yum downgrade microcode_ctl xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools

What to test:

basic functions of the hypervisor, VMs...

Read the advisory carefully: installing the updates is not enough. There are other steps to be taken for complete protection.

Citrix announcement: https://support.citrix.com/article/CTX251995

Ultra2D

@stormi said in Updates announcements and testing:

yum update --enablerepo='xcp-ng-updates_testing' microcode_ctl-2.1-26.xs5.x86_64 xen-dom0-libs-4.7.6-6.5.1.xcpng.x86_64 xen-dom0-tools-4.7.6-6.5.1.xcpng.x86_64 xen-hypervisor-4.7.6-6.5.1.xcpng.x86_64 xen-libs-4.7.6-6.5.1.xcpng.x86_64 xen-tools-4.7.6-6.5.1.xcpng.x86_64

Should the pool master be updated first? Or it doesn't matter for there updates?

stormi

@Ultra2D Yes, always the pool master first.

stormi

@Ultra2D Technically, some updates could be installed in any order, but it's safer and simpler to consider that master is always updated first.

Ultra2D

@stormi Sure, but I'd rather test on a pool member first if trying as an early adopter .

stormi

@Ultra2D For this update, I think you could try with a slave first, since it's only an update of xen (and microcode_ctl), so I don't expect incompatibilities in pool management. But I wouldn't guarantee that cross-host operations such as migrations would work fine.