XCP-ng 8.3 betas and RCs feedback đ
-
@ph7 I did push a rebuilt netdata for Xen 4.17 yesterday, so it should have worked, but I admit I haven't tested the update process myself so maybe I've missed something.
Can you extract the error messages from the output of
yum history?yum historyto get the list of operations, andyum history info NUMBERfor the details of a given call. -
@Houbsi What was the update method used?
-
@stormi
I used "yum update" with the 8.3 repo in the yum.repos.d file activated.After that i always do a sanity reboot.
-
@Houbsi So what job got killed waiting for the reboot? I don't have all context here
-
[ OK ] Stopped The Xen xenstore. [ !! ] Forcibly rebooting: job timed outThat's all what i got from the console, if there is a log somewhere with more information, i'll be happy to provide it to you!
-
@archw Maybe similar issue to what Houbsi saw, long shutdown due to a stuck task.
-
@Houbsi anything above this that would tell us what job was stuck? The xenstore stop ended well, as the
[ OK ]message indicates. -
All i have from a screenshot, and from what i saw was ok, down i'll put the log i screenshoted to a colleague in the datacenter
[ OK ] Stopped TLS tunnel for xapi. [ OK ] Stopped Generate TLS certificates for x Stopping Generate TLS certificates for [ OK ] Stopped Xapi storage script plugin serv [ OK ] Stopped target System Time Synchronized [ OK ] Stopped Wait for chrony to synchronize Stopping Wait for chrony to synchronize Stopping NTP client/server... [ OK ] Stopped NTP client/server. [ OK ] Stopped Tapback daemon for blktap3. Stopping The Xen xenstore... [ OK ] Stopped The Xen xenstore. [ !! ] Forcibly rebooting: job timed outWhat i just remember, is that my monitoring for my mailserver, running on a vm with alma and mailcow-dockerized, did not trigger until the Forced rebooting. Maybe in the "reboot" process the shutdown of the VM's hindered the process? I hadn't had that problem before. Just when i try to do the normal reboot process in XOA. This VM has the options Auto power on", "Protect from accidental deletion", "Protect from accidental shutdown" set to true. (Just verified the settings)
-
@Houbsi Anyway, I believe this was a transient issue, maybe related to the big jump in versions in the latest packages. Keep us in touch if this happens again.
-
S stormi forked this topic on
-
@stormi said in XCP-ng 8.3 beta
:@ph7 I did push a rebuilt netdata for Xen 4.17 yesterday, so it should have worked, but I admit I haven't tested the update process myself so maybe I've missed something.
Can you extract the error messages from the output of
yum history?yum historyto get the list of operations, andyum history info NUMBERfor the details of a given call.[12:23 X3 ~]# yum history InlĂ€sta insticksmoduler: fastestmirror ID | Kommandorad | Datum och tid | Ă tgĂ€rd(er) | Ăndrade ------------------------------------------------------------------------------- 11 | update --disablerepo=* - | 2024-04-23 00:29 | I, U | 53 EE 10 | remove netdata-ui.x86_64 | 2024-04-23 00:26 | Erase | 2 9 | update --disablerepo=* - | 2024-04-18 18:11 | Update | 21 8 | update --disablerepo=* - | 2024-04-04 17:50 | Update | 1 7 | reinstall netdata.x86_64 | 2024-04-03 16:41 | Reinstall | 1 6 | reinstall netdata-ui | 2024-04-03 16:34 | Reinstall | 1 5 | --enablerepo=* install n | 2024-03-30 11:41 | Install | 1 4 | update --disablerepo=* - | 2024-03-29 14:13 | Update | 21 3 | install netdata-ui | 2024-03-23 13:59 | Install | 9 EE 2 | update --disablerepo=* - | 2024-03-01 16:00 | Update | 6 1 | update --disablerepo=* - | 2024-02-29 22:12 | I, UF, U | 63 history list --------------------------- [12:23 X3 ~]# yum history info NUMBER 10 InlĂ€sta insticksmoduler: fastestmirror Transaktions-ID: 10 Starttid : Tue Apr 23 00:26:27 2024 Start-rpmdb : 548:9dff1ffa674baaf209143925833d8346df54beac Sluttid : 00:27:27 2024 (60 sekunder) Slut-rpmdb : 546:17e105aef8586ef4615b3761b221b4051cc02275 AnvĂ€ndare : System <ej satt> Returkod : Lyckades Kommandoradsfel: remove netdata-ui.x86_64 netdata.x86_64 Transaktionen utförd med: Installerade rpm-4.11.3-32.el7.x86_64 @xcp:main/$releasever Installerade yum-3.4.3-158.el7.centos.noarch @xcp:main/$releasever Installerade yum-plugin-fastestmirror-1.1.31-46.el7_5.noarch @xcp:main/$releasever Ăndrade paket: Radering netdata-1.19.0-6.xcpng8.3.x86_64 @xcp-ng-base Radering netdata-ui-1.19.0-6.xcpng8.3.x86_64 @xcp-ng-base history info Excluding mirror: updates.xcp-ng.org * xcp-ng-updates: mirrors.xcp-ng.org Uppdaterade intel-ice-1.11.17.1-1.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 1.11.17.1-2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade intel-microcode-20231009-1.xcpng8.3.noarch @xcp-ng-base Uppdatering 20240130-1.xcpng8.3.noarch @xcp-ng-base Uppdaterade kernel-4.19.19-8.0.32.1.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 4.19.19-8.0.33.1.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade kexec-tools-1:2.0.15-18.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 1:2.0.15-19.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade libcgroup-0.41-15.el7.x86_64 @?xcp-ng-base Uppdatering 0.41-21.el7.x86_64 @xcp-ng-base Uppdaterade libcgroup-tools-0.41-15.el7.x86_64 @?xcp-ng-base Uppdatering 0.41-21.el7.x86_64 @xcp-ng-base Uppdaterade message-switch-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade python2-xapi-storage-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Ber-inst python3-scapy-2.4.5-3.xcpng8.3.noarch @xcp-ng-base Uppdaterade qemu-2:4.2.1-5.2.4.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 2:4.2.1-5.2.7.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade qlogic-fastlinq-8.55.13.0-1.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 8.74.0.2-1.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade rrdd-plugins-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade sm-3.0.12-6.2.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 3.0.12-12.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade sm-cli-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade squeezed-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade varstored-1.2.0-1.3.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 1.2.0-2.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade varstored-guard-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade varstored-tools-1.2.0-1.3.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 1.2.0-2.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade vcputune-2.0.1-5.xcpng8.3.noarch @?xcp-ng-base Uppdatering 2.0.2-1.xcpng8.3.noarch @xcp-ng-base Uppdaterade vhd-tool-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade vncterm-10.2.0-3.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 10.2.1-2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade wsproxy-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xapi-core-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xapi-nbd-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xapi-rrd2csv-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xapi-storage-script-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xapi-tests-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xapi-xe-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xcp-clipboardd-1.0.3-7.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 1.0.3-8.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xcp-featured-1.1.6-3.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 1.1.6-4.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xcp-networkd-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xcp-ng-release-8.3.0-18.x86_64 @xcp-ng-base Uppdatering 8.3.0-19.x86_64 @xcp-ng-base Uppdaterade xcp-ng-release-config-8.3.0-18.x86_64 @xcp-ng-base Uppdatering 8.3.0-19.x86_64 @xcp-ng-base Uppdaterade xcp-ng-release-presets-8.3.0-18.x86_64 @xcp-ng-base Uppdatering 8.3.0-19.x86_64 @xcp-ng-base Uppdaterade xcp-rrdd-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xen-crashdump-analyser-2.5.6-1.xcpng8.3.x86_64 @?xcp-ng-base Uppdatering 2.6.1-1.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xen-dom0-libs-4.13.5-10.58.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 4.17.3-4.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xen-dom0-tools-4.13.5-10.58.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 4.17.3-4.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xen-hypervisor-4.13.5-10.58.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 4.17.3-4.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xen-libs-4.13.5-10.58.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 4.17.3-4.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xen-tools-4.13.5-10.58.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 4.17.3-4.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xenopsd-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xenopsd-cli-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xenopsd-xc-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade xenserver-status-report-1.3.14-1.xcpng8.3.noarch @xcp-ng-base Uppdatering 2.0.3-1.xcpng8.3.noarch @xcp-ng-base Skriptutdata: 1 warning: /etc/sysconfig/xencommons saved as /etc/sysconfig/xencommons.rpmsave history info -------------------------- [12:25 X3 ~]# yum history info NUMBER 11 InlĂ€sta insticksmoduler: fastestmirror Transaktions-ID: 11 Starttid : Tue Apr 23 00:29:47 2024 Start-rpmdb : 546:17e105aef8586ef4615b3761b221b4051cc02275 Sluttid : 00:31:19 2024 (92 sekunder) Slut-rpmdb : 547:ab22718930a5c79c4616bc82131af44e4ed6cb50 AnvĂ€ndare : System <ej satt> Returkod : Lyckades Kommandoradsfel: update --disablerepo=* --enablerepo=xcp-ng-base,xcp-ng-updates -y Transaktionen utförd med: Installerade rpm-4.11.3-32.el7.x86_64 @xcp:main/$releasever Installerade yum-3.4.3-158.el7.centos.noarch @xcp:main/$releasever Installerade yum-plugin-fastestmirror-1.1.31-46.el7_5.noarch @xcp:main/$releasever Ăndrade paket: Uppdaterade blktap-3.54.6-2.1.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 3.54.6-3.1.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade edk2-20220801-1.7.3.2.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 20220801-1.7.4.1.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade forkexecd-23.31.0-1.7.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 24.11.0-1.2.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade gpumon-0.25.0-6.1.xcpng8.3.x86_64 @xcp-ng-base Uppdatering 0.25.0-14.1.xcpng8.3.x86_64 @xcp-ng-base Uppdaterade guest-templates-json-2.0.6-1.1.xcpng8.3.noarch @xcp-ng-base Uppdatering 2.0.9-1.1.xcpng8.3.noarch @xcp-ng-base Uppdaterade guest-templates-json-data-linux-2.0.6-1.1.xcpng8.3.noarch @xcp-ng-base Uppdatering 2.0.9-1.1.xcpng8.3.noarch @xcp-ng-base Uppdaterade guest-templates-json-data-other-2.0.6-1.1.xcpng8.3.noarch @xcp-ng-base Uppdatering 2.0.9-1.1.xcpng8.3.noarch @xcp-ng-base Uppdaterade guest-templates-json-data-windows-2.0.6-1.1.xcpng8.3.noarch @xcp-ng-base Uppdatering 2.0.9-1.1.xcpng8.3.noarch @xcp-ng-base Loading mirror speeds from cached hostfile Excluding mirror: updates.xcp-ng.org * xcp-ng-base: mirrors.xcp-ng.org --------------------------- I also got the result from my update when I discovered net netdata issue: yum update --disablerepo=* --enablerepo=xcp-ng-base,xcp-ng-updates -y ....... Fel: Paket: netdata-1.19.0-6.xcpng8.3.x86_64 (@xcp-ng-base) Behöver: libxenlight.so.4.13()(64bit) Tar bort: xen-dom0-libs-4.13.5-10.58.xcpng8.3.x86_64 (@xcp-ng-base) libxenlight.so.4.13()(64bit) Uppdaterat av: xen-dom0-libs-4.17.3-4.xcpng8.3.x86_64 (xcp-ng-base) ~libxenlight.so.4.17()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.4-10.27.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.4-10.35.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.4-10.36.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.40.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.42.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.42.1.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.42.3.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.51.xcpng8.3.x86_64 (xcp-ng-base) libxenlight.so.4.13()(64bit) Fel: Paket: netdata-1.19.0-6.xcpng8.3.x86_64 (@xcp-ng-base) Behöver: libxenstat.so.4.13()(64bit) Tar bort: xen-dom0-libs-4.13.5-10.58.xcpng8.3.x86_64 (@xcp-ng-base) libxenstat.so.4.13()(64bit) Uppdaterat av: xen-dom0-libs-4.17.3-4.xcpng8.3.x86_64 (xcp-ng-base) ~libxenstat.so.4.17()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.4-10.27.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.4-10.35.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.4-10.36.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.40.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.42.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.42.1.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.42.3.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) TillgĂ€ngliga: xen-dom0-libs-4.13.5-10.51.xcpng8.3.x86_64 (xcp-ng-base) libxenstat.so.4.13()(64bit) Du kan försöka anvĂ€nda --skip-broken för att gĂ„ runt problemet Du kan försöka köra: rpm -Va --nofiles --nodigestSorry for the Swedish languish and the long text
Do You need a translation ???? -
I edited your post to correctly use Markdown code syntax.
-
@ph7 It didn't see
netdata-1.19.0-7.xcpng8.3.x86_64was available. Probably you got the repo metadata in the time between when I published all the packages but netdata, and the time when I published netdata, and then yum didn't refresh the metadata because they were "fresh enough" in its eyes.Installing
netdatashould work now. -
@stormi
I was hot yesterday evening
maybe i should wait a few minutes at the next updateI installed
netdata-1.19.0-7.xcpng8.3.x86_64netdata-ui-1.19.0-7.xcpng8.3.x86_64
Can confirm that it works fine direct in the browser.
Not tested inside xoAnd as you can see, I have now taken a quick course in markdown for dummies
-
after last bunch of updates via XO and toolstack restart, maintenance mode not disabled automaticaly.
going fine after reboot. -
@stormi said in XCP-ng 8.3 beta
:@xerxist said in XCP-ng 8.3 beta
:Which kernel are you looking at since 4.19 will be EOL in 9 months?
So, the main blocker in the way to upgrade the kernel is a kernel module we use for storage access from the VMs.
I'm curious: which module?
-
A couple of issues in my environment:
-
Stats are still broken for me, even after ensuring XO was updated (from source) and that the latest patches were applied to my pool. Was there a public GH issue tracking this breakage? (I hope to use the issue to track back to the code to find out why and see if there's something unique about my environment that might be the cause)
-
I know pure PV VMs are entirely unsupported, so I don't expect this to go anywhere, so no big deal if this remains broken (for me anyways!) but after the latest updates XAPI throws "shadow_allocation_set ${sz}MB invalid argument" (where ${sz} depends on static memory config for that VM) when I try to start a PV VM. To the best of my understanding, shadow pages are only supported for HVM, right?
Thanks!
Kevin -
-
@r0ssar00 You should convert PV VMs to HVM. However, if you really can't, you can switch them from
pvtopv-in-pvh. It's not documented yet but this blog post will give you an example: https://xcp-ng.org/blog/2022/01/17/removing-support-for-32-bit-pv-guests/ -
@r0ssar00 Regarding broken stats, you could try to find logs related to them. Failures related to "stat" or "rrd" keywords, possibly. There was no github issue opened, so most details about the issue others had will only be found in this thread.
-
@r0ssar00 Could this be your issue with missing stats?
If statistics (all VMs and hosts) are not showing for a specific pool, check if there is a Backup network configured on your pool (setting is in the Advanced tab of the pool) and make sure XO can access all hosts of the pool via this network.
-
@Danp this was exactly it, thanks! Although I'm 100% sure the network is accessible to both hosts (they're directly connected to each other, no switch in between, ping to/from hosts/VMs-on-either-host works just fine), clearing it resolved the problem. I tried the other two networks I have set up (one is directly connected, one goes through a switch and is shared with the rest of my LAN) and only using the primary management interface worked (which kinda defeats the purpose...).
If this strikes you as a bug, happy to collect logs/etc, just let me know what and (a pointer to) how!
