XCP-ng

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups

    Updates announcements and testing

    News
    61
    545
    192674
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stormi
      stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by

      Unrelated to the above: a security update for sudo was published. I don't think it's very likely to be an actual threat in the context of your use of XCP-ng, but it might be in specific contexts.

      https://xcp-ng.org/blog/2023/01/31/january-2023-security-update/

      brezlord 1 Reply Last reply Reply Quote 0
      • brezlord
        brezlord @stormi last edited by brezlord

        @stormi Applied the update through XO and now XO can not login to the host with the below error.

        connect ECONNREFUSED 192.168.40.201:443
        

        I rebooted the host and I can no longer login as root.

        ssh: connect to host 192.168.40.201 port 22: Connection refused
        

        Any ideas?

        1 Reply Last reply Reply Quote 0
        • stormi
          stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by stormi

          Was it the only update applied? Is the stunnel service running?

          Oh, I also read that you can't connect as root.

          brezlord 1 Reply Last reply Reply Quote 0
          • brezlord
            brezlord @stormi last edited by

            @stormi Yes only update.

            1 Reply Last reply Reply Quote 0
            • stormi
              stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by

              Were you using sudo on the host before?

              brezlord 1 Reply Last reply Reply Quote 0
              • brezlord
                brezlord @stormi last edited by

                @stormi No just default install

                1 Reply Last reply Reply Quote 0
                • stormi
                  stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by

                  I hardly see a cause-effect link between the update and the issues (both SSH and XAPI not responding anymore?), but computers are full of surprises.

                  Did you change the firewall configuration? Could the IP address have changed or the same be attributed to another device?

                  brezlord 1 Reply Last reply Reply Quote 0
                  • brezlord
                    brezlord @stormi last edited by

                    @stormi Done nothing but apply the update through XO web console. I have yanked the plug and making sure it actually reboots.

                    1 Reply Last reply Reply Quote 0
                    • brezlord
                      brezlord last edited by

                      That fixed it I can login via ssh with root and XO sees the host.

                      1 Reply Last reply Reply Quote 0
                      • stormi
                        stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by

                        Maybe it was still rebooting, stuck on the shutdown phase, waiting for some kind of I/O or something. This would explain why it didn't respond.

                        brezlord 1 Reply Last reply Reply Quote 0
                        • olivierlambert
                          olivierlambert Vates πŸͺ Co-Founder🦸 CEO πŸ§‘β€πŸ’Ό last edited by olivierlambert

                          I concur. If the shutdown process is stuck somewhere (eg an NFS share), you can't connect at all (connection refused in SSH, no XAPI connection) and it can stays like this for a while.

                          1 Reply Last reply Reply Quote 0
                          • brezlord
                            brezlord @stormi last edited by

                            @stormi It was not responding after the update from XO. I could long in via ssh and restarted the tool stack but this did not help XO still could not login. I issued a reboot command via ssh which dropped the ssh session and the host did not reboot most likely due to running VMs. I then yanked the power and the host rebooted and everything is working as it should. The update definitely caused the issue.

                            1 Reply Last reply Reply Quote 0
                            • olivierlambert
                              olivierlambert Vates πŸͺ Co-Founder🦸 CEO πŸ§‘β€πŸ’Ό last edited by

                              I did update my home lab without any issue, before, during and after the update (I did test just after the update without any reboot).

                              brezlord 1 Reply Last reply Reply Quote 0
                              • brezlord
                                brezlord @olivierlambert last edited by

                                @olivierlambert This is my home lab as well running on a small form factor PC with an Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz

                                1 Reply Last reply Reply Quote 0
                                • stormi
                                  stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by stormi

                                  XO restarts the toolstack after installing updates. So what first went wrong is this: it couldn't restart. You can't say for sure it's caused by the update, because there are many other reasons that can make this fail. The logs would tell. For example a known bug, being fixed, in xenospd-xc, which makes it unable to restart when something specific happened to VM metadata.

                                  So we'll keep an eye and ears open for any other occurrence of this issue in relation with the update, but I still think there's little chance an update of sudo would itself cause this.

                                  We'll do a few additional tests to see if we can reproduce.

                                  brezlord 1 Reply Last reply Reply Quote 0
                                  • brezlord
                                    brezlord @stormi last edited by

                                    @stormi If you direct me to where the log you need are I can provide them.

                                    1 Reply Last reply Reply Quote 0
                                    • stormi
                                      stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by

                                      /var/log/xensource.log and /var/log/daemon.log would be the first ones to check.

                                      1 Reply Last reply Reply Quote 0
                                      • brezlord
                                        brezlord last edited by

                                        @stormi you can download the logs here.

                                        1 Reply Last reply Reply Quote 0
                                        • stormi
                                          stormi Vates πŸͺ XCP-ng Team πŸš€ last edited by

                                          I see this in daemon.log, a message from systemd attempting to shut the system down:

                                          Feb  1 22:11:35 xcp-ng-01 systemd[1]: Unmounted /run/sr-mount/5f5a9343-b95a-9bfa-bd3a-bc30d7368058.
                                          Feb  1 22:11:35 xcp-ng-01 systemd[1]: Failed to propagate agent release message: Transport endpoint is not connected
                                          Feb  1 22:11:35 xcp-ng-01 systemd[1]: Failed to propagate agent release message: Transport endpoint is not connected
                                          Feb  1 22:11:35 xcp-ng-01 systemd[1]: Failed to propagate agent release message: Transport endpoint is not connected
                                          Feb  1 22:11:35 xcp-ng-01 systemd[1]: Failed to propagate agent release message: Transport endpoint is not connected
                                          Feb  1 22:11:35 xcp-ng-01 systemd[1]: Failed to propagate agent release message: Transport endpoint is not connected
                                          

                                          There definitely was a network mountpoint (a NFS SR) which was not connected anymore. This explains the long reboot time.

                                          Going up the logs, I see this:

                                          Feb  1 22:01:43 xcp-ng-01 systemd[1]: xenopsd-xc.service: main process exited, code=exited, status=2/INVALIDARGUMENT
                                          Feb  1 22:01:43 xcp-ng-01 systemd[1]: Unit xenopsd-xc.service entered failed state.
                                          Feb  1 22:01:43 xcp-ng-01 systemd[1]: xenopsd-xc.service failed.
                                          

                                          This explains the failed XAPI restart and is likely the known issue with xenopsd I mentioned above.

                                          So, if I'm not wrong, it's good news:

                                          • The xenospd issue is known and a fix is on its way and usually disappears after a reboot.
                                          • The update itself probably didn't cause your issues.
                                          1 Reply Last reply Reply Quote 0
                                          • olivierlambert
                                            olivierlambert Vates πŸͺ Co-Founder🦸 CEO πŸ§‘β€πŸ’Ό last edited by

                                            haha my "gut feeling" Β© approved πŸ˜„

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post