XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Windows Server 2016 & 2019 freezing on multiple hosts

    Scheduled Pinned Locked Moved Development
    33 Posts 12 Posters 17.9k Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • borzelB Offline
      borzel XCP-ng Center Team @olivierlambert
      last edited by borzel

      @olivierlambert the first three digits of the version number are the same, the last is usually the buildnumber... I assume they backportet some codechanges from master ... but this is nothing we can easily detect

      Edit: maybe somone with IDA can compare the last two version an do a (graphical -> flowchart?) diff?

      Edit2: I try to check the git logs in the evening ... maybe some change pop's up 🙂

      1 Reply Last reply Reply Quote 2
      • borzelB Offline
        borzel XCP-ng Center Team
        last edited by

        I asked for help: https://bugs.xenserver.org/browse/XSO-928

        1 Reply Last reply Reply Quote 1
        • michaelM Offline
          michael @300cpilot
          last edited by

          @300cpilot

          I'm going to try the remote event viewer over the next few days to see if anything of interest comes up. As for remote desktop, and ping I get no response.

          @olivierlambert @borzel

          I'll pull the latest builds and see if I have any luck. I'm also going to pull the latest Windows Server updates on a few of the VMs (seems they just came out a few hours ago as I checked for updates earlier and had nothing)

          1 Reply Last reply Reply Quote 1
          • borzelB Offline
            borzel XCP-ng Center Team
            last edited by

            As you can read here: https://bugs.xenserver.org/browse/XSO-928 there is no legal chance to get the changes made by CITRIX, because the originating code is BSD licensed 😞

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by olivierlambert

              Yeah but the Open source drivers must be updated somehow, because there is some people using Xen out there with Windows load (AWS? IBM Rackspace?)

              borzelB 1 Reply Last reply Reply Quote 0
              • borzelB Offline
                borzel XCP-ng Center Team @olivierlambert
                last edited by borzel

                @olivierlambert

                The OpenSource Drivers from the XEN-Project (https://www.xenproject.org/downloads/windows-pv-drivers/winpv-drivers-8/winpv-drivers-821.html) are the same version like ours: https://github.com/xcp-ng/win-pv-drivers/releases/tag/v8.2.1-beta1

                Maybe IBM or Rackspace uses own builds?

                1 Reply Last reply Reply Quote 0
                • michaelM Offline
                  michael
                  last edited by

                  Quick Update

                  I haven't had any crashes since posting here. Before posting here it was each of my 2016 & 2019 VMs on XCP-NG that would lock up. Now it's none of them. Some of them I updated, while others I did nothing... Strange.

                  I'll update this post again in a few days, or sooner if anything changes.

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    Thanks a lot for your feedback @michael !

                    1 Reply Last reply Reply Quote 0
                    • michaelM Offline
                      michael
                      last edited by michael

                      I had one lock up on me. Surprising the one with the shortest uptime.

                      EDIT: I just updated the xcp-emu-manager-0.0.9-1 that was posted about a bit ago. Not sure if this will help or not.

                      Have you heard of anyone else having these issues? I'm debating on doing a fresh install of XCP-NG to see if that helps.

                      EDIT 2: There are no error logs given in Windows. One minute it's working, the next it's not. I'm going through the XCP-NG logs at the moment to see if there is anything there.

                      EDIT 3: Here is a list of every line with the UUID of the VM associated with it: https://pastebin.com/ip30uyMN

                      borzelB 1 Reply Last reply Reply Quote 0
                      • borzelB Offline
                        borzel XCP-ng Center Team @michael
                        last edited by

                        @michael can you point to a specific time in the log file when your VM was locked up?

                        michaelM 1 Reply Last reply Reply Quote 0
                        • michaelM Offline
                          michael @borzel
                          last edited by michael

                          @borzel Sometime between the 7:40 and 12:12 on the first line. I'm guessing it went down right as I was posting here.

                          Like I said whatever it's doing it doesn't seem to log it. Is there any other log files I should check?

                          EDIT: I have a theory, testing it now.

                          EDIT 2: I'm wondering if this has something to do with XCP-NG Center. The KMS server (the one that locked up) was the only one I logged into when checking if everything was working. Trying to duplicate this now.

                          1 Reply Last reply Reply Quote 0
                          • michaelM Offline
                            michael
                            last edited by

                            Update: The same VM crashed again, the others are working fine as of this post. I left it up as a remote desktop session last night and I came back to my office today and it was still up as a RDP session, but not responsive. I checked XCP-NG Center and the same blue screen with a cursor was there.

                            Still trying to find the cause of this.

                            apayneA 1 Reply Last reply Reply Quote 0
                            • E Offline
                              etlweather
                              last edited by

                              Were you able to resolve this in the end?

                              1 Reply Last reply Reply Quote 0
                              • apayneA Offline
                                apayne @michael
                                last edited by

                                @michael this may sound silly, but perhaps this isn’t a software issue? Maybe you have a faulty stick of RAM hiding in the machine? Bad RAM will make all kinds of strange and flaky things happen. This is just a guess, I’m just putting this idea out there because I’ve seen similar weird behavior in machines where RAM failure isn’t easy to spot (no front panel on the server with a fault light).

                                nikadeN 1 Reply Last reply Reply Quote 0
                                • nikadeN Offline
                                  nikade Top contributor @apayne
                                  last edited by

                                  @apayne This is a good suggestion, all tho it is happening on a lot of different physical machines if im not mistaken.

                                  Im running a couple of Windows Server 2016 on xcp-ng 7.4 and 7.5 without any issues, im hoping this isnt something related to the specific version xcp-ng 7.6 since we're planning on migrating from XenServer to this version.

                                  1 Reply Last reply Reply Quote 0
                                  • C Offline
                                    cg
                                    last edited by cg

                                    Any updates on this?
                                    I'm planing to run 2 HPE DL20 on XCP-ng 7.6 (Install image for 8 is currently broken to run on that - reason still unknown).

                                    B 1 Reply Last reply Reply Quote 0
                                    • B Offline
                                      bikepope @cg
                                      last edited by

                                      @cg Hello
                                      I run into the same issue with Vserver (Windows Server 2016). It looks like not only XCP has this problem. Also Citrix, VMWare and Hyper-v face this. Wondering if there is any solution!!!

                                      1 Reply Last reply Reply Quote 0
                                      • olivierlambertO Offline
                                        olivierlambert Vates 🪐 Co-Founder CEO
                                        last edited by

                                        Could it be related to a bug in Windows itself?

                                        S 1 Reply Last reply Reply Quote 0
                                        • S Offline
                                          speedy01 @olivierlambert
                                          last edited by

                                          iirc vmware had hit problems with windows 10 1809 and up...there was some finger pointing, but vmware released a patch in update 3. Since the reported problem popped up after 1809, then Id suspect it more of a microsoft issue...

                                          1 Reply Last reply Reply Quote 0
                                          • E Offline
                                            etlweather
                                            last edited by

                                            For what it's worth, I was having Windows 2016 freezes regularly (every few days or week). I could never trace it back to a specific cause, though I noticed very high number of handles and threads when it was hanging.

                                            Per my records, it stopped hanging after I upgraded the PV drivers using Guest Tool ISO 7.41.0-1.

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post