XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Endless Xapi#getResource /rrd_updates in tasks list

    Scheduled Pinned Locked Moved XCP-ng
    85 Posts 10 Posters 27.6k Views 10 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • MathieuM Offline
      Mathieu @olivierlambert
      last edited by

      @olivierlambert
      Simpler setup on my side:

      Pool 1 - 1 x ASRock Rack 1U4LW-X570/2L2T RPSU with about 10 VMs
      Pool 2 - 1 x HP DL360 Gen9 with only the XO VM (Debian 11).

      Each host has 2 x 10 Gb NIC in use (one for VMs/MGMT, the other one for NFS storage).

      The storage is a NFS share on a QNAP NAS (except for the XO VM which is on the local storage of the DL360 host).

      The 2 hosts and the NFS storage are connected on the same10Gbit Ubiquiti EdgeSwitch.

      Same plugins as @14wkinnersley + audit and sdn-controller.

      The issue with stuck task is only appearing on the pool 1, not on the pool 2 with only the XO VM.

      1 Reply Last reply Reply Quote 0
      • 1 Offline
        14wkinnersley @olivierlambert
        last edited by

        @olivierlambert Will do. Plugins are disabled and I'm updating back to master right now. Will report back.

        MathieuM 1 Reply Last reply Reply Quote 1
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by olivierlambert

          Thank you very much, both of you 🙂

          1 Reply Last reply Reply Quote 0
          • MathieuM Offline
            Mathieu @14wkinnersley
            last edited by

            I'm gonna try the same and will let you know.

            1 Reply Last reply Reply Quote 1
            • 1 Offline
              14wkinnersley @olivierlambert
              last edited by

              @olivierlambert Updated to master (Commit: 79c9ef0) with perf alert and load balancer plugin's disabled. Cleared the toolstack for good measure, and I got no hanging tasks.

              For some further testing, I flipped the load balancer plugin back on and waited about 5 hours and received no hanging tasks. I turned load balancer plugin back off (Just in case), cleared the toolstack, and turned the perf alert plugin on and received the first hanging task within about 10 minutes, and subsequently had more tasks hang every ~10-12 minutes.

              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                That would have been my next question 😄 So it's the perf alert thing likely. Thank you, this will help @julien-f a lot

                MathieuM 1 Reply Last reply Reply Quote 0
                • MathieuM Offline
                  Mathieu @olivierlambert
                  last edited by

                  @olivierlambert
                  It took a while to validate it, but I can also confirm that the perf-alert plugin is the culprit.

                  Here are my plugin's parameters, if it can help chasing the goose:

                  13a55af0-f2f9-462e-b050-74072d7bb500-image.png

                  M 1 Reply Last reply Reply Quote 0
                  • M Offline
                    manilx @Mathieu
                    last edited by

                    @Mathieu I just activated this plugin as a test on our 2 EPYC hosts and I also get these tasks now. So yes, this plugin has an issue.

                    Deactivated.

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by

                      Thanks @julien-f is now reviewing the plugin code 🙂

                      1 Reply Last reply Reply Quote 0
                      • R Offline
                        rmarion
                        last edited by

                        I am seeing this same issue on master build cc049 if I turn on perf-alert it starts with in a few minutes. If I turn it off and restart the tool stack it stops.

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO Offline
                          olivierlambert Vates 🪐 Co-Founder CEO
                          last edited by

                          It's been fixed since a bit now. Double check you have the correct plugin code rebuilt.

                          1 1 Reply Last reply Reply Quote 0
                          • 1 Offline
                            14wkinnersley @olivierlambert
                            last edited by 14wkinnersley

                            @olivierlambert Unfortunately, I am still having this issue if I enable perf-alert. Im on the latest commit (cc049d3) and perf-alert (v0.3.7). I removed the plugin entirely from /packages/xo-server-perf-alert and rebuilt to test further (and restarted toolstack). Still having the tasks hang up 😕

                            1 Reply Last reply Reply Quote 0
                            • olivierlambertO Offline
                              olivierlambert Vates 🪐 Co-Founder CEO
                              last edited by

                              Can you try with an XOA on latest release channel?

                              1 1 Reply Last reply Reply Quote 0
                              • R Offline
                                rmarion
                                last edited by

                                Just rebuild open source from scratch and still getting the same issue

                                859590be-81b1-4fc3-9acb-7bda183d5cbf-image.png
                                fd9a3945-3b81-4969-83f4-7bb377d66c9c-image.png

                                9c7540d1-7d4c-4a19-accf-beffdde03255-image.png

                                1 Reply Last reply Reply Quote 0
                                • 1 Offline
                                  14wkinnersley @olivierlambert
                                  last edited by

                                  @olivierlambert Maybe I am doing something wrong but I went ahead and deployed xoa, and under plugins I only have two plugins and neither are for perf-alert. Ive never used XOA, only built XO, so its probably user error.

                                  1 Reply Last reply Reply Quote 0
                                  • R Offline
                                    rmarion
                                    last edited by

                                    you have to register XOA in order to activate all of the plugins. You do it under the XOA tab. I would role it out to test but I have already registered it one and can't do it again. So I to have been running the open source code for my testing and evaluations.

                                    f7d97e44-be5d-4fee-8d35-5f7ab04ae2bd-image.png

                                    1 1 Reply Last reply Reply Quote 0
                                    • 1 Offline
                                      14wkinnersley @rmarion
                                      last edited by

                                      @rmarion Im registered but dont have a license. Dont really want to use up the free trial, and it still only shows the two plugins. So maybe someone else can test.

                                      R 1 Reply Last reply Reply Quote 0
                                      • R Offline
                                        rmarion @14wkinnersley
                                        last edited by

                                        @14wkinnersley Ya I don't work for Yates so I don't have any other option for you on that front. I was just advising on how you could test it with the trial.

                                        1 Reply Last reply Reply Quote 0
                                        • olivierlambertO Offline
                                          olivierlambert Vates 🪐 Co-Founder CEO
                                          last edited by

                                          @14wkinnersley Please use the trial, we can extend it anytime if you need.

                                          1 1 Reply Last reply Reply Quote 0
                                          • 1 Offline
                                            14wkinnersley @olivierlambert
                                            last edited by 14wkinnersley

                                            @olivierlambert I activated the trial and imported my configuration from xo, making sure the perf-alert plugin was enabled/configured. I have not received any hanging tasks in XOA.

                                            1 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post