XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. fasterfourier
    3. Best
    F
    Offline
    • Profile
    • Following 0
    • Followers 0
    • Topics 0
    • Posts 13
    • Groups 0

    Posts

    Recent Best Controversial
    • RE: Alert: Control Domain Memory Usage

      Our Citrix ticket has been worked and they concluded that the NIC driver is to blame here as well. They had us collect debug info using:

      /opt/xensource/libexec/xen-cmdline --set-dom0 page_owner=on
      

      They then confirmed the memory leak was from the NIC driver. They are intending to release a public hotfix for this issue.

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      @olivierlambert

      Probably plenty of Citrix customers were affected, but they would rather reboot on schedule than spend months working through the support process 🙂

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      @olivierlambert

      Official Citrix update has been posted: https://support.citrix.com/article/CTX306529

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      I am seeing similar behavior with Citrix Hypervisor 8.2LTSR after upgrading from 7.1CU2, which was not affected. We have a pool with 5 Poweredge R730 hosts and 2 R720 hosts. All have Intel 10G and 1G NICs (ixgbe and igb drivers) and we use iSCSI storage. I have had two hosts use up all their control domain memory, requiring an evacuate/reboot of the host. One host was the pool master, which runs only one VM (xen orchestra appliance) but is generally busy with various iSCSI tasks due to snapshot coalesce after daily backups. The other host has ~20 VMs that are pretty busy with network activity. No userspace processes that seem to be using an abnormal amount of memory.

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      One more observation here. This issue does not occur on a different pool of ours that's also running CH8.2LTSR. That pool has lower loading overall, 2 hosts instead of 7, and does not contain any NICs using the ixgbe driver. Other aspects of the pool are identical.

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      @stormi

      I am also suspicious of this diagnosis, and I think this is likely related to checking off the "misalignments" in our configuration before escalating the case to the next level of troubleshooting support. That said, I figured I'd run it by the group here to see if there's any correlation between users with dynamic memory on their VMs and this issue.

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      @garyabrahams

      Sorry I was unclear, but we are not running Kubernetes in our environment. We are running Citrix Hypervisor 8.2 LTSR.

      posted in Compute
      F
      fasterfourier
    • RE: Alert: Control Domain Memory Usage

      @stormi The ID for the case I just opened is 80240347. If you have a bugtracker issue open, you may want to mention that ticket. I just now opened the ticket, though, so it will be a while before it makes its way out of tier 1, etc.

      EDIT: Had the wrong case number at first. Updated case number.

      posted in Compute
      F
      fasterfourier