[02:27 xen19 ~]# uname -a
Linux xen19 4.19.142 #1 SMP Tue Nov 3 11:27:36 CET 2020 x86_64 x86_64 x86_64 GNU/Linux
[02:30 xen19 ~]# yum list installed | grep kernel
kernel.x86_64 4.19.19-7.0.9.1.xcpng8.2 @xcp-ng-updates
kernel-alt.x86_64 4.19.142-1.xcpng8.2 @xcp-ng-base
memory graph so far:
looking very good!!!
@r1 our complete setup:
[FreeNAS NFS] <----shared-storage----> [Pool of 2 servers (xen22 + xen23)] ----XAPI---> [XO from sources] -----remote----> [FreeNAS NFS]
All [servers] are real hardware servers, no VMs involved.
Same chain of servers for xen19, execpt there are more pool members (and VMs).
@stormi I'm not complete sure if I did the iso upgrade or not... 
But it's a good idea to reinstall the poolmaster from scratch...
Good news from the kernel-alt (server xen19): No RAM leaks so far 
@stormi on server xen19 I think so, on server xen22 I'm not sure
I looked more close on my memory graphs and saw, that the memory baseline increases every night:
"bump" every day:
closer look in week 53:
Dez 31. - Jan 01.
Our Backups run from 18.00 till 3 or 4 in the morning (including coalesce).
--> maybe the heavy IO load leads to memory leaks "somewhere"?
@r1 yes, ever line "installed" in yum.log is an Upgrade from XCP-ng.
Problems started with XCP-ng 8.x
@r1 in general we do not restart many of our VMs, its all very static, only manual operated
xen19 is now rebootet (we need it in production) with kernel-alt - highest id is currently 4
xen22 (pool master of another affected pool) - highest id is curently 30
memory graphs of xen22
yum.log of xen22 (Problem here also after installing kernel-4.19.19-6.0.10.1.xcpng8.1.x86_64)
yum.log.5.gz:Dec 19 00:52:47 Updated: kernel-4.4.52-4.0.12.x86_6
yum.log.3.gz:Nov 08 10:07:40 Updated: kernel-4.4.52-4.0.13.x86_64
yum.log.1:Apr 10 20:31:01 Installed: kernel-4.19.19-6.0.10.1.xcpng8.1.x86_64
yum.log.1:Aug 31 23:10:50 Updated: kernel-4.19.19-6.0.11.1.xcpng8.1.x86_64
yum.log.1:Dec 11 18:00:54 Updated: kernel-4.19.19-6.0.12.1.xcpng8.1.x86_64
yum.log.1:Dec 19 12:52:00 Updated: kernel-4.19.19-6.0.13.1.xcpng8.1.x86_64
yum.log.1:Dec 19 12:54:13 Updated: kernel-4.19.19-7.0.9.1.xcpng8.2.x86_64
looked in my yum.log on this server (xen19):
our problems startet exactly since "Apr 10 18:10:29 Installed: kernel-4.19.19-6.0.10.1.xcpng8.1.x86_64"
yum.log.4.gz:Oct 03 17:35:54 Installed: kernel-4.4.52-4.0.7.1.x86_64
yum.log.4.gz:Nov 20 18:29:29 Updated: kernel-4.4.52-4.0.12.x86_64
yum.log.2.gz:Oct 10 20:19:31 Updated: kernel-4.4.52-4.0.13.x86_64
yum.log.1:Apr 10 18:10:29 Installed: kernel-4.19.19-6.0.10.1.xcpng8.1.x86_64
yum.log.1:Jul 07 17:46:34 Updated: kernel-4.19.19-6.0.11.1.xcpng8.1.x86_64
yum.log.1:Dec 10 17:59:07 Updated: kernel-4.19.19-6.0.12.1.xcpng8.1.x86_64
yum.log.1:Dec 19 13:53:39 Updated: kernel-4.19.19-6.0.13.1.xcpng8.1.x86_64
yum.log.1:Dec 19 13:55:20 Updated: kernel-4.19.19-7.0.9.1.xcpng8.2.x86_64
yum.log:Jan 18 17:35:07 Installed: kernel-alt-4.19.142-1.xcpng8.2.x86_64
I noticed in my monitoring graphs, that since we have this issue, the SWAP is not used like before the issue:
We also have an issue with growing control domain memory:
Today I install the alternate kernel on one of our poolmaster to see if that resolves our issue.