Memory reporting incorrect values
-
@olivierlambert said in CPU reporting incorrect values:
Check carefully your htop output. See the huge yellow bar? That's the buffer/cache used
So Xen Orchestra see cached value as ram in use and not the actual active used ram?
-
It's not XO. It's what's reported by the tools.
-
@olivierlambert But all modern OS will use the RAM as cache for performance etc. So how can I ensure I get the correct info? At the moment I bombarded with email saying high RAM usage
-
@fred974 This is normal behavior, most modern OSes will use up as much RAM as they possibly can for caching, unused RAM is wasted RAM.
I'd adjust your alert settings if you're getting high RAM usage alerts. You also could change things in the OS to prevent as much RAM being used for caching, but I'd avoid that if possible.
The OS reports, via the tools, to the hypervisor about it's RAM usage, so if the OS is caching a lot, and therefore using most of it's RAM, it will report that to the hypervisor.
I think, could be wrong, Windows doesn't report cached RAM via the tools so it may show "real" RAM usage, but this is OS specific stuff and not really XCP-ng or any hypervisor specific stuff.
-
Adding @yann and @TeddyAstie in the convo (since we maybe need to report both, IDK)
-
It could be interesting to report all the information the guest can provide us, but there is no xenstore schema for that; and no support from xcp-rrdd either.
Aside that, I don't think xe-guest-utilities consider cache memory as used; but there may be corner cases.
What givescat /proc/meminfo -
@planedrop I'm using "citrix" tools for windows and I haven't noticed any of mine reporting any cached ram. Yet
-
@jebrown OK good to know. This may be different on different OSes though, Windows may just report actually used RAM and leave out anything that is cached.
-
Thank you all for your feedback on this matter. The OS used in this case ins Ubuntu 22.04.5 LTS.
@TeddyAstie said in Memory reporting incorrect values:
What gives cat /proc/meminfo
cat /proc/meminfo MemTotal: 8123972 kB MemFree: 440556 kB MemAvailable: 5419780 kB Buffers: 410852 kB Cached: 4761852 kB SwapCached: 328 kB Active: 2995088 kB Inactive: 3405912 kB Active(anon): 723800 kB Inactive(anon): 1399348 kB Active(file): 2271288 kB Inactive(file): 2006564 kB Unevictable: 27620 kB Mlocked: 27620 kB SwapTotal: 4194300 kB SwapFree: 4191228 kB Dirty: 1508 kB Writeback: 0 kB AnonPages: 1255676 kB Mapped: 939728 kB Shmem: 885788 kB KReclaimable: 995468 kB Slab: 1154076 kB SReclaimable: 995468 kB SUnreclaim: 158608 kB KernelStack: 4656 kB PageTables: 13848 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 8256284 kB Committed_AS: 5873420 kB VmallocTotal: 34359738367 kB VmallocUsed: 18896 kB VmallocChunk: 0 kB Percpu: 4192 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 307200 kB DirectMap2M: 8077312 kB DirectMap1G: 1048576 kB -
@fred974 the tools should be reporting the value from
MemAvailable, so buffer/cache should not be considered as in-use. What do you see (reported by the agent) in xenstore ?From within the guest:
# xenstore ls data meminfo_free = "324060" meminfo_total = "973736" os_distro = "Debian" os_name = "Debian 11" os_uname = "5.10.0-21-amd64" updated = "1"The
meminfo_freeshould match what you see in/proc/meminfo. -
@yann said in Memory reporting incorrect values:
xenstore ls data
Here is what I see:
cat /proc/meminfoMemTotal: 8123972 kB MemFree: 428388 kB MemAvailable: 5419696 kB Buffers: 411604 kB Cached: 4771524 kB SwapCached: 328 kB Active: 3060024 kB Inactive: 3339116 kB Active(anon): 736300 kB Inactive(anon): 1374628 kB Active(file): 2323724 kB Inactive(file): 1964488 kB Unevictable: 27620 kB Mlocked: 27620 kB SwapTotal: 4194300 kB SwapFree: 4191228 kB Dirty: 284 kB Writeback: 0 kB AnonPages: 1243360 kB Mapped: 919268 kB Shmem: 885852 kB KReclaimable: 997192 kB Slab: 1155308 kB SReclaimable: 997192 kB SUnreclaim: 158116 kB KernelStack: 4432 kB PageTables: 13128 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 8256284 kB Committed_AS: 5854320 kB VmallocTotal: 34359738367 kB VmallocUsed: 18656 kB VmallocChunk: 0 kB Percpu: 4192 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 307200 kB DirectMap2M: 8077312 kB DirectMap1G: 1048576 kBxenstore ls datameminfo_free = "422892" meminfo_total = "8123972" os_distro = "ubuntu" os_majorver = "22" os_minorver = "04" os_name = "Ubuntu 22.04.5 LTS" os_uname = "5.15.0-128-generic" updated = "Mon Jan 6 11:52:45 2025" volumes = "" 0 = "" extents = "" 0 = "xvda" filesystem = "ext4" free = "142059319296" mount_points = "" 0 = "/" name = "/dev/xvda1(072768fc-9db5-4fa4-a6a1-c98afdd82d32)" size = "171682283008" 1 = "" extents = "" 0 = "xvda" free = "4138934272" name = "/dev/xvda14" size = "4194304" 2 = "" extents = "" 0 = "xvda" filesystem = "vfat" free = "103057408" mount_points = "" 0 = "/boot/efi" name = "/dev/xvda15(D9A6-D2AA)" size = "111149056" 3 = "" extents = "" 0 = "xvdb" free = "4138934272" name = "/dev/xvdb1(355A-4FC2)" size = "10486272"htop
XOA
XO Community
-
It would be great to have a bit for details on the memory so we can monitor the system better.
-
@yann said in Memory reporting incorrect values:
@fred974 the tools should be reporting the value from MemAvailable, so buffer/cache should not be considered as in-use.
Doesn't seem to be the case
-
@fred974 right, it is reporting
MemFreehere.What does
xenstore ls attr/PVAddonsshow for this VM? And which kernel version is it using?Old
xe-guest-utilitiesbefore v7.30 indeed reportedMemFree, which XenServer then improved in v8.4 to still report something for kernels older than 3.14 which did not haveMemAvailableyet, but Ubuntu 22.04 ships a decently-recent kernel, so I suspect it would be rather due to an oldxe-guest-utilities. -
@yann said in Memory reporting incorrect values:
What does xenstore ls attr/PVAddons show for this VM? And which kernel version is it using?
BuildVersion = "0" Installed = "1" MajorVersion = "6" MicroVersion = "80" MinorVersion = "6"hostnamectl statusStatic hostname: WEB1-OLS-QHUK Transient hostname: web1-ols-qh-uk Icon name: computer-vm Chassis: vm Machine ID: 3b1da9b8e5cf4ecda845219bxxx Boot ID: 19308dbfd7d347c699985e82xxx Virtualization: xen Operating System: Ubuntu 22.04.5 LTS Kernel: Linux 5.15.0-128-generic Architecture: x86-64 Hardware Vendor: Xen Hardware Model: HVM domU -
@fred974 v6.80.6 is definitely very old, and did report
MemFree, we ship v7.30 with XCP-ng 8.2. You will likely want to update it, from the "XCP-ng tools" SR. -
@yann said in Memory reporting incorrect values:
@fred974 v6.80.6 is definitely very old, and did report MemFree, we ship v7.30 with XCP-ng 8.2. You will likely want to update it, from the "XCP-ng tools" SR.
Probably because I installed xe-guest-utilities from ububtu repo.. I tried to update from the CD and got the following error:
bash /mnt/Linux/install.shDetected `Ubuntu 22.04.5 LTS' (ubuntu version 22). The following changes will be made to this Virtual Machine: * update arp_notify sysctl. * packages to be installed/upgraded: - xe-guest-utilities_7.30.0-12_amd64.deb Continue? [y/n] y (Reading database ... 136958 files and directories currently installed.) Preparing to unpack .../xe-guest-utilities_7.30.0-12_amd64.deb ... Unpacking xe-guest-utilities (7.30.0-12) over (7.20.2-0ubuntu1~22.04.2) ... dpkg: error processing archive /mnt/Linux/xe-guest-utilities_7.30.0-12_amd64.deb (--install): trying to overwrite '/usr/bin/xenstore', which is also in package xenstore-utils 4.16.0-1~ubuntu2.1 dpkg-deb: error: paste subprocess was killed by signal (Broken pipe) Errors were encountered while processing: /mnt/Linux/xe-guest-utilities_7.30.0-12_amd64.debCould anyone advice of how to fix this issue? I tried apt remove xe-guest-utilities and got the same error
-
@fred974 there is an undeclared conflict between the
xe-guest-utilitiesin the ISO andxenstore-utils, you can install the former after removing the latter (but yes, it is a problem, the Go implementation ofxenstoretools should not be in this package). -
@yann I removed the tools using
apt remove xe-guest-utilitiesand tried to install from the cd and had the same error
apt remove xe-guest-utilitiesReading package lists... Done Building dependency tree... Done Reading state information... Done The following packages were automatically installed and are no longer required: dbconfig-common dbconfig-mysql icc-profiles-free libaprutil1-dbd-sqlite3 libaprutil1-ldap libjs-bootstrap4 libjs-codemirror libjs-jquery-mousewheel libjs-jquery-timepicker libjs-jquery-ui libjs-popper.js libjs-sizzle liblua5.3-0 libmcrypt4 node-jquery php-common php-google-recaptcha php-mariadb-mysql-kbs php-nikic-fast-route php-phpmyadmin-motranslator php-phpmyadmin-shapefile php-phpseclib php-psr-cache php-psr-container php-psr-log php-symfony-cache php-symfony-cache-contracts php-symfony-dependency-injection php-symfony-deprecation-contracts php-symfony-expression-language php-symfony-polyfill-php80 php-symfony-polyfill-php81 php-symfony-service-contracts php-symfony-var-exporter php-tcpdf ssl-cert Use 'apt autoremove' to remove them. The following packages will be REMOVED: xe-guest-utilities 0 upgraded, 0 newly installed, 1 to remove and 16 not upgraded. After this operation, 1889 kB disk space will be freed. N: Ignoring file 'mariadb.list.old_1' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension Do you want to continue? [Y/n] y (Reading database ... 136958 files and directories currently installed.) Removing xe-guest-utilities (7.20.2-0ubuntu1~22.04.2) ... needrestart is being skipped since dpkg has failed root@WEB1-OLS-QHUK:/home/fetheve/Linux# apt autoremove Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: dbconfig-common dbconfig-mysql icc-profiles-free libaprutil1-dbd-sqlite3 libaprutil1-ldap libjs-bootstrap4 libjs-codemirror libjs-jquery-mousewheel libjs-jquery-timepicker libjs-jquery-ui libjs-popper.js libjs-sizzle liblua5.3-0 libmcrypt4 node-jquery php-common php-google-recaptcha php-mariadb-mysql-kbs php-nikic-fast-route php-phpmyadmin-motranslator php-phpmyadmin-shapefile php-phpseclib php-psr-cache php-psr-container php-psr-log php-symfony-cache php-symfony-cache-contracts php-symfony-dependency-injection php-symfony-deprecation-contracts php-symfony-expression-language php-symfony-polyfill-php80 php-symfony-polyfill-php81 php-symfony-service-contracts php-symfony-var-exporter php-tcpdf ssl-cert 0 upgraded, 0 newly installed, 36 to remove and 15 not upgraded. After this operation, 42.0 MB disk space will be freed. N: Ignoring file 'mariadb.list.old_1' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension Do you want to continue? [Y/n] y (Reading database ... 136950 files and directories currently installed.) Removing dbconfig-mysql (2.0.21) ... Removing dbconfig-common (2.0.21) ... Removing php-tcpdf (6.4.4+dfsg1-1) ... Removing icc-profiles-free (2.0.1+dfsg-1.1) ... Removing libaprutil1-dbd-sqlite3:amd64 (1.6.1-5ubuntu4.22.04.2) ... Removing libaprutil1-ldap:amd64 (1.6.1-5ubuntu4.22.04.2) ... Removing libjs-bootstrap4 (4.6.0+dfsg1-4) ... Removing libjs-codemirror (5.65.0+~cs5.83.9-1) ... Removing libjs-jquery-mousewheel (1:3.1.13-5) ... Removing libjs-jquery-timepicker (1.6.3-1) ... Removing libjs-jquery-ui (1.13.1+dfsg-1) ... Removing libjs-popper.js (1.16.1+ds-5) ... Removing libjs-sizzle (2.3.6+ds+~2.3.3-1) ... Removing liblua5.3-0:amd64 (5.3.6-1build1) ... Removing libmcrypt4 (2.5.8-7) ... Removing node-jquery (3.6.0+dfsg+~3.5.13-1) ... Removing php-phpmyadmin-motranslator (5.2.0-1build1) ... Removing php-symfony-expression-language (5.4.4+dfsg-1ubuntu8) ... Removing php-symfony-cache (5.4.4+dfsg-1ubuntu8) ... Removing php-symfony-var-exporter (5.4.4+dfsg-1ubuntu8) ... Removing php-symfony-dependency-injection (5.4.4+dfsg-1ubuntu8) ... Removing php-symfony-service-contracts (2.4.0-1ubuntu2) ... Removing php-google-recaptcha (1.2.4-3) ... Removing php-mariadb-mysql-kbs (1.2.12-1) ... Removing php-nikic-fast-route (1.3.0-3) ... Removing php-phpmyadmin-shapefile (2.1-5) ... Removing php-phpseclib (2.0.36-1) ... Removing php-symfony-cache-contracts (2.4.0-1ubuntu2) ... Removing php-psr-cache (3.0.0-1) ... Removing php-psr-container (2.0.2-1) ... Removing php-psr-log (3.0.0-1) ... Removing php-symfony-deprecation-contracts (2.4.0-1ubuntu2) ... Removing php-symfony-polyfill-php80 (1.24.0-1ubuntu2) ... Removing php-symfony-polyfill-php81 (1.24.0-1ubuntu2) ... Removing ssl-cert (1.1.2) ... Removing php-common (2:92ubuntu1) ... Warning: Stopping phpsessionclean.service, but it can still be activated by: phpsessionclean.timer Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.8) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host.bash /mnt/Linux/install.shDetected `Ubuntu 22.04.5 LTS' (ubuntu version 22). The following changes will be made to this Virtual Machine: * update arp_notify sysctl. * packages to be installed/upgraded: - xe-guest-utilities_7.30.0-12_amd64.deb Continue? [y/n] y Selecting previously unselected package xe-guest-utilities. (Reading database ... 134088 files and directories currently installed.) Preparing to unpack .../xe-guest-utilities_7.30.0-12_amd64.deb ... Unpacking xe-guest-utilities (7.30.0-12) ... dpkg: error processing archive /mnt/Linux/xe-guest-utilities_7.30.0-12_amd64.deb (--install): trying to overwrite '/usr/bin/xenstore', which is also in package xenstore-utils 4.16.0-1~ubuntu2.1 dpkg-deb: error: paste subprocess was killed by signal (Broken pipe) Errors were encountered while processing: /mnt/Linux/xe-guest-utilities_7.30.0-12_amd64.debHow can I fix this?
-
@fred974 said in Memory reporting incorrect values:
xenstore-utils
apt remove xenstore-utilsfix the installation issue.
