Resource monitoring with perf-alert

[fred974]

Hi,

I just enable perf-alert on Xen Orchestra community edition and I have been flooded with CPU alert. I left all the plugin settings as default for now.

Here is the email

ALERT: host CPU usage > 40%
uk-dc1-prod-hv2: 50.2%
Description
Raises an alarm when the average usage of any CPU is higher/lower than the threshold

Sent from Xen Orchestra perf-alert plugin

When I look at the host I can see this:

The Host only has 1 VM running on it and I cannot see any spike in CPU

I have 2 questions:

1. why is the default warning threshold set to 40%? This seem very low to be but before I start poking, I'll like to know if the default is there for a reason

2. Why do I keep getting CPU warning when the VM isn't doing anything? What do I need to investigated here?

The Host is the primary in a cluster of 4x hosts.
XCP-ng 8.2.1
xo-server 5.116.3
xo-web 5.119.1

Thank you all in advance

[olivierlambert]

1. We had to choose a value if you left it empty. No particular reason outside the fact you might take a look if it's more than a very low number (idle machine).
2. If there's no CPU usage in the VM but only from the "host" perspective, check your Dom0.

[fred974]

@olivierlambert said in Resource monitoring with perf-alert:

check your Dom0.

I am right to say Dom0 is the hypervisor itself?
I just had another cpu spike on the host and when I looked at the host resources via htop, I cannot see any cpu spike at all.

Am I looking at the wrong place?

[olivierlambert]

Dom0 is the special VM that has access to all PCI physical devices (but only some CPUs and a part of the memory).

On the screenshot I can see your host is really outdated. So I would update/reboot and see the situation after that.

[fred974]

@olivierlambert said in Resource monitoring with perf-alert:

On the screenshot I can see your host is really outdated. So I would update/reboot and see the situation after that

Do you mean the patch or xcp-ng is outdated?

[fred974]

@olivierlambert said in Resource monitoring with perf-alert:

On the screenshot I can see your host is really outdated. So I would update/reboot and see the situation after that.

I tried to update the patches but I get this error:

pool.rollingUpdate
{
  "pool": "68d0a3f2-ff33-8711-f8b3-bcd85507881b"
}
{
  "code": "-1",
  "params": [
    "Command '['yum', 'update', '--disablerepo=*', '--enablerepo=xcp-ng-base,xcp-ng-updates', '-y']' returned non-zero exit status 1",
    "",
    "Traceback (most recent call last):
  File \"/etc/xapi.d/plugins/xcpngutils/__init__.py\", line 119, in wrapper
    return func(*args, **kwds)
  File \"/etc/xapi.d/plugins/updater.py\", line 96, in decorator
    return func(*args, **kwargs)
  File \"/etc/xapi.d/plugins/updater.py\", line 157, in update
    raise error
CalledProcessError: Command '['yum', 'update', '--disablerepo=*', '--enablerepo=xcp-ng-base,xcp-ng-updates', '-y']' returned non-zero exit status 1
"
  ],
  "call": {
    "method": "host.call_plugin",
    "params": [
      "OpaqueRef:b48630d1-f865-4315-9bac-21c94ad68369",
      "updater.py",
      "update",
      {}
    ]
  },
  "message": "-1(Command '['yum', 'update', '--disablerepo=*', '--enablerepo=xcp-ng-base,xcp-ng-updates', '-y']' returned non-zero exit status 1, , Traceback (most recent call last):
  File \"/etc/xapi.d/plugins/xcpngutils/__init__.py\", line 119, in wrapper
    return func(*args, **kwds)
  File \"/etc/xapi.d/plugins/updater.py\", line 96, in decorator
    return func(*args, **kwargs)
  File \"/etc/xapi.d/plugins/updater.py\", line 157, in update
    raise error
CalledProcessError: Command '['yum', 'update', '--disablerepo=*', '--enablerepo=xcp-ng-base,xcp-ng-updates', '-y']' returned non-zero exit status 1
)",
  "name": "XapiError",
  "stack": "XapiError: -1(Command '['yum', 'update', '--disablerepo=*', '--enablerepo=xcp-ng-base,xcp-ng-updates', '-y']' returned non-zero exit status 1, , Traceback (most recent call last):
  File \"/etc/xapi.d/plugins/xcpngutils/__init__.py\", line 119, in wrapper
    return func(*args, **kwds)
  File \"/etc/xapi.d/plugins/updater.py\", line 96, in decorator
    return func(*args, **kwargs)
  File \"/etc/xapi.d/plugins/updater.py\", line 157, in update
    raise error
CalledProcessError: Command '['yum', 'update', '--disablerepo=*', '--enablerepo=xcp-ng-base,xcp-ng-updates', '-y']' returned non-zero exit status 1
)
    at Function.wrap (/opt/xo/xo-builds/xen-orchestra-202306071622/packages/xen-api/src/_XapiError.js:16:12)
    at /opt/xo/xo-builds/xen-orchestra-202306071622/packages/xen-api/src/transports/json-rpc.js:35:21
    at runNextTicks (node:internal/process/task_queues:60:5)
    at processImmediate (node:internal/timers:447:9)
    at process.callbackTrampoline (node:internal/async_hooks:130:17)"
}

Could anyone please help me resolve this issue?

Thank you