Nehalem cpu power management
-
@olivierlambert This cannot be overridden by a kernel boot parameter ? Because the reverse is possible.
-
Is this message coming from Xen or Dom0 kernel?
-
@olivierlambert Yes, xl dmesg:
[18:25 home-xcp ~]# xl dmesg |grep C- (XEN) [000000871c2971a8] Disable C3 and C6 C states on Nehalem processors due to error. (XEN) [ 2.652059] mwait-idle: max C state 1 reached (XEN) [ 2.698434] mwait-idle: max C state 1 reached -
-
I've looked through the xen documentation/forum, tried the parmeters, but nothing helped.
I'm thinking of putting an old 6.x xenserver on, because it doesn't have the higher c-state disabled
-
Maybe @andyhhp knows which parameter to use
-
@andyhhp Can you help me ?
-
Hello @bogikornel
The
Intel_errata_workaroundsandprobe_c3_errataroutines are executed unconditionally (there's no boot parameter to disable these checks)The only way through would be to recompile your Xen. I'm not sure that this is a good idea because Intel documentation talks about Unpredictable System Behaviour.
If that is really what you need, I can help you with this (or do this for you), so let me know...
-
@andSmv thanks for the reply
The server is a home test, power consumption is more important than stability, so I'll try to recompile xen.I guess it will be enough to recompile only version 4.13.4 ?PS.: I just looked at the source code and I don't see that there is a switch to turn it off. I will modify mwait-idle.c to see if it works
PS2: Done :
[09:50 home-xcp ~]# xenpm get-cpuidle-states 0 Max possible C-state: C9 cpu id : 0 total C-states : 5 idle time(ms) : 134037 C0 : transition [ 5351] residency [ 6507 ms] C1 : transition [ 13] residency [ 15 ms] C2 : transition [ 71] residency [ 120 ms] C3 : transition [ 701] residency [ 6956 ms] C4 : transition [ 4566] residency [ 120437 ms] pc3 : [ 13371 ms] pc6 : [ 23594 ms] cc3 : [ 14446 ms] cc6 : [ 63719 ms]The intel.c had to be modified to be correct. Let's see how stable it will be.
-
Interesting
Keep us posted! -
@olivierlambert
In the original xen 4.13.4 source code I modified 2 files, one is mwait-idle.c :--- mwait-idle.c.orig 2023-01-14 17:30:29.635655771 +0100 +++ mwait-idle.c 2023-01-14 09:06:04.274001499 +0100 @@ -864,8 +864,6 @@ static const struct idle_cpu idle_cpu_nehalem = { .state_table = nehalem_cstates, - .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE, - .disable_promotion_to_c1e = 1, }; static const struct idle_cpu idle_cpu_atom = {the other is intel.c
--- intel.c.orig 2023-01-14 09:42:27.751861245 +0100 +++ intel.c 2023-01-14 09:43:05.036530787 +0100 @@ -326,8 +326,8 @@ x86_match_cpu(models) ) { printk(XENLOG_WARNING - "Disabling C-states C3 and C6 due to CPU errata\n"); - max_cstate = 1; + "Nehalem errata deleted ...\n"); + max_cstate = 9; } }so cstate is not restricted, but many patches are missing that xcp-ng provides. At the moment windows VMs won't even start.
I am now trying to build a xen with patches by xcp-ng, although if someone could help me with the above modifications, I would appreciate it.
PS: Rebuilt the xen-hypervisor-4.13.4-10.36.xcpng8.3.x86_64 package with the above modification, and removed the detect-nehalem-c-state.patch
-
This is unconditional for a reason. The CSTATE errata in Nehalem are crippling - IIRC a core going in an out of a deep C state does not maintain cache coherency correctly, resulting in arbitrary memory corruption.
You really do care about not hitting this errata, even on a test/hobby server.
