Non-server CPU compatibility - Ryzen and Intel

olivierlambert

IIRC, boost is the real of Xen, not the Dom0 kernel (because Xen deals with memory and CPU, not the Dom0, which is a VM). Sometimes the frontier is blurry and it's not trivial to know (or decide when you dev it) who should handle what. I'll ask around to be sure about boost responsibility (even if I'm pretty sure it's Xen)

gecant

@olivierlambert said in Non-server CPU compatibility - Ryzen and Intel:

I'll ask around to be sure about boost responsibility (even if I'm pretty sure it's Xen)

Thank you!

I guess the amd_pstate driver is not backported on 4.19 kernel that XCP-ng uses, right?

If anyone with a Zen4 CPU can check the CPU frequencies that VMs are able to reach by default, I guess this would be useful.

redakula

@gecant said in Non-server CPU compatibility - Ryzen and Intel:

If anyone with a Zen4 CPU can check the CPU frequencies that VMs are able to reach by default, I guess this would be useful.

I have a Ryzen 9 7900 which reports this:

[14:50 xenserver ~]# xenpm get-cpufreq-para 0
cpu id               : 0
affected_cpus        : 0
cpuinfo frequency    : max [3700000] min [3000000] cur [3000000]
scaling_driver       : powernow
scaling_avail_gov    : userspace performance powersave ondemand
current_governor     : ondemand
  ondemand specific  :
    sampling_rate    : max [10000000] min [10000] cur [20000]
    up_threshold     : 80
scaling_avail_freq   : 3700000 *3000000
scaling frequency    : max [3700000] min [3000000] cur [3000000]
turbo mode           : enabled

But i might have disabled boost in BIOS for power saving - i can check later

Also very interested in details on the support of newer Ryzens. From what i understand power management for new zen architectures is fairly recent in the kernel so i do wonder how well supported these cpu's are currently.
From xenpm output it does not appear to scale below 3GHz - my previous intel cpu would scale to 800MHz.

gecant

@redakula Indeed, low power consuption on "amd_pstate" driver allows much lower frequencies down to 400 MHz.

Can you also please try to see the output of command xenpm start 1|grep "Avg freq" under some CPU load?
This allows to see the CPU frequency regardless of the available scaling frequencies.

For example, you can run in one VM stress-ng -c 4 to have 4 CPUs at full load and while this is running see the output of xenpm start 1|grep "Avg freq" on your Dom0 to see the CPU frequencies achieved under that stress.

Thank you.

redakula

@gecant said in Non-server CPU compatibility - Ryzen and Intel:

@redakula Indeed, low power consuption on "amd_pstate" driver allows much lower frequencies down to 400 MHz.

That is what i suspected from the p-state driver being heavily developed in kernel 6.5 and 6.9... Hope we will get support soon in XCP-NG.
The 7900 is ridiculously overkill for most of my homelab use so each watt saved helps

Can you also please try to see the output of command xenpm start 1|grep "Avg freq" under some CPU load?
This allows to see the CPU frequency regardless of the available scaling frequencies.

For example, you can run in one VM stress-ng -c 4 to have 4 CPUs at full load and while this is running see the output of xenpm start 1|grep "Avg freq" on your Dom0 to see the CPU frequencies achieved under that stress.

Thank you.

Seems to scale above base frequency (3,7GHz for the 7900) just fine - but does not go below 3GHz. 5GHz is pretty close to the max boost of 5,4GHz.
System is running 8.3 Beta with xen 4.17.
This is running stress on a few cores:

[17:25 xenserver ~]# xenpm start 1|grep "Avg freq"
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      2923000 KHz
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      2960000 KHz
  Avg freq      3922000 KHz
  Avg freq      3108000 KHz
  Avg freq      5069000 KHz
  Avg freq      5069000 KHz
  Avg freq      5069000 KHz
  Avg freq      5069000 KHz
  Avg freq      2886000 KHz
  Avg freq      2886000 KHz
  Avg freq      5069000 KHz
  Avg freq      5069000 KHz
  Avg freq      2923000 KHz
  Avg freq      3034000 KHz
  Avg freq      5069000 KHz
  Avg freq      5069000 KHz

gecant

@redakula By looking at your xenpm start 1|grep "Avg freq" output, this 5 GHz seems normal for your Ryzen 9 7900, because I can see at least 8 CPU threads already running on high frequency.
To get to 5.4 GHz you need less CPUs on high frequencies, you see the more CPUs are boosted at the same time, the lower the max frequency they can reach. For about 8-9 CPUs already boosted, 5 GHz seems reasonable.

So from your output I believe your CPUs are reaching the boosted/high frequencies without problem.

Thank you for your tests here on this.

redakula

@gecant

Yep - i just tried stressing a single core and got to 5.4GHz.

But would be nice to get power management also for these cpu's as they are pretty awesome

tmservo433

Largely depends on your homelab. I'm currently in my homelab using a 3970X Threadripper But, I got that board/chip free after we did a client upgrade to the new Threadrippers, and there isn't a single amount of money I could save on energy to match that savings.

That said, think about what you need to use your homelab configuration for; I often find that the availability of RAM and total available cores. But I guess different needs matter too..

lovvel

I'm picking up on your discussion again to ask another question about the 7950X3D with 16 CPU cores:

If I'm reading this correctly, this is an “asymmetric” CPU with 8 cores featuring 3D V-Cache technology and 8 standard cores for higher clock speeds. How does this work with dynamic allocation in XEN? Do you have any experience with this? Or do you assign domUs to specific cores? But if so, how do I know which cores are the performance cores?

That's a lot of questions, but basically, I'm just not entirely sure because XEN usually uses symmetric CPUs where all cores and threads deliver the same performance.

olivierlambert

It's entirely transparent/invisible, nothing specific is done, so you can't know which kind of vCPU it uses.

andyhhp

Xen has no awareness of 3D V-Cache. All 16 cores will be considered equal. Your vCPU may be on a 3D V-Cache core one millisecond, then on a no-3D V-Cache core the next.

If you really want to alter this, you can pin your VM to one group of cores or the other.

However, do not make the mistake of thinking of some of these cores as "performance cores" while the others not. The ones with 3D V-Cache will outperform the others on a wide variety of workloads despite not being able to turbo to the same degree.

lovvel

Hey @olivierlambert and @andyhhp , thanks for your quick reply. That means it's not a problem at all. But we can't use all of the CPU's performance features either. However, these would only be used for specific computing operations (e.g., games) and would have little to no relevance for standard server applications, right?