[Information] amd-pstate v4 disabled on non-APUs, needs kernel parameter to force-enable about community-patches

[ptr1337]

amd_pstate.shared_mem=1 is the correct one

[notpentadactyl]

I have the same issue (also on 5950x), it did work on v3.

[AdelKS]

I thought APUs were affected, not ours. I may have been wrong, but if I am right something new needs to be done

[AdelKS]

Okay I have some new information from here:

Full MSR Support

Some new Zen3 processors such as Cezanne provide the MSR registers directly
while the :c:macro:X86_FEATURE_CPPC CPU feature flag is set.
amd-pstate can handle the MSR register to implement the fast switch
function in CPUFreq that can shrink latency of frequency control on the
interrupt context.

Shared Memory Support

If :c:macro:X86_FEATURE_CPPC CPU feature flag is not set, that means the
processor supports shared memory solution. In this case, amd-pstate
uses the cppc_acpi helper methods to implement the callback functions
of amd_pstate_perf_funcs.

AMD P-States and ACPI hardware P-States always can be supported in one
processor. But AMD P-States has the higher priority and if it is enabled
with :c:macro:MSR_AMD_CPPC_ENABLE or cppc_set_enable, it will respond
to the request from AMD P-States.

Given that Cezanne is the code name for Ryzen 5000 APUs, I was wrong: support is buggy for our CPUs and got disabled by default. Now if one wants to enable it nonetheless one needs to set that shared_mem parameter. I think it can be set at boot time, aka adding shared_mem=1 to the boot parameters ? Lets see.

[ptr1337]

/sys/devices/system/cpu/cpufreq/policy1
❯ ls
affected_cpus                     amd_pstate_max_freq  cpuinfo_transition_latency   scaling_cur_freq  scaling_max_freq
amd_pstate_highest_perf           cpuinfo_max_freq     related_cpus                 scaling_driver    scaling_min_freq
amd_pstate_lowest_nonlinear_freq  cpuinfo_min_freq     scaling_available_governors  scaling_governor  scaling_setspeed

youve got the same entry's now ?

[AdelKS]

Yep it looks like it

➜  ~ cd /sys/devices/system/cpu/cpufreq/policy1
➜  policy1 ls
affected_cpus                     cpuinfo_min_freq             scaling_driver
amd_pstate_highest_perf           cpuinfo_transition_latency   scaling_governor
amd_pstate_lowest_nonlinear_freq  related_cpus                 scaling_max_freq
amd_pstate_max_freq               scaling_available_governors  scaling_min_freq
cpuinfo_max_freq                  scaling_cur_freq             scaling_setspeed

But it looks like it's not a complete support

➜  ~ cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.08 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.08 GHz.
                  The governor "schedutil" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 2.84 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no

Because the example from here gives this

root@hr-test1:/home/ray# cpupower frequency-info
analyzing CPU 0:
   driver: amd-pstate
   CPUs which run at the same hardware frequency: 0
   CPUs which need to have their frequency coordinated by software: 0
   maximum transition latency: 131 us
   hardware limits: 400 MHz - 4.68 GHz
   available cpufreq governors: ondemand conservative powersave userspace performance schedutil
   current policy: frequency should be within 400 MHz and 4.68 GHz.
                   The governor "schedutil" may decide which speed to use
                   within this range.
   current CPU frequency: Unable to call hardware
   current CPU frequency: 4.02 GHz (asserted by call to kernel)
   boost state support:
     Supported: yes
     Active: yes
     AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.68 GHz.
     AMD PSTATE Nominal Performance: 117. Nominal Frequency: 3.30 GHz.
     AMD PSTATE Lowest Non-linear Performance: 39. Lowest Non-linear Frequency: 1.10 GHz.
     AMD PSTATE Lowest Performance: 15. Lowest Frequency: 400 MHz.

[ptr1337]

at me right now after changing the scheduler

❯ cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.07 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.07 GHz.
                  The governor "ondemand" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 2.92 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes
    Boost States: 0
    Total States: 3
    Pstate-P0:  3700MHz
    Pstate-P1:  2800MHz
    Pstate-P2:  2200MHz

[notpentadactyl]

at me right now after changing the scheduler

No changes for me, same observation as @AdelKS.
Switching between schedulers doesn't seem to matter on the 5950x:

cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.08 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.08 GHz.
                  The governor "schedutil" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 3.53 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no

cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.08 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.08 GHz.
                  The governor "ondemand" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 3.92 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no

[ptr1337]

at me right now after changing the scheduler

No changes for me, same observation as @AdelKS. Switching between schedulers doesn't seem to matter on the 5950x:

cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.08 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.08 GHz.
                  The governor "schedutil" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 3.53 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no

cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.08 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.08 GHz.
                  The governor "ondemand" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 3.92 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no

Ive got a 5900x so should be no difference

[AdelKS]

Probably a BIOS entry needs to be changed. And this shows why the driver got disabled for now.

[ptr1337]

Probably a BIOS entry needs to be changed. And this shows why the driver got disabled for now.

Yeah i have CPPC and CPPC-PREFERED-CORES enabled

[AdelKS]

Yea I have that enabled too. For example enabling x2apic makes frequency reporting buggy.

[notpentadactyl]

I have now enabled both CPPC and CPPC-PREFERED-CORES. No change.

[ptr1337]

Kernel Config also like this ?

# CPU frequency scaling drivers
#
CONFIG_X86_INTEL_PSTATE=y
CONFIG_X86_PCC_CPUFREQ=m
CONFIG_X86_AMD_PSTATE=y
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_ACPI_CPUFREQ_CPB=y
CONFIG_X86_POWERNOW_K8=m
CONFIG_X86_AMD_FREQ_SENSITIVITY=m
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
CONFIG_X86_P4_CLOCKMOD=m

[notpentadactyl]

Kernel Config also like this ?

# CPU frequency scaling drivers
#
CONFIG_X86_INTEL_PSTATE=y
CONFIG_X86_PCC_CPUFREQ=m
CONFIG_X86_AMD_PSTATE=y
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_ACPI_CPUFREQ_CPB=y
CONFIG_X86_POWERNOW_K8=m
CONFIG_X86_AMD_FREQ_SENSITIVITY=m
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
CONFIG_X86_P4_CLOCKMOD=m

Not quite:

CONFIG_X86_INTEL_PSTATE=y
# CONFIG_X86_PCC_CPUFREQ is not set
CONFIG_X86_AMD_PSTATE=y
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_ACPI_CPUFREQ_CPB=y
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_AMD_FREQ_SENSITIVITY is not set
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_P4_CLOCKMOD is not set

[ptr1337]

i removed the cmdline entry and set cppc to auto, calling cpupower as sudo gives other results of output.

/sys/devices/system/cpu/cpu1/cpufreq
❯ sudo cpupower frequency-info
analyzing CPU 0:
  driver: acpi-cpufreq
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 2.20 GHz - 5.16 GHz
  available frequency steps:  3.70 GHz, 2.80 GHz, 2.20 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 2.20 GHz and 3.70 GHz.
                  The governor "ondemand" may decide which speed to use
                  within this range.
  current CPU frequency: 3.70 GHz (asserted by call to hardware)
  boost state support:
    Supported: yes
    Active: yes
    Boost States: 0
    Total States: 3
    Pstate-P0:  3700MHz
    Pstate-P1:  2800MHz
    Pstate-P2:  2200MHz

/sys/devices/system/cpu/cpu1/cpufreq
❯ ls
affected_cpus  cpuinfo_cur_freq  cpuinfo_transition_latency  scaling_available_frequencies  scaling_driver    scaling_min_freq
bios_limit     cpuinfo_max_freq  freqdomain_cpus             scaling_available_governors    scaling_governor  scaling_setspeed
cpb            cpuinfo_min_freq  related_cpus                scaling_cur_freq               scaling_max_freq  stats

[AdelKS]

Sudo indeed gives the other output:

➜  ~ sudo cpupower frequency-info 
Password: 
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 550 MHz - 5.08 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 5.08 GHz.
                  The governor "schedutil" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 1.99 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes
    Boost States: 0
    Total States: 3
    Pstate-P0:  3400MHz
    Pstate-P1:  2800MHz
    Pstate-P2:  2200MHz

My config

CONFIG_X86_INTEL_PSTATE=y
CONFIG_X86_PCC_CPUFREQ=m
CONFIG_X86_AMD_PSTATE=y
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_ACPI_CPUFREQ_CPB=y
# CONFIG_X86_POWERNOW_K8 is not set
CONFIG_X86_AMD_FREQ_SENSITIVITY=m
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_P4_CLOCKMOD is not set
# CONFIG_X86_X32 is not set

[ptr1337]

@AdelKS
Are you still using the cmdline parameter ?

[AdelKS]

Yes I have amd_pstate.shared_mem=1 in the cmdline otherwise amd-pstate doesn't even get loaded.

[ptr1337]

Yes I have amd_pstate.shared_mem=1 in the cmdline otherwise amd-pstate doesn't even get loaded.

Yes, facing into same issue.

[AdelKS]

Yes, facing into same issue.

It's normal, v4 explicitly disables amd-cppc for our CPUs because it's buggy

[ptr1337]

Mostly has both driver the same output, only driver is "another". Weird.

[gardotd426]

Yes I have amd_pstate.shared_mem=1 in the cmdline otherwise amd-pstate doesn't even get loaded.

Yes, facing into same issue.

Yes, that's intended, it's not really an "issue." It's only explicitly enabled on some Epyc processors right now. It worked without the kernel parameter on earlier revisions but they received some reports of bugginess (including from me, I exchanged several emails with the amd kernel developers about it), so apparently they disabled it for the vast majority of Zen 2 and 3 Ryzen (non-Epyc) CPUs.

Which I think is a good thing, considering you might notice that if you use the kernel parameter to force amd-pstate, and use nmon to monitor your cpu frequencies while your CPU governor is set to performance, you'll get 5.16GHz as the reported frequency on all cores. Which is obviously not remotely accurate.

I'd say this issue should either be closed, or one of us just files a PR that adds

needs `amd_pstate.shared_mem=1` in your kernel parameters to enable support for most Ryzen CPUs

to the community patches linux-tkg README.md files.

[gardotd426]

Here's relevant info from the documentation concerning whether you need the kernel parameter or not:

The way to check whether current processor is `Full MSR Support <perf_cap_>`_
or `Shared Memory Support <perf_cap_>`_ : ::

  ray@hr-test1:~$ lscpu | grep cppc
  Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm

If CPU Flags have cppc, then this processor supports `Full MSR Support <perf_cap_>`_. Otherwise it supports `Shared Memory Support <perf_cap_>`_.

So if lscpu | grep cppc returns nothing, you need the kernel parameter to use the new driver. I'm working on maybe doing a PR for v6 that just came out, and I'll include a line in the README if I do.

[gardotd426]

I've filed the PR to update to v6, and added the note to the README so users can check if they need amd_pstate.shared_mem=1 in their kernel parameters.

Anyway, I'm using it right now with linux516-tkg-cfs-5.16.rc6 and it seems to work the same as before, I haven't checked any benchmarks though.

[ptr1337]

I've filed the PR to update to v6, and added the note to the README so users can check if they need amd_pstate.shared_mem=1 in their kernel parameters.

Anyway, I'm using it right now with linux516-tkg-cfs-5.16.rc6 and it seems to work the same as before, I haven't checked any benchmarks though.

You should also include the Changelogs into the patch.

You can check if the pstate cmdline is needed, you could just watch the dmesg output.

Due to some issues on shared memory solution, so we fallback to build as a
module and add a parameter "shared_mem" to disable AMD P-State on processors
with shared memory solution for the moment till we complete tunning on this
function.
Changelogs:

Changes from V4 -> V5:
- cpufreq:
- - Fix subject typo.
- - Fix Clang-13 build warning.
- - Update RST documentation for the latest update.
- cpupower:
- - Fix the table check condition at cpufreq_get_sysfs_value_from_table.

Changes from V5 -> V6:
- cpufreq:
- - Fix online cpu to present cpu.
- - Fix comment code style and use macro instead of raw value.
- - Move static call implementation into the patch of shared memory
  support.
- - Move freq_qos requests into patch of boost support.
- - Add brief driver and core data structure descriptions at the start.
- - Add maintainer entry.