Get the number of CPUs in Rust
License: Apache License 2.0
Count the number of CPUs on the current machine.
Add to Cargo.toml:
[dependencies]
num_cpus = "1.0"In your main.rs or lib.rs:
extern crate num_cpus;
// count logical cores this process could try to use
let num = num_cpus::get();
num_cpus 1.6.1 broke semver on Android as _SC_NPROCESSORS_CONF doesn't exist on those platforms.
You can see it missing here: https://github.com/rust-lang/libc/blob/master/src/unix/notbsd/android/mod.rs#L257
So #45 needs to be partially reverted and 1.6.1 possibly yanked.
rustix provides a safe interface over some libc calls. In addition it provides a direct interface over Linux system calls, which can significantly reduce the instruction count in many cases.
rustix doesn't support some of the system calls that this crate needs. If there is interest I can make PRs to add them to rustix.
Now that Haiku support got merged (in #76), it would be nice for the ecosystem to use it.
A patch release seems appropriate for this, as that PR only improves things for this one platform, and doesn’t change the API or ABI in any way.
Glancing over the code I noticed that get and get_physical always returns at least 1, this isn't documented anywhere as far as I looked.
My proposition would be returning NonZeroUsize instead of usize.
But I believe at least this should be documented.
There should be example code in the Rustdoc for num_cpus::get
Or at least prominently point people towards std? https://doc.rust-lang.org/stable/std/thread/fn.available_parallelism.html
Currently num_cpus doesn't support cgroupv2, maybe we should support it?
Spec: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html
K8s ref: https://kubernetes.io/docs/concepts/architecture/cgroups/
While num_cpus() gives the correct number of physical CPUs, it does not take into account which of those CPU's the current process is actually allowed to use. It would be neat to be able to specify which of the two numbers to get.
For example, consider a simple num_cpus program that just prints the value of num_cpus::get();, and then exits:
$ nproc
80
$ num_cpus
80
$ taskset -c 0,1 nproc
2
$ taskset -c 0,1 num_cpus
80
If it's at all helpful, the source for nproc (which is included in coreutils) can be seen here.
Unfortunately, cpu_set_t is not enough for very large CPU maks.
Would you be open to android support?
A simple wrap for num_cpus and lstopo
pub fn get_logical_processor_number() -> i32 { num_cpus::get() as i32 }physical core numbers number: 8 (lstopo) logical core number: 13(num_cpus)
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
Stepping: 4
CPU MHz: 3374.405
BogoMIPS: 5999.98
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 256 KiB
L1i cache: 256 KiB
L2 cache: 8 MiB
L3 cache: 24.8 MiB
NUMA node0 CPU(s): 0-15
Vulnerability Itlb multihit: KVM: Vulnerable
Vulnerability L1tf: Mitigation; PTE Inversion
Vulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT
Host state unknown
Vulnerability Meltdown: Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sa
nitization
Vulnerability Spectre v2: Mitigation; Full generic retpoline, STIBP disabled, RSB fi
lling
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Vulnerable: Clear CPU buffers attempted, no microcode; SMT
Host state unknown
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca c
mov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx p
dpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nons
top_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse
3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_dead
line_timer aes xsave avx f16c rdrand hypervisor lahf_lm ab
m 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi
1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq
rdseed adx smap clflushopt clwb avx512cd avx512bw avx512v
l xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke
Current documentation for get_physical() is quite limited and should probably be expanded. Explaining how physical cores are counted on different architectures may be useful.
Discussion of architecture limitations could be expanded to list architectures the physical count is not available on & why.
I'm working with an intel CPU that has 4 cores and 8 hyperthreads, but the hyperthreads have been disabled in the bios (for various application-specific reasons). In this configuration num_cpus::get() still returns 8.
Is this intentional behavior or a consequence of my weird setup? Ideally it should return 4 in this case since there are only 4 parallel threads available.
Thanks in advance!
I've checked that you only have one test case, to test it's a positive integer so I am not surprised that it didn't catch the broken get().
We need to add a test to make sure it's not a unrealistic number of cores. The highest number of cores to date is 32,000 cores (TH2 until Cray's new server comes out). I would recommend to set the upper bound as 236,450 => ceiling(e^2 * 32000)
It would be very nice if it was possible to ask this library if HT is present or not.
Often when doing computationally intensive work in thread pools and similar using the full number of threads is not desired (at least not in my experience).
Having an X-core cpu with HT will make this library return 2X. Not very strange, this is how it usually works. The downside is that doing heavy computation on 2X threads gives very little performance boost compared to X threads and has the downside that it slows down the computer very much, specially if the same computer run anything with a GUI.
Would this be possible to query from the OS? Maybe I can have a look at this myself, but I don't have time now so I leave this here.
Are there tasks to do (issues, PR) before making a new release ? If so, how can I help ?
OS: Arch Linux 5.3.7
CPU: AMD Ryzen Threadripper 2970WX 24-Core Processor
It seems we're trying to count by core id, but that just marks the die that holds the cores. In this machine there are 6 dies with 4 cores each, and 2 threads per core. get_physical() returns 6, and when I checked the code it groups by core id. I also see a column cup cores that gives the right number 24 and could be used directly.
When compiling with the newest nightly compiler and cargo build --target=asmjs-unknown-emscripten, this happens:
Compiling num_cpus v1.1.0 (file:///path/to/num_cpus)
error[E0425]: unresolved name `get_num_cpus`
--> src/lib.rs:12:5
|
12 | get_num_cpus()
| ^^^^^^^^^^^^ unresolved name
error[E0425]: unresolved name `get_num_cpus`
--> src/lib.rs:28:5
|
28 | get_num_cpus()
| ^^^^^^^^^^^^ unresolved name
num_cpus::get() is totally broken (or my Macbook Air has surpassed the axioms of physics and now has 18,446,744,073,709,551,615 cores).
This is braking a lot of APIs so please treat this as urgent!
I spotted it as I am using nickel.rs and my server wouldn't start and debugged it to find out that Hyper was throwing an arithmetic overflow at server/mod.rs:90 and it's due to a broken num_cpus::get()
There should be example code in the Rustdoc for num_cpus::get_physical.
Quoting from mozilla.dev.platform:
sysconf(_SC_NPROCESSORS_ONLN) is used a number of places in the tree to get
a core count for parallel algorithms. The web says that that phrase does
not really return the number of available cores on ARM Linuxish systems in
general, including Android - because the return value does not account for
cores that are sleeping for power saving reasons. (I've confirmed this
locally on an ARM board.) So you may get lucky when you ask. Or not.sysconf(_SC_NPROCESSORS_CONF) works better here.
JS engine issue with some background and links to other discussions:
https://bugzilla.mozilla.org/show_bug.cgi?id=1333059.
Current documentation for get() is quite limited and should probably be expanded.
Linux "sched affinity" is noted but could usefully be explained better (what does this mean for the resulting count?)
On linux, num_cpus::get() currently returns 1 if the calling thread has its sched_affinity set to only one thread.
I have a use case where a main thread is pinned to core 0, and worker threads are pinned to cores 1, 2, 3 etc.
In this use case, the main thread can no longer use num_cpus::get() to determine the number of cpus in the system. In my case, I had a problem where a thread pool-implementation erroneously decided to use a single worker thread, because it thought the machine had only one CPU, since the pool was initialized on a thread which was pinned to a specific core.
I get the idea of returning "usable cores", but I think there is also a use case for a call to return the total number of cpus (the set which could be used in a call to sched_setaffinity).
Would you be open to pull request to add such a call?
Hi thanks for the lib! I wonder how to know the number of big and little cores in ARM big.LITTLE architecture? For example, I may only want to spawn number of threads that equals the big cores.
The compiler has a whole bunch of targets running in its TravisCI, maybe we can do the same.
Here's a list of OSes that have specific support in this crate, and whether CI is setup to test them:
In case you were not aware, there's a quite annoying LXCFS (which, among others, provides /proc under LXC) bug, lxc/lxcfs#469
Is num_cpus impacted by this? I am not aware of any code I would be using under LXC which is using this crate, but the potential impact is quite wide.
A while ago the issue of crate stability on crates.io came up on Reddit. Steve Klabnik suggested to nudge authors to release 1.0. Now the issue came up again in this thread, so I’ll make the suggestion:
num_cpus has been on crates.io for more than a year. It’s been downloaded half a million times. It exposes a single function, whose signature or behaviour will likely never change. If there is any crate on crates.io that would be in the “obviously no bugs” rather than the “no obvious bugs” category, num_cpus would be a good candidate with its ~100 line implementation. The past five releases would not have been breaking changes if num_cpus were stable.
So, why not release 1.0?
The Fuchsia CI target was disabled, due to this message (seen here):
toolchain 'stable-x86_64-unknown-linux-gnu' does not contain component 'rust-std' for target 'x86_64-unknown-fuchsia'
I don't know why, or what is needed to fix it, but it should likely be fixed.
Is this function:
Line 200 in ae4492e
I got a error count:
This is my /proc/cpuinfo:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 4055.673
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3970.033
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 1
cpu cores : 18
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 4072.285
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 2
cpu cores : 18
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3924.707
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 3
cpu cores : 18
apicid : 6
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 4
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 4000.926
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 4
cpu cores : 18
apicid : 8
initial apicid : 8
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 5
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3979.606
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 5
cpu cores : 18
apicid : 10
initial apicid : 10
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 6
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3978.038
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 0
cpu cores : 6
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3961.449
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 1
cpu cores : 4
apicid : 3
initial apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 8
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 4011.806
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 2
cpu cores : 2
apicid : 5
initial apicid : 5
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 9
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3958.259
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 3
cpu cores : 6
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 10
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 4086.821
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 4
cpu cores : 65534
apicid : 9
initial apicid : 9
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
processor : 11
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0x84
cpu MHz : 3933.493
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 5
cpu cores : 65532
apicid : 11
initial apicid : 11
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4416.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
Linux cgroups(7) can limit cpu shares for a process (typically via Docker). Processes that use num_cpus, however, may be shown the number of CPUs available on the host OS and not the CPU shares as provided by cgroups.
I'm not 100% sure of the state of things, but we should build a small dockerized demo application to validate that CPU limits are reflected through num_cpus. If it's not cgroups-aware, we should make it so.
As the author of dua I noticed that on the M1 chip IO performance gets worse if the high-efficiency cores are taken into consideration when configuring thread pools.
Thus for now I hardcode the value for optimal performance knowing that it might break sometime later this year.
Do you think it's in scope to add such capability to num_cpus?
Currently, cgroup properties cpu.cfs_quota_us and cpu.cfs_period_us are used to determine the number of CPUs. These properties were removed in cgroups v2 and replaced with cpu.max. The standard library function std::thread::available_parallelism handles this correctly, detecting the cgroups version at runtime and using a separate routine for each: https://github.com/rust-lang/rust/blob/master/library/std/src/sys/unix/thread.rs#L412
This is especially a problem for Rust daemons that use a tokio multi-threaded runtime (tokio can't use available_parallelism due to MSRV). Some distributions use cgroups v2 by default (Arch definitely does, and I've seen evidence that Debian >= 11 does as well), and CPU limits set in systemd are implemented with cgroups. This can lead to way more worker threads than would be reasonable on machines with a lot of cores.
For anyone looking for a workaround, you can instantiate the runtime yourself and use available_parallelism to set the number of worker threads: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b5083320c328bd5acb16216f0a0faf54
This issue was automatically generated. Feel free to close without ceremony if
you do not agree with re-licensing or if it is not possible for other reasons.
Respond to @cmr with any questions or concerns, or pop over to
#rust-offtopic on IRC to discuss.
You're receiving this because someone (perhaps the project maintainer)
published a crates.io package with the license as "MIT" xor "Apache-2.0" and
the repository field pointing here.
TL;DR the Rust ecosystem is largely Apache-2.0. Being available under that
license is good for interoperation. The MIT license as an add-on can be nice
for GPLv2 projects to use your code.
The MIT license requires reproducing countless copies of the same copyright
header with different names in the copyright field, for every MIT library in
use. The Apache license does not have this drawback. However, this is not the
primary motivation for me creating these issues. The Apache license also has
protections from patent trolls and an explicit contribution licensing clause.
However, the Apache license is incompatible with GPLv2. This is why Rust is
dual-licensed as MIT/Apache (the "primary" license being Apache, MIT only for
GPLv2 compat), and doing so would be wise for this project. This also makes
this crate suitable for inclusion and unrestricted sharing in the Rust
standard distribution and other projects using dual MIT/Apache, such as my
personal ulterior motive, the Robigalia project.
Some ask, "Does this really apply to binary redistributions? Does MIT really
require reproducing the whole thing?" I'm not a lawyer, and I can't give legal
advice, but some Google Android apps include open source attributions using
this interpretation. Others also agree with
it.
But, again, the copyright notice redistribution is not the primary motivation
for the dual-licensing. It's stronger protections to licensees and better
interoperation with the wider Rust ecosystem.
To do this, get explicit approval from each contributor of copyrightable work
(as not all contributions qualify for copyright) and then add the following to
your README:
## License
Licensed under either of
* Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any
additional terms or conditions.
and in your license headers, use the following boilerplate (based on that used in Rust):
// Copyright (c) 2016 num_cpus developers
//
// Licensed under the Apache License, Version 2.0
// <LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0> or the MIT
// license <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. All files in the project carrying such notice may not be copied,
// modified, or distributed except according to those terms.
Be sure to add the relevant LICENSE-{MIT,APACHE} files. You can copy these
from the Rust repo for a plain-text
version.
And don't forget to update the license metadata in your Cargo.toml to:
license = "MIT/Apache-2.0"
I'll be going through projects which agree to be relicensed and have approval
by the necessary contributors and doing this changes, so feel free to leave
the heavy lifting to me!
To agree to relicensing, comment with :
I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option
Or, if you're a contributor, you can check the box in this repo next to your
name. My scripts will pick this exact phrase up and check your checkbox, but
I'll come through and manually review this issue later as well.
This crate atm doesn't show up at all when seaching for cpu. Because of this, I had a lot of trouble today finding this crate.
Adding this tag to the toml would make the crate slightly easier to find.
cargo new example
cd example
cat << EOF >> Cargo.toml
num_cpus = "1.7"
blas = "0.18"
EOF
cat << EOF > src/lib.rs
extern crate num_cpus;
extern crate blas;
#[test]
fn test_cpus() {
assert_eq!(num_cpus::get(), 32);
}
unsafe fn link() {
blas::c::daxpy(0, 1.0, &[], 1, &mut [], 1);
}
EOF
cargo testresults in
running 1 test
test test_cpus ... FAILED
failures:
---- test_cpus stdout ----
thread 'test_cpus' panicked at 'assertion failed: `(left == right)`
left: `1`,
right: `32`', src/lib.rs:6:4
note: Run with `RUST_BACKTRACE=1` for a backtrace
On my machine I am expecting 32 cpus. Commenting out the link function causes the test to pass.
I can honestly say that I have very little clue as to what's going on here.
I am running x86_64 linux.
The get_physical implementation for Linux should consider the affinity mask when collecting the number of physical cores on the system.
on an aws windows instance, num_cpus::get() returned 48 for a machine with 96 logical cores. I expected the result would be 96, not 48, the number of physical cores.
num_cpus v1.10.1
rustc: 1.38.0-nightly 2019-07-18
CPU readout from windows task manager:
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
Base speed: 2.50 GHz
Sockets: 2
Cores: 48
Logical processors: 96
Virtualization: Enabled
L1 cache: 3.0 MB
L2 cache: 48.0 MB
L3 cache: 2.0 MB
Thanks for your hard work on the library!
(edit: adds num_cpus version)
Crate level documentation can probably be usefully expanded. It should follow the guidelines set out in RFC1574.
As a part of cargo crev I was reviewing this crate in version 1.10.1 . All was good, but I have one idea.
It seems like on Windows this crate calls the GetLogicalProcessorInformation twice. Any reason not to start with a buffer of a fix size (eg. 64 entries)? Maybe this initial buffer could be on stack, so it wouldn't really cost anything. On most machines that would cut the number of calls to just 1.
The current signature is pub fn get_physical() -> usize. The documentation says that it will return the same as get() if it isn't able to get the number of physical cores on that architecture. I'm wondering if it would be a good idea to have get_physical() return a Result<usize, usize> to inform the user of whether that fallback was utilized or not.
I think version 0.2.8 is failing on Windows at the moment because of the removal of SYSTEM_INFO and GetSystemInfo from libc (rust-lang/libc@c4d4c70). I think they're available in winapi now instead.
~\num_cpus-0.2.8\src\lib.rs:16:26: 16:43 error: use of undeclared type name `libc::SYSTEM_INFO` [E0412]
~\num_cpus-0.2.8\src\lib.rs:16 let mut sysinfo: libc::SYSTEM_INFO = ::std::mem::uninitialized();
^~~~~~~~~~~~~~~~~
~\num_cpus-0.2.8\src\lib.rs:16:26: 16:43 help: run `rustc --explain E0412` to see a detailed explanation
~\num_cpus-0.2.8\src\lib.rs:17:9: 17:28 error: unresolved name `libc::GetSystemInfo` [E0425]
~\num_cpus-0.2.8\src\lib.rs:17 libc::GetSystemInfo(&mut sysinfo);
^~~~~~~~~~~~~~~~~~~
~\num_cpus-0.2.8\src\lib.rs:17:9: 17:28 help: run `rustc --explain E0425` to see a detailed explanation
error: aborting due to 2 previous errors
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.