Comments (7)
Right now if you want to provide taskset for parallel benchmarks things are manual and error prone. You have to edit the JSON file for each benchmark to put the correct cpu-list in.
In an ideal world we would have a function get_taskset_command <num_domains>
which returns the taskset command by probing the machine to figure out which CPUs should be used (and it could also handle the cross platform problems where taskset may not be available).
There are a couple of things the user wants control over:
- should 'hyper-thread' cores be used?
- which NUMA zones should be used on machines with multiple NUMA zones?
- in situations with isolated cores, how do we specify the cores?
Given these features. I think the following might work as a proposal:
get_taskset_command <available_cpus> <num_domains>
where
<available_cpus>
is a comma separated ordered list of available CPUs<num_domains>
is the number of CPUs to take from the head of<available_cpus>
For example:
get_taskset_command 0,2,4,6 3
would return
taskset --cpu-list 0,2,4
This would allow us to specify the CPUs and the order to take them in a single place and so make the config file easier to use.
On a non-isolated Linux machine where you want to use hyper-threaded cores, you could run lscpu | egrep 'NUMA node[0-9]'
to get <available_cpus>
and copy them to the config. Users that want more control can construct the list in alternative ways (e.g. 'only isolated cores', 'use no hyperthread pairs', etc.)
from sandmark.
get_taskset_command
seems like a good idea. Where do you think this command would get invoked? If you make this as part of orun
then the other wrappers will have to somehow replicate this.
One possibility is to generate the run_config.json
file with the correct CPU list for taskset
using the get_taskset_command
from sandmark.
My thinking was that this would be a stand alone UNIX command that could be composed with others. It could even be a shell script.
You could imagine the user having it in the wrappers
section of the run_config.json
:
"wrappers": [
{
"name": "orun",
"command": "orun -o %{output} -- get_taskset_command %{ordered_cpu_list} %{num_domains} %{command}"
}
],
...
"benchmarks": [
{
"executable": "benchmarks/multicore-numerical/binarytrees5_multicore.exe",
"name": "binarytrees5_multicore",
"runs": [
{ "params": "1 23", "num_domains": "1" },
{ "params": "2 23", "num_domains": "2" },
The changes being:
- utilize the new
get_taskset_command
in thewrappers
section - have
num_domains
in theruns
section of thebenchmarks
- find a way to introduce the
ordered_cpu_list
variable; not sure how to do this, we could just make people put it as a list into the wrapper. Might be nicer to make it a variable somehow
I'd rather have get_taskset_command
as a first class command that can be composed (instead of say embedded in orun
) because this might be easier to ensure we support more platforms (e.g. OS X, Windows, etc) and also a bit more transparent to the user when doing custom experiments.
from sandmark.
Ok. that makes sense. For ordered_cpu_list
, this should be part of the variant json file describing the compiler. The ordered_cpu_list
will be the same for all of the benchmarks. We need to just have different variant files for different machines.
Ideally, I want to get Sanmark to a point where all the variables either in the variant json file or the run_config json file. The various environment variables that we pass to the Makefile can be moved to the variant json file.
from sandmark.
ordered_cpu_list
is a bit new for us as it is a parameter describing the machine. I would request not putting it in the compiler variant json file - that would be a really unexpected place to find it (at least for me), since the description of how to find the source and build parameters for a compiler has nothing to do with machine specs for the experiment.
It feels like the run_config.json
is the right place to put it as that describes the run plan of a given experiment. The wrappers
block of the run_config.json
file isn't the worst place to put ordered_cpu_list
(at least for now).
I agree with the goal of having a 'single source of truth' for what will be run in an experiment. Having things in unexpected places is a pain.
from sandmark.
I guess the direction I'm trying to go is that as a user, it would be great if I could post a single config file to somebody and they can recreate the run as well as see what got run. Maybe one day the variant files need to end up in the run_config.json
?
from sandmark.
Ok. That works. Having everything in a single location sounds like a fine idea.
from sandmark.
Related Issues (20)
- Report hardware resources in parallel architectures using `hwloc`
- Integrate SPECpower benchmarks and measurements in Sandmark
- Multicore parallel benchmarks support for 4.14 HOT 2
- Fix Uuidm.create, Stdlib.Printf.kprintf and ocaml_deprecated_auto_include compiler warnings
- `mtime.1.2.0` fails to build with 5.1.0+trunk HOT 1
- Add MaPLe benchmarks to Sandmark HOT 1
- graph500 failing HOT 1
- Add benchmarks from frama-c Open Source case studies
- Include pausetimes in jupyter notebook HOT 1
- Remove the old implementations of graph500
- Update Sandmark trunk to build for 5.2.0 HOT 1
- A number of parallel benchmarks seem to be broken HOT 2
- Include percentiles in the pausetimes bench data HOT 2
- Round off errors for mean and max latencies HOT 2
- Unpin `runtime_events_tools` and pin `ppxlib` to a version compatible with OCaml 5.1 and 5.2 HOT 2
- Get latency and throughput information from the same benchmark runs
- Running parallel benchmarks seems to fail sometimes because chrt permissions are lost?
- Benchmark runs seem to be failing due to base.v0.16.3 install failure
- Benchmark runs seem to be failing due to changes on trunk
- Remote access to Turing broken HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sandmark.