isabella232 / snmpbench

[Work In Progress] Benchmark SNMP libraries

snmpbench

Benchmark for snmp libraries.

The benchmark is focused on how the library perform (Speed, Memory, CPU usage, etc) under different settings (batch size, rounds, sessions, etc).

Results can be viewed here: https://datadoghq.dev/snmpbench/

Benchmarked libraries

• pysnmp
• net-snmp using python3-netsnmp (bindings)
• gosnmp

Methodology

We run different scenarios for each library and collect measures we are interested in.

For each library, we created a program that accept various parameters (pysnmp, netsnmp, gosnmp)

Scenarios parameters:

• Batch size: the number of OID requested per call
• Rounds: the number of time the scenario is run
• Sessions: the number of instances of the library session/engine we create

Measurements:

• Duration per OID (ms): Average time to retrieve an OID value
• Max RSS (KBytes): maximum resident set size used by the program while running the scenario
• User time (sec): total user time taken to run the scenario
• System Time (sec): total system time taken to run the scenario
• Elapsed Time (sec): total elapsed wall clock time taken to run the scenario

Except for Duration per OID, other measurements are collected using /usr/bin/time.

How to run locally

Requirements:

apt install -y libsnmp-dev
pip install -r requirements.txt
go get github.com/soniah/gosnmp

Run snmpsim:

docker-compose -f environment/  up -d

Run benchmark:

python run_bench.py localhost 1161 --oid-batch-size 10 --sessions 1 --rounds 1

Run scenarios:

python build_batch_size.py
python build_round.py
python build_session.py

See results in docs/generated_data