inovex / mqtt-stresser Goto Github PK
View Code? Open in Web Editor NEWLoad testing tool to stress MQTT message broker
License: Apache License 2.0
Load testing tool to stress MQTT message broker
License: Apache License 2.0
This tool would be amazing if it could be used on ARM. Many mqtt scenarios use devices with the ARM architecture.
I want to simulate a situation where we have many clients, sending a message once a minute.
I'd suggest a new parameter that accepts a wait time in seconds, like wait-period and then in the worker, add a thread sleep (or similar) to wait for the required period of time.
Awesome tool by the way, thanks!
Is there any opportunity to send file as payload.
hello:
I found your project very useful, but I had to add support for TLS authentication to make it work in our brokers.
I created a PR in case you find that feature of any value. Let me know if you would like me to change something.
thanks for sharing.
Hi guys,
I've been trying to use the stresser in the form of docker image on Windows 7 Ultimate using docker Toolbox. Without tls certificate it works great, but when i tried to pass a certificate through the "docker run" command, and even when i tried to bind mount, it still says the cert "doesn't exist" at the path i mounted it to. Here's the command:
$ docker run -v /e/Docker\ Toolbox/:/home inovex/mqtt-stresser -broker tcp://192.168.0.10:8883 -cert /home/stress_client.crt -num-clients 10 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20
I know i haven't specified CA cert yet, but i suspect to run into the same problem.
Could you let me know what is the intended way to pass in certificate when using the docker image, or if i'm doing anything wrong with the command above
Thank you for your time
My receiving client deserializes the paylod from cbor format. Is there a way to send an array of bytes as payload?
using the existing Dockerfile as base
root@ip-172-31-22-25:~/mqtttester/mqtt-stresser# make
GOOS=windows GOARCH=amd64 go build -o build/mqtt-stresser.exe
/bin/sh: 1: go: not found
Makefile:46: recipe for target 'build/mqtt-stresser-windows-amd64.zip' failed
make: *** [build/mqtt-stresser-windows-amd64.zip] Error 127
Hello,
i got an unexpected behavior while testing our broker: Test timeout. Wait 5s to allow disconnection of clients.
The Debug Log:
DEBUG: 13:03:17.305348 worker.go:190: [5] starting control loop test/test/76027e37ce19/worker5-1693134756
DEBUG: 13:03:17.305464 worker.go:207: [6] all messages published
DEBUG: 13:03:17.305663 worker.go:207: [0] all messages published
DEBUG: 13:03:17.305671 worker.go:207: [5] all messages published
DEBUG: 13:03:17.306942 worker.go:190: [2] starting control loop test/test/76027e37ce19/worker2-1550364868
DEBUG: 13:03:17.307255 worker.go:207: [2] all messages published
DEBUG: 13:03:17.307308 worker.go:179: [9] connecting publisher
DEBUG: 13:03:17.307320 worker.go:190: [8] starting control loop test/test/76027e37ce19/worker8-1457327270
DEBUG: 13:03:17.308286 worker.go:207: [8] all messages published
DEBUG: 13:03:17.313748 worker.go:190: [7] starting control loop test/test/76027e37ce19/worker7-112932577
DEBUG: 13:03:17.314074 worker.go:207: [7] all messages published
DEBUG: 13:03:17.327766 worker.go:190: [9] starting control loop test/test/76027e37ce19/worker9-1605856701
DEBUG: 13:03:17.327976 worker.go:207: [9] all messages published
Test timeout. Wait 5s to allow disconnection of clients.
DEBUG: 13:06:37.242693 worker.go:240: [5] received abort signal due to test timeout
DEBUG: 13:06:37.242730 worker.go:260: [5] worker finished
no results are displayed.
i have been setting the -constant-payload flag, but I seem to be getting the default messages each time. this seems to happen with the topic set by the flag -topic-base-path(see the logs below). I am using rust's paho_mqtt for my client and mosquitto as the broker.
Here is my command:
docker run --rm inovex/mqtt-stresser -broker tcp://xxxxxxx:1883 -num-clients 1 -num-messages 1 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s -skip-tls-verification true -constant-payload hello -topic-base-path 1231asss/1212asss/event
I am using rust to deserialize the payload using the serde crate on my client side:
let report = std::str::from_utf8(payload_str);
info!("THIS IS A TEST: {:?}", report);
and this is the result from my logs:
2021-09-09T12:57:32.240024471-04:00 INFO mqtt_bin::mqttmod::mqtt_bin - do async_wait_message topic received: internal/mqtt-stresser/b262e40f3baa/worker0-722306144
2021-09-09T12:57:32.242970317-04:00 INFO mqtt_bin::mqttmod::mqtt_event_process::event_process - event_porcess_func-payload: [116, 104, 105, 115, 32, 105, 115, 32, 109, 115, 103, 32, 35, 48, 33]
2021-09-09T12:57:32.243107067-04:00 INFO mqtt_bin::mqttmod::mqtt_event_process::event_process - THIS IS A TEST: Ok("this is msg #0!")
Hello,
i tried many times to connect to a local broker (127.0.0.1) (docker image eclipse-mosquitto) without success (mqtt-stresser container).
With another broker in my local network (ip adress) it's working flawlessly.
I tried to debug without success.
Here some tries (i verified with mosquitto_pub that the broker is up)
~/bin/apache-jmeter-5.4/bin $ docker run inovex/mqtt-stresser -broker :1883 -num-clients 1 -num-messages 10 -log-level 2
1 worker started
DEBUG: 11:47:58.796291 worker.go:89: [0] initializing
DEBUG: 11:47:58.796361 worker.go:104: [0] topic=internal/mqtt-stresser/de1628ebb3d9/worker0-2132015686 subscriberClientId=mqtt-stresser-sub-de1628ebb3d9-worker0-2132015686 publisherClientId=mqtt-stresser-pub-de1628ebb3d9-worker0-2132015686
DEBUG: 11:47:58.796384 worker.go:133: [0] connecting subscriber
DEBUG: 11:48:03.796613 worker.go:155: [0] subscribing to topic
DEBUG: 11:48:03.796696 worker.go:146: [0] unsubscribe
DEBUG: 11:48:03.796758 main.go:291: 1/1 events received
failed to unsubscribe: Not Connected
~/bin/apache-jmeter-5.4/bin $ docker run inovex/mqtt-stresser -broker localhost:1883 -num-clients 1 -num-messages 10 -log-level 2
1 worker started
DEBUG: 11:48:47.676236 worker.go:89: [0] initializing
DEBUG: 11:48:47.676318 worker.go:104: [0] topic=internal/mqtt-stresser/9038eac90665/worker0-1771842039 subscriberClientId=mqtt-stresser-sub-9038eac90665-worker0-1771842039 publisherClientId=mqtt-stresser-pub-9038eac90665-worker0-1771842039
DEBUG: 11:48:47.676346 worker.go:133: [0] connecting subscriber
failed to unsubscribe: Not Connected
DEBUG: 11:48:52.676640 worker.go:155: [0] subscribing to topic
DEBUG: 11:48:52.676783 worker.go:146: [0] unsubscribe
DEBUG: 11:48:52.676858 main.go:291: 1/1 events received
~/bin/apache-jmeter-5.4/bin $ docker run inovex/mqtt-stresser -broker 127.0.0.1:1883 -num-clients 1 -num-messages 10 -log-level 2
1 worker started
DEBUG: 11:49:25.085916 worker.go:89: [0] initializing
DEBUG: 11:49:25.086003 worker.go:104: [0] topic=internal/mqtt-stresser/dd21babcd6b3/worker0-1451970135 subscriberClientId=mqtt-stresser-sub-dd21babcd6b3-worker0-1451970135 publisherClientId=mqtt-stresser-pub-dd21babcd6b3-worker0-1451970135
DEBUG: 11:49:25.086023 worker.go:133: [0] connecting subscriber
EDEBUG: 11:49:30.086227 worker.go:155: [0] subscribing to topic
DEBUG: 11:49:30.086319 main.go:291: 1/1 events received
DEBUG: 11:49:30.086382 worker.go:146: [0] unsubscribe
Working with mosquitto_pub:
~/bin/apache-jmeter-5.4/bin $ mosquitto_pub -m "{\"status\":\"off\"}" -t "test/topic" --repeat 1000 --repeat-delay 2 -p 1883 -d
Client (null) sending CONNECT
Client (null) received CONNACK (0)
Client (null) sending PUBLISH (d0, q0, r0, m1, 'test/topic', ... (16 bytes))
Client (null) sending PUBLISH (d0, q0, r0, m2, 'test/topic', ... (16 bytes))
Client (null) sending PUBLISH (d0, q0, r0, m3, 'test/topic', ... (16 bytes))
Sorry if I forgot something.
Thank you,
ruio
Ps: i tried to run the binaries without success
~/Downloads $ ./mqtt-stresser-linux-amd64 -broker :1883 -num-clients 1 -num-messages 10 -log-level 2
open : no such file or directory
Ps2: I tried to make without success
~/workspace/mqtt-stresser (master)$ make
GO111MODULE=on GOOS=windows GOARCH=amd64 go build -mod=vendor -o build/mqtt-stresser-windows-amd64.exe
flag provided but not defined: -mod
usage: build [-o output] [-i] [build flags] [packages]
Run 'go help build' for details.
Makefile:81: recipe for target 'build/mqtt-stresser-windows-amd64.exe' failed
make: *** [build/mqtt-stresser-windows-amd64.exe] Error 2
Any number of clients with -num-messages exceeding 220 will result in a crash.
ERROR: main.go:162: {0 TimeoutExceeded 3.4400055s 0s 220 300 true }
When exceeding 220 the script freezes until the global timeout hits.
I am facing error running sample emqx free server
Below is the command
docker run inovex/mqtt-stresser -broker mqtt://broker.emqx.io:1883 -num-clients 100 -num-messages 10 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s -skip-tls-verification true
Error i am getting is
Efailed to unsubscribe: Not Connected
Efailed to unsubscribe: Not Connected
The problem occurs while stress testing some brokers like Bevywise MQTT Route and HiveMq as they support only 25 clients to be connected in their evaluation versions.
MQTT Route keeps a client connection count which works based on the Clean Session. Hence what happens while connecting to the Broker Server(using MQTT Stresser) the second connection request from the same client (MQTT Stresser ) to publish/subscribe some msgs is being treated as a New client and client counter gets increased by 1 and thus exceeds 25.
If we could initiate a client connection to the MQTTRoute setting the Clean session as false ( 0 ), It will treats the second connection request with the same unique Client ID as a re-connection and will not increase the connected client count and I can conduct some fare stress test keeping all the testing parameters constant for all the brokers that I put for evaluation of performace.
./mqtt-stresser -broker tcp://broker.mqttdashboard.com:1883 -num-clients 10 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
open : no such file or directory
< 147 msg/sec 75%
< 155 msg/sec 93%
< 164 msg/sec 96%
< 181 msg/sec 97%
< 190 msg/sec 98%
< 215 msg/sec 99%
< 232 msg/sec 100%
msg/sec only, but what's the the length of each msg?
I`m with problem to execute mqtt-stresser. I already updated the modules
./build/mqtt-stresser-linux-amd64 -broker tcp://localhost:1883 -num-clients 100 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
open : no such file or directory
Hello!
I'm stress testing a broker and and I noticed a lot bigger receiving througput with QoS 1 and 2. Otherwise same parameters.
Broker is on AWS.
No errors, all 100% published/received.
Why is that?
./mqtt-stresser-linux-amd64 \
-broker tcp://localhost:1883 \
-num-clients 1000 \
-num-messages 10 \
-rampup-delay 5s \
-global-timeout 180s \
-timeout 20s \
-publisher-qos 2 \
-constant-payload $(cat payload40kb.txt) \
> mqtt_stresser_40kb_10_qos2.txt
/
QoS 0
# Receiving Througput
Fastest: 126 msg/sec
Slowest: 23 msg/sec
Median: 77 msg/sec
QoS 1
# Receiving Througput
Fastest: 49685 msg/sec
Slowest: 71 msg/sec
Median: 550 msg/sec
QoS 2
# Receiving Througput
Fastest: 31005 msg/sec
Slowest: 69 msg/sec
Median: 567 msg/sec
Sorry if is not a bug :)
I am asking how we can start mqtt stresser to generate
{ "test:" "this is lodaing test message"}
docker run --rm inovex/mqtt-stresser -broker tcp://xxxxxxx:8883 -num-clients 50 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s -username xxxx -password xxx -skip-tls-verification true
I was testing my mosquitto broker with the following command
mqtt-stresser -broker tcp://192.168.1.221:1883 \
-num-clients 10000 \
-num-messages 10 \
-rampup-delay 200ms \
-rampup-size 1000 \
-global-timeout 180s \
-timeout 20s
However the test stopped in the middle with the Not Connected
error. Does this simply mean my broker can not handle 10K concurrent connections?
1000 worker started - waiting 200ms
2000 worker started - waiting 200ms
3000 worker started - waiting 200ms
4000 worker started - waiting 200ms
5000 worker started - waiting 200ms
6000 worker started - waiting 200ms
7000 worker started - waiting 200ms
8000 worker started - waiting 200ms
9000 worker started - waiting 200ms
10000 worker started
.........................(many-dots-omitted-here)...........................Not Connected
time.Sleep(rampUpDelay). // <-- here is first
select {
case <-time.NewTimer(rampUpDelay).C: // <-- here is second
case s := <-signalChan:
fmt.Printf("Got signal %s. Cancel test.\n", s.String())
cancelFunc()
stopStartLoop = true
}
why need two ?
via CI builds?
When run the exe with ./mqtt-stresser-linux-amd64-static -broker tcp://45.76.163.6:1883 -num-clients 32219 -num-messages 100 -publisher-qos 2 -subscriber-qos 2
at the platform of Ubuntu 14.04 x64.
(45.76.163.6 is my centos6 1G 1CPU vps in vultr.com in Singapore, if you want, you can use it to test, and it's broker's web service port is 18083)
Here is the output,
100 worker started - waiting 500ms
200 worker started - waiting 500ms
300 worker started - waiting 500ms
400 worker started - waiting 500ms
500 worker started - waiting 500ms
600 worker started - waiting 500ms
700 worker started - waiting 500ms
800 worker started - waiting 500ms
900 worker started - waiting 500ms
1000 worker started - waiting 500ms
1100 worker started - waiting 500ms
1200 worker started - waiting 500ms
1300 worker started - waiting 500ms
1400 worker started - waiting 500ms
1500 worker started - waiting 500ms
1600 worker started - waiting 500ms
1700 worker started - waiting 500ms
1800 worker started - waiting 500ms
1900 worker started - waiting 500ms
2000 worker started - waiting 500ms
2100 worker started - waiting 500ms
2200 worker started - waiting 500ms
2300 worker started - waiting 500ms
2400 worker started - waiting 500ms
2500 worker started - waiting 500ms
2600 worker started - waiting 500ms
2700 worker started - waiting 500ms
2800 worker started - waiting 500ms
2900 worker started - waiting 500ms
3000 worker started - waiting 500ms
3100 worker started - waiting 500ms
3200 worker started - waiting 500ms
3300 worker started - waiting 500ms
3400 worker started - waiting 500ms
3500 worker started - waiting 500ms
3600 worker started - waiting 500ms
3700 worker started - waiting 500ms
3800 worker started - waiting 500ms
3900 worker started - waiting 500ms
4000 worker started - waiting 500ms
4100 worker started - waiting 500ms
4200 worker started - waiting 500ms
4300 worker started - waiting 500ms
4400 worker started - waiting 500ms
4500 worker started - waiting 500ms
4600 worker started - waiting 500ms
4700 worker started - waiting 500ms
4800 worker started - waiting 500ms
4900 worker started - waiting 500ms
5000 worker started - waiting 500ms
5100 worker started - waiting 500ms
5200 worker started - waiting 500ms
5300 worker started - waiting 500ms
5400 worker started - waiting 500ms
5500 worker started - waiting 500ms
5600 worker started - waiting 500ms
5700 worker started - waiting 500ms
5800 worker started - waiting 500ms
5900 worker started - waiting 500ms
6000 worker started - waiting 500ms
6100 worker started - waiting 500ms
panic: close of nil channel
goroutine 31121 [running]:
github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho%2emqtt%2egolang.(*client).closeStop(0xc01de78380)
/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho.mqtt.golang/client.go:482 +0x12f
github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho%2emqtt%2egolang.(*client).disconnect(0xc01de78380)
/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho.mqtt.golang/client.go:495 +0x2f
github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho%2emqtt%2egolang.(*client).Disconnect(0xc01de78380, 0x5)
/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho.mqtt.golang/client.go:439 +0x1f9
main.(*Worker).Run(0xc01dd71140, 0x719f40, 0xc000054420)
/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/worker.go:127 +0xf23
created by main.main
/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/main.go:174 +0x592
Every-time I try to stress-test test.mosquitto.org, I am getting the following output/error(but works fine with other publicly hosted brokers). Please help me understand this and guide how can I get the full result.
thanks in advance.
./mqtt-stresser -broker tcp://test.mosquitto.org:1883 -nm-clients 100 -num-messages 10 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
10 worker started - waiting 1s
20 worker started - waiting 1s
30 worker started - waiting 1s
40 worker started - waiting 1s
50 worker started - waiting 1s
60 worker started - waiting 1s
70 worker started - waiting 1s
80 worker started - waiting 1s
90 worker started - waiting 1s
100 worker started
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEpanic: close of nil channel
goroutine 23 [running]:
panic(0x6f54c0, 0xc820336670)
/usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
github.com/eclipse/paho%2emqtt%2egolang.(*client).closeStop(0xc82015af00)
/home/install/go/src/github.com/eclipse/paho.mqtt.golang/client.go:482 +0x1b2
github.com/eclipse/paho%2emqtt%2egolang.(*client).disconnect(0xc82015af00)
/home/install/go/src/github.com/eclipse/paho.mqtt.golang/client.go:495 +0x2b
github.com/eclipse/paho%2emqtt%2egolang.(*client).Disconnect(0xc82015af00, 0x5)
/home/install/go/src/github.com/eclipse/paho.mqtt.golang/client.go:439 +0x314
main.(*Worker).Run.func2(0xc820092500, 0x7fc524d3d910, 0xc82015af00, 0xc820012900, 0x3f)
/home/install/Downloads/Brokers/benchmark/mqtt-stresser-master/worker.go:70 +0x304
main.(*Worker).Run(0xc820092500)
/home/install/Downloads/Brokers/benchmark/mqtt-stresser-master/worker.go:163 +0x23fe
created by main.main
/home/install/Downloads/Brokers/benchmark/mqtt-stresser-master/main.go:137 +0x96a
How to start the stesser with user name and password for mqtt authentication?
It would be of value to be able to set the QoS level for subscribers and publishers. The retain value for publishers is also valuable when stress testing different scenarios.
I suggest:
-publisher-qos
-subscriber-qos
-retain
Hello,
For clarification purposes, would it be possible to add some description covering which layers and protocols this tool explores? For reliability tests, this would be pretty useful! Thanks!
I would just suggest you to list pre-requisities for the build and using mqtt-stresser. You can figure out you are using go but would be great to put a reference to it etc.
Hello:
this is not a big deal, but I'm finding something a little confusing in the report output. When, for example, I run the program with -num-clients 10 -num-messages 10
The report shows Messages / Client: 100 , that is the total number of messages (clients * messages)
My expectation is that this value would be the same as --num-messages to reflect the configuration of that run.
Sample output:
# Configuration
Concurrent Clients: 10
Messages / Client: 100
# Results
Published Messages: 100 (100%)
Received Messages: 100 (100%)
Completed: 10 (100%)
Errors: 0 (0%)
If this is something that should be fixed, I'll be happy to make the PR with the adjustment (it should be minimal), but if I'm understanding this wrong, please disregard.
All the best
To ease systematic benchmark, we should export the benchmark results in a machine readable format. This allows import in evaluation tool chains like jupyter notebooks, R or Excel.
I want to use the MQTT-Stresser to test a MQTT broker which only supports secure links and self signed certificates.
I am using the docker but am not able to connect to the broker. Since it is a self signed certificate I added -skip-tls-verification Do I still have to provide a CA file wit this option.
I tried too with the HiveMQ broker from the example with gave the exact same result I running on Ubuntu
sudo docker run --rm inovex/mqtt-stresser -broker tcp://broker.mqttdashboard.com:8883 -skip-tls-verification -num-clients 2 -num-messages 2 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
Error message:
failed to unsubscribe: Not Connected
failed to unsubscribe: Not Connected
btw if I do add a CA certificate with the -ca_file option I get, independent of which certificate file I use
panic: tls: failed to find any PEM data in certificate input
goroutine 12 [running]:
main.(*Worker).Run(0xc000066580, 0x77ee20, 0xc000046420)
/go/src/github.com/inovex/mqtt-stresser/worker.go:131 +0x1b6d
created by main.main
/go/src/github.com/inovex/mqtt-stresser/main.go:271 +0x910
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.