klarrio / open-stream-processing-benchmark

This repository contains the code base for the Open Stream Processing Benchmark.

OSPBench: Open Stream Processing Benchmark

This repository contains the code of the open stream processing benchmark.

All documentation can be found in our wiki.

It includes:

• benchmark: benchmark pipeline implementations (docs).
• data-stream-generator: data stream generator to generate input streams locally or on a DC/OS cluster (docs).
• output-consumer: consumes the output of the processing job and metrics-exporter from Kafka and stores it on S3 (docs).
• evaluator: computes performance metrics on the output of the output consumer (docs).
• result analysis: Jupyter notebooks to visualize the results (docs).
• deployment: deployment scripts to run the benchmark on an DC/OS setup on AWS (docs).
• kafka-cluster-tools: Kafka scripts to start a cluster and read from a topic for local development (docs).
• metrics-exporter: exports metrics of JMX and cAdvisor and writes them to Kafka (docs).

Currently the benchmark includes Apache Spark (Spark Streaming and Structured Streaming), Apache Flink and Kafka Streams.

References, Publications and Talks

• van Dongen, G., & Van den Poel, D. (2020). Evaluation of Stream Processing Frameworks. IEEE Transactions on Parallel and Distributed Systems, 31(8), 1845-1858. The Supplemental Material of this paper can be found here.

• van Dongen, G., & Van den Poel, D. (2021). A Performance Analysis Fault Recovery in Stream Processing Frameworks. IEEE Access.

• van Dongen, G., & Van den Poel, D. (2021). Influencing Factors in the Scalability of Distributed Stream Processing Jobs. IEEE Access.

• Earlier work-in-progress publication: van Dongen, G., Steurtewagen, B., & Van den Poel, D. (2018, July). Latency measurement of fine-grained operations in benchmarking distributed stream processing frameworks. In 2018 IEEE International Congress on Big Data (BigData Congress) (pp. 247-250). IEEE. Talks related to this publication:

• Spark Summit Europe 2019: Stream Processing: Choosing the Right Tool for the Job - Giselle van Dongen

Issues, questions or need help?

Are you having issues with anything related to the project? Do you wish to use this project or extend it? The fastest way to contact me is through:

LinkedIn: giselle-van-dongen

Email: [email protected]

open-stream-processing-benchmark's Issues

Bugs in code

I am trying to run this for local deployment with my windows machine, I tried all the steps mentioned in the local deployment, i installed the docker containers with Kafka and zookeeper

I am able to compile the benchmark when i run it with sbt compile but i am facing java run time issue when i am giving sbt run. I am not sure there is bug in the code which I am unable to figure it out. Please help me with the error . I have added the error message. I also tried it with WSL still i get the same error.

WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by sbt.TrapExit$ (file:/C:/Users/HP/.sbt/boot/scala-2.10.7/org.scala-sbt/sbt/0.13.18/run-0.13.18.jar)
WARNING: Please consider reporting this to the maintainers of sbt.TrapExit$
WARNING: System::setSecurityManager will be removed in a future release
[info] Loading project definition from E:\benchmark\open-stream-processing-benchmark\benchmark\project\project
[info] Loading project definition from E:\benchmark\open-stream-processing-benchmark\benchmark\project
[info] Set current project to stream-benchmark (in build file:/E:/benchmark/open-stream-processing-benchmark/benchmark/)
java.lang.RuntimeException: No main class detected.
at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last benchmark/compile:run for the full output.
[error] (benchmark/compile:run) No main class detected.
[error] Total time: 0 s, completed 08-Nov-2023, 2:10:13 pm