sam-tsai / siddhi

Stream Processing and Complex Event Processing Engine

Siddhi

Siddhi is a java library that listens to events from data streams, detects complex conditions described via a Streaming SQL language, and triggers actions. It performs both Stream Processing and Complex Event Processing.

Overview

Siddhi supports the following:

• Data preprocessing
• Generating alerts based on thresholds
• Calculating aggregations over a short window or a long time period
• Joining multiple data streams
• Correlating data while finding missing and erroneous events
• Interacting streaming data with databases
• Detecting temporal event patterns
• Tracking (something over space or time)
• Analyzing trends (rise, fall, turn, tipple bottom)
• Making real-time predictions with existing and online machine learning models
• And many more ... For more information, see Patterns of Streaming Realtime Analytics

Siddhi is free and open source, under Apache Software License v2.0.

Get Started!

Get started with Siddhi in a few minutes by following the Siddhi Quick Start Guide

Why use Siddhi ?

• It is fast. UBER uses it to process 20 Billion events per day (300,000 events per second).
• It is lightweight (<2MB), and embeddable in Android and RaspberryPi.
• It has over 40 Siddhi Extensions
• It is used by over 60 companies including many Fortune 500 companies in production. Following are some examples:
  • WSO2 uses Siddhi for the following purposes:
    • To provide stream processing capabilities in their products such as WSO2 Stream Processor.
    • As the edge analytics library of WSO2 IoT Server.
    • As the core of WSO2 API Manager's throttling.
    • As the core of WSO2 products' analytics.
  • UBER uses Siddhi for fraud analytics.
  • Apache Eagle uses Siddhi as a policy engine.
• Solutions based on Siddhi have been finalists at ACM DEBS Grand Challenge Stream Processing competitions in 2014, 2015, 2016, 2017.
• Siddhi has been the basis of many academic research projects and has over 60 citations.

If you are a Siddhi user, we would love to hear more.

Develop Siddhi using IntelliJ IDEA

Install IDEA plugin to get the following features:

• Siddhi Query Editor with syntax highlighting and with basic auto completion
• Siddhi Runner and Debugger support to test Siddhi Application

Try Siddhi with WSO2 Stream Processor

WSO2 Stream Processor is a server version of Siddhi that is also released under Apache Software License v2.0. It was a Strong Performer in The Forrester Wave: Big Data Streaming Analytics, Q1 2016 (Report) and a Cool Vendors in Internet of Things Analytics, 2016.

If you use WSO2 Stream Processor, you can use the Siddhi functionality with the following additional features:

• The Siddhi Query Editor tool with syntax highlighting and advanced auto completion support
• The Siddhi Runner and Debugger tool
• The Event Simulator tool
• Run Siddhi as a server with high availability and scalability.
• Monitoring support for Siddhi
• Realtime dashboard
• Business user-friendly query generation and deployment

There are domain specific solutions built using Siddhi, including Fraud Detection, Stock Market Surveillance, Location analytics, Proximity Marketing, Contextual Recommendation, Ad Optimization, Operational Analytics, and Detecting Chart Patterns.

For more information please contact us via http://wso2.com/support/.

Siddhi Versions

Find the released Siddhi libraries here.

• Active development version of Siddhi : v4.0.0 built on Java 8.

  Siddhi Query Guide for Siddhi v4.x.x

  Architecture of Siddhi v4.x.x

• Latest Stable Release of Siddhi : v3.0.5 built on Java 7.

  Siddhi Query Guide for Siddhi v3.x.x

Latest API Docs

Latest API Docs is 4.3.8.

Jenkins Build Status

Siddhi Branch	Jenkins Build Status
master

How to Contribute

• Report issues at GitHub Issue Tracker.
• Feel free to try out the Siddhi source code and send your contributions as pull requests to the master branch.

Contact us

• Post your questions with the "Siddhi" tag in Stackoverflow.
• For more details and support contact us via http://wso2.com/support/

Support

• We are committed to ensuring support for Siddhi (with its extensions) and WSO2 Stream Processor from development to production.
• Our unique approach ensures that all support leverages our open development methodology and is provided by the very same engineers who build the technology.
• For more details and to take advantage of this unique opportunity, contact us via http://wso2.com/support/.

Siddhi was joint research project initiated by WSO2 and University of Moratuwa, Sri Lanka.