isabella232 / tranquility

Tranquility helps you send real-time event streams to Druid and handles partitioning, replication, service discovery, and schema rollover, seamlessly and without downtime.

Tranquility

“Stay close, my friends, and I will heal your wounds.”
         —Mythen of the Wild

Tranquility helps you send event streams to Druid, the raddest data store ever (http://druid.io/), in real-time. It handles partitioning, replication, service discovery, and schema rollover for you, seamlessly and without downtime. Tranquility is written in Scala, and bundles idiomatic Java and Scala APIs that work nicely with Finagle, Samza, Spark, Storm, and Trident.

This project is a friend of Druid. For discussion, feel free to use the normal Druid channels: http://druid.io/community/

Documentation

General:

• Overview - Introduction to Tranquility concepts, including details about how it creates and manages Druid tasks.
• Configuration - The first step to using Tranquility is to configure it appropriately for your Druid dataSource. This is generally done through a configuration file, with optional refinements in code.
• Troubleshooting - Solutions to common problems.

Modules:

• Core - The most basic data-sending API. You will likely use this one unless you are using one of the higher-level modules.
• Server - HTTP server that allows you to use Tranquility without developing a JVM app.
• Samza - Tranquility includes a Samza SystemProducer.
• Spark - Tranquility works with RDDs and DStreams.
• Storm - Tranquility includes a Storm Bolt and a Trident State.
• Kafka - Application to push messages from Kafka into Druid through Tranquility.
• Flink - Tranquility includes a Flink Sink.

Getting Tranquility with Maven

Tranquility Core, Samza, Spark, Storm and Flink are meant to be included in an application that you write. Those modules are hosted on Maven Central to make them easy to include. The current stable versions are:

<dependency>
  <groupId>io.druid</groupId>
  <artifactId>tranquility-core_2.11</artifactId>
  <version>0.8.2</version>
</dependency>
<dependency>
  <groupId>io.druid</groupId>
  <artifactId>tranquility-samza_2.11</artifactId>
  <version>0.8.2</version>
</dependency>
<dependency>
  <groupId>io.druid</groupId>
  <artifactId>tranquility-spark_2.11</artifactId>
  <version>0.8.2</version>
</dependency>
<dependency>
  <groupId>io.druid</groupId>
  <artifactId>tranquility-storm_2.11</artifactId>
  <version>0.8.2</version>
</dependency>
<dependency>
  <groupId>io.druid</groupId>
  <artifactId>tranquility-flink_2.11</artifactId>
  <version>0.8.2</version>
</dependency>

You only need to include the modules you are actually using.

All Tranquility modules are built for Scala 2.11.

This version is built to work with Druid 0.7.x and 0.8.x. If you are using Druid 0.6.x, you may want to use Tranquility v0.3.2, which is the most recent version built for use with Druid 0.6.x.

Tranquility is built with SBT. If you want to build the jars yourself, you can run sbt +package. To build a distribution tarball, run sbt ++2.11.8 'project distribution' universal:packageZipTarball.

Downloadable Distribution

The Tranquility downloadable distribution includes the Server and Kafka programs, which are standalone programs that can be used without writing any code. You can download the distribution and run them directly. The distribution also includes the Core API artifacts, if you prefer to download them rather than get them through Maven.

The current distribution is: tranquility-distribution-0.8.2.

To use it, first download it and then unpack it into your directory of choice by running tar -xzf tranquility-distribution-0.8.2.tgz.

How to Contribute

See CONTRIBUTING.md.