GithubHelp home page GithubHelp logo

docker-streamserver's Introduction

StreamServer with Docker

Docker Installation

Install Docker (on linux) or Docker-toolbox (on Windows/Mac)

Linux only: Use docker without sudo

Stream Server Installation

  1. Open a terminal (Docker Quickstart Terminal on Windows/Mac or standard terminal on Linux).
  2. Download zip and unzip or “git clone” from Stream Server repository
$ git clone https://github.com/madgik/Docker-StreamServer.git
  1. Linux only:
$ sudo service docker start
  1. Navigate to the Stream Server Directory:
$ cd <path to Docker-StreamServer>
  1. Build Stream Server image (this may take a few minutes the first time):
$ docker build -t streamserver .

Run Stream Server container

  1. Execute:
$ docker run -i -t --rm -p 8989:8989 --name streamserver streamserver

Alt text 2. Leave this console open while you are working and then stop the container. 3. Find your docker machine IP

  1. On Linux is: localhost
  2. On Windows/Mac open a new Docker Quickstart Terminal and run:
```
$ docker-machine ip
```
It will return your docker-machine ip **(from now on use this instead of localhost if you are on Windows or Mac)**.

Test the Stream Server

Test the Stream Server by calling curl on http://docker-machine-ip:8989/measurements

$ curl http://docker-machine-ip:8989/measurements
{"schema":[["timestamp",null],["assembly",null],["sensor",null],["value",null]]}
["2016-03-07T20:13:55+00:00",1.127384714E9,"TC255",58.7]
["2016-03-07T20:13:56+00:00",1.127384715E9,"TC255",58.099]
["2016-03-07T20:13:57+00:00",1.127384716E9,"TC255",57.5]
["2016-03-07T20:13:58+00:00",1.127384717E9,"TC255",56.9]
["2016-03-07T20:13:59+00:00",1.127384718E9,"TC255",56.2]
["2016-03-07T20:14:00+00:00",1.127384719E9,"TC255",55.799]
["2016-03-07T20:14:01+00:00",1.12738472E9,"TC255",55.2]
["2016-03-07T20:14:02+00:00",1.127384721E9,"TC255",54.7]
["2016-03-07T20:14:03+00:00",1.127384722E9,"TC255",54.099]
["2016-03-07T20:14:04+00:00",1.127384723E9,"TC255",53.7]

or open http://docker-machine-ip:8989/measurements on Chrome

Alt text

Exit Stream Server container

To gracefully stop your docker container:

  1. Select your Stream Server docker console.
  2. Press Ctrl+C.
  3. Close the console.

Run multiple Stream Servers

You can run multiple stream servers, concurently, on different docker containers. Just choose a different container name and forward 8989 port to a different port on your local machine.

  1. Open a terminal (Docker Quickstart Terminal on Windows/Mac or standard terminal on Linux).
  2. Execute the following command to start a streamserver named second_streamserver that produces a stream accesible by your docker machine IP:8990
docker run -i -t --rm -p 8990:8989 --name second_streamserver streamserver
  1. Test the Stream Server by opening http://docker-machine-ip:8990/measurements on your browser

Add datasets

To load your own dataset into the stream server you must place your csv file into the "dataset" folder. The name of the fiule must hasve a ".csv" suffix. The filename determines the uri address, i.e. the "measurements.csv" dataset is available at the "/measurements" URI.

The csv files must have the following format:

HEADER : The first row must contain the name of the columns for example (timestamp, sensor, value). The first column must be always the timestamp.
VALUES : The timestamp must be in the RCF3339 format. For example 2016-03-07T20:13:55+00:00 is a valid timestamp.

Streamserver use the files in the "dataset" directory to produce infinite stream data. The timetstamps must be orderedchronogically from older to newer. Streamserver use the timestamps only to simulate the intervals between two different tuples of the stream. The streamserver replaces the timestamps with current timestamps.

After this you must re-built and re-run the docker container.

##Troubleshoot

  • If you are getting an error like:

    docker: Error response from daemon: Conflict. The name "/streamserver" is already in use by container b13022c72864ad6e7651c1681764ec6ed1554f247e11a1070675b952041fbc78. You have to remove (or rename)

    Run:

    docker stop streamserver

    or

    docker kill streamserver

docker-streamserver's People

Contributors

alexpap avatar csvingos avatar johnchronis avatar vnikolopoulos avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-streamserver's Issues

Different frequences for different sensors

Hi,

How is the Stream generator behaving with a mix of sensors with different intervals for measurements?
In the example below: S001 gets a value every second, S002 every 1 minute and S003 every 30 sec.
I never get any answer from Starql queries for S002 and S003 even though I see them listed using http://docker-machine-ip:8989/measurement.
2016-02-19T08:03:00+00:00,S001,0.0421
2016-02-19T08:03:00+00:00,S002,1.5953
2016-02-19T08:03:00+00:00,S003,12.951

2016-02-19T08:03:01+00:00,S001,0.0421
2016-02-19T08:03:02+00:00,S001,0.0694
2016-02-19T08:03:03+00:00,S001,0.0323
2016-02-19T08:03:04+00:00,S001,0.0694
2016-02-19T08:03:05+00:00,S001,0.0566
2016-02-19T08:03:06+00:00,S001,0.0694
2016-02-19T08:03:07+00:00,S001,0.0694
2016-02-19T08:03:08+00:00,S001,0.0694
2016-02-19T08:03:09+00:00,S001,0.0596
2016-02-19T08:03:10+00:00,S001,0.061
2016-02-19T08:03:11+00:00,S001,0.0587
2016-02-19T08:03:12+00:00,S001,0.0694
2016-02-19T08:03:13+00:00,S001,0.0177
2016-02-19T08:03:14+00:00,S001,0.0468
2016-02-19T08:03:15+00:00,S001,0.0694
2016-02-19T08:03:16+00:00,S001,0.0694
2016-02-19T08:03:17+00:00,S001,0.0694
2016-02-19T08:03:18+00:00,S001,0.054
2016-02-19T08:03:19+00:00,S001,0.0365
2016-02-19T08:03:20+00:00,S001,0.0694
2016-02-19T08:03:21+00:00,S001,0.0452
2016-02-19T08:03:22+00:00,S001,0.0694
2016-02-19T08:03:23+00:00,S001,0.0203
2016-02-19T08:03:24+00:00,S001,0.0694
2016-02-19T08:03:25+00:00,S001,0.0194
2016-02-19T08:03:26+00:00,S001,0.0694
2016-02-19T08:03:27+00:00,S001,0.0279
2016-02-19T08:03:28+00:00,S001,0.0593
2016-02-19T08:03:29+00:00,S001,0.0674
2016-02-19T08:03:30+00:00,S001,0.0674
2016-02-19T08:03:30+00:00,S003,12.959
2016-02-19T08:03:31+00:00,S001,0.0674
2016-02-19T08:03:32+00:00,S001,0.0694
2016-02-19T08:03:33+00:00,S001,0.0694
2016-02-19T08:03:34+00:00,S001,0.0523
2016-02-19T08:03:35+00:00,S001,0.0694
2016-02-19T08:03:36+00:00,S001,0.0437
2016-02-19T08:03:37+00:00,S001,0.0203
2016-02-19T08:03:38+00:00,S001,0.0694
2016-02-19T08:03:39+00:00,S001,0.0694
2016-02-19T08:03:40+00:00,S001,0.021
2016-02-19T08:03:41+00:00,S001,0.0205
2016-02-19T08:03:42+00:00,S001,0.0694
2016-02-19T08:03:43+00:00,S001,0.0315
2016-02-19T08:03:44+00:00,S001,0.0694
2016-02-19T08:03:45+00:00,S001,0.0315
2016-02-19T08:03:46+00:00,S001,0.0694
2016-02-19T08:03:47+00:00,S001,0.0559
2016-02-19T08:03:48+00:00,S001,0.0358
2016-02-19T08:03:49+00:00,S001,0.054
2016-02-19T08:03:50+00:00,S001,0.0694
2016-02-19T08:03:51+00:00,S001,0.0694
2016-02-19T08:03:52+00:00,S001,0.054
2016-02-19T08:03:53+00:00,S001,0.0632
2016-02-19T08:03:54+00:00,S001,0.0546
2016-02-19T08:03:55+00:00,S001,0.0239
2016-02-19T08:03:56+00:00,S001,0.0694
2016-02-19T08:03:57+00:00,S001,0.0634
2016-02-19T08:03:58+00:00,S001,0.0661
2016-02-19T08:03:59+00:00,S001,0.0694
2016-02-19T08:04:00+00:00,S001,0.0449
2016-02-19T08:04:00+00:00,S002,1.6873
2016-02-19T08:04:00+00:00,S003,12.966

2016-02-19T08:04:01+00:00,S001,0.0694
2016-02-19T08:04:02+00:00,S001,0.0688
2016-02-19T08:04:03+00:00,S001,0.0694
2016-02-19T08:04:04+00:00,S001,0.0694
2016-02-19T08:04:05+00:00,S001,0.0386
2016-02-19T08:04:06+00:00,S001,0.0694
2016-02-19T08:04:07+00:00,S001,0.0694
2016-02-19T08:04:08+00:00,S001,0.0694
2016-02-19T08:04:09+00:00,S001,0.0694
2016-02-19T08:04:10+00:00,S001,0.0694
2016-02-19T08:04:11+00:00,S001,0.0694
2016-02-19T08:04:12+00:00,S001,0.0694
2016-02-19T08:04:13+00:00,S001,0.0694
2016-02-19T08:04:14+00:00,S001,0.0694
2016-02-19T08:04:15+00:00,S001,0.0694
2016-02-19T08:04:16+00:00,S001,0.0694
2016-02-19T08:04:17+00:00,S001,0.0694
2016-02-19T08:04:18+00:00,S001,0.0694
2016-02-19T08:04:19+00:00,S001,0.0503
2016-02-19T08:04:20+00:00,S001,0.032
2016-02-19T08:04:21+00:00,S001,0.0694
2016-02-19T08:04:22+00:00,S001,0.0339
2016-02-19T08:04:23+00:00,S001,0.0694
2016-02-19T08:04:24+00:00,S001,0.0572
2016-02-19T08:04:25+00:00,S001,0.0694

Maybe I need to create another Stream Engine?
Thanks.

Jean-Raphaël

question related to generation of streams with non-regular intervals

Hi,

Ref to

Streamserver use the files in the "dataset" directory to produce infinite stream data. The timetstamps must be orderedchronogically from older to newer. Streamserver use the timestamps only to simulate the intervals between two different tuples of the stream. The streamserver replaces the timestamps with current timestamps.

The event csv doesn't have a regular time interval for events. I'm struggling to get data out of this stream. Could that be the reason?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.