remind101 / kinesis

Go program and library for streaming to Amazon Kinesis.

Kinesis for Unix

This is a small Go library and unix command for streaming to an Amazon Kinesis stream.

Installation

$ go get -u github.com/remind101/kinesis

Usage

Stream a file to a kinesis stream.

$ cat /var/log/syslog | kinesis <stream>

Stream to a particular partition key.

$ cat /var/log/syslog | kinesis <stream> -p <partitionKey>

Go Library

The Go library exposes an io.Writer implementation that writes to an Amazon Kinesis stream.

import "github.com/remind101/kinesis/kinesis"

stream = bufio.NewWriter(kinesis.NewDefaultWriter("streamName", "partitionKey"))
io.Copy(stream, os.Stdin)

kinesis's Issues

Periodically flush buffered io

If no data is received for some period of time, the buffered io should be flushed.