austinonrails / twitter2twilio Goto Github PK

Tweeter Keeper is built on top of other people's work - I didn't see any reason to reinvent the wheel.

TweetStream gem - reads tweets off of Twitter's streaming API and provides a Ruby block with the tweets.

Twitter gem - for use with the other Twitter API's - use to get followers for a user, user's tweet timeline, etc. We can use this to backfill in the history for users we follow on the streaming side. Some methods on this will require OAuth authentication

MongoDB - NoSQL database that uses BSON (similar to JSON) for inserting and retrieving data, making it easy to dump data directly from Twitter without processing. 

Mongo gem - Use the Mongo Ruby Driver directly, instead of an abstraction library

Yaml gem - Used for basic configuration tasks

# Install
=============

Environment - requires Ruby and MongoDB.

gem install bundle
bundle install


# Usage
==============

To run, open up config.yaml and change the USERNAME and PASSWORD to your Twitter account's username and password. 

bundle exec ruby tweeter-keeper.rb

Pick any tracking keywords you want, and add them to the @tracking_keywords array. 

This version has support for loading in one Twitter user's friends (the Twitter users they are following) and tracking their tweets as well - change the Twitter user's id from "jefflinwood" to your own or whomever. As long as the account isn't protected, you don't need the account's password or an OAuth authentication.

Open up a mongo shell and take a look at all the tweets you've collected! For sample data, I highly suggest "bieber" as a keyword.

> use tweeterkeeper

> db.tweets.count();
2156

# Future Plans
================
Get keywords directly from a MongoDB collection
Follow users from a MongoDB collection
Backfill user and keyword data using the REST API when a new user or keyword gets added
Handle errors/rate limits from the Twitter Streaming API
Auto-recovery if the stream gets dropped/errors