GithubHelp home page GithubHelp logo

technoweenie / twitter-node Goto Github PK

View Code? Open in Web Editor NEW
446.0 446.0 549.0 226 KB

Discontinued: check out nTwitter

Home Page: https://github.com/AvianFlu/ntwitter

License: MIT License

JavaScript 100.00%

twitter-node's Introduction

twitter-node

Creates a streaming connection with twitter, and pushes any incoming statuses to a tweet event.

Installation

Depends on ntest.

Use NPM:

npm install twitter-node

Otherwise create a symlink in ~/.node_libraries

$ ln -s /path/to/twitter-node/lib/twitter-node ~/.node_libraries/twitter-node

Events

TwitterNode emits these events:

  • tweet(json) - This is emitted when a new tweet comes in. This will be a parsed JSON object.
  • limit(json) - This is emitted when a new limit command comes in. Currently, limit detection only works with parsed JSON objects.
  • delete(json) - This is emitted when a new delete command comes in. Currently, delete detection only works with parsed JSON objects.
  • end(response) - This is emitted when the http connection is closed. The HTTP response object is sent.

See the streaming API docs for examples of the limit and delete commands.

Usage

// twitter-node does not modify GLOBAL, that's so rude
var TwitterNode = require('twitter-node').TwitterNode
  , util         = require('util')

// you can pass args to create() or set them on the TwitterNode instance
var twit = new TwitterNode({
  user: 'username', 
  password: 'password',
  host: 'my_proxy.my_company.com',         // proxy server name or ip addr
  port: 8080,							   // proxy port!
  track: ['baseball', 'football'],         // sports!
  follow: [12345, 67890],                  // follow these random users
  locations: [-122.75, 36.8, -121.75, 37.8] // tweets in SF
});

// adds to the track array set above
twit.track('foosball');

// adds to the following array set above
twit.follow(2345);

// follow tweets from NYC
twit.location(-74, 40, -73, 41)

// http://apiwiki.twitter.com/Streaming-API-Documentation#QueryParameters
twit.params['count'] = 100;

// http://apiwiki.twitter.com/Streaming-API-Documentation#Methods
twit.action = 'sample'; // 'filter' is default

twit.headers['User-Agent'] = 'whatever';

// Make sure you listen for errors, otherwise
// they are thrown
twit.addListener('error', function(error) {
  console.log(error.message);
});

twit
  .addListener('tweet', function(tweet) {
    util.puts("@" + tweet.user.screen_name + ": " + tweet.text);
  })

  .addListener('limit', function(limit) {
    util.puts("LIMIT: " + util.inspect(limit));
  })

  .addListener('delete', function(del) {
    util.puts("DELETE: " + util.inspect(del));
  })

  .addListener('end', function(resp) {
    util.puts("wave goodbye... " + resp.statusCode);
  })

  .stream();
  
// We can also add things to track on-the-fly
twit.track('#nowplaying');
twit.follow(1234);

// This will reset the stream
twit.stream();

Pre-Launch Checklist

See http://apiwiki.twitter.com/Streaming-API-Documentation. Keep these points in mind when getting ready to use TwitterNode in production:

  • Not purposefully attempting to circumvent access limits and levels?
  • Creating the minimal number of connections?
  • Avoiding duplicate logins?
  • Backing off from failures: none for first disconnect, seconds for repeated network (TCP/IP) level issues, minutes for repeated HTTP (4XX codes)?
  • Using long-lived connections?
  • Tolerant of other objects and newlines in markup stream? (Non objects...)
  • Tolerant of duplicate messages?

TODO

  • Handle failures as recommended from the Twitter stream documentation.

\m/

  • Tim Smart
  • Matt Secoske (secos)
  • kompozer
  • Twitter

Note on Patches/Pull Requests

  • Fork the project.
  • Make your feature addition or bug fix.
  • Add tests for it. This is important so I don't break it in a future version unintentionally.
  • Commit, do not mess with version or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
  • Send me a pull request. Bonus points for topic branches.

Copyright

Copyright (c) 2010 rick. See LICENSE for details.

twitter-node's People

Contributors

cmwelsh avatar cvee avatar dustin10 avatar icez avatar jdub avatar mnutt avatar secos avatar technoweenie avatar tim-smart avatar tracend avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

twitter-node's Issues

EPIPE when following lots of users

When I try and follow a large number of users (>2000), I get this error:

events:12
throw arguments[1];
^
Error: EPIPE, Broken pipe
at Client._writeImpl (net:300:14)
at Client._writeOut (net:731:25)
at Client.flush (net:793:24)
at IOWatcher._doFlush as callback
at node.js:764:9

I'm running twitter-node 0.0.2 and node 0.2.0. It's not possible to pin down an exact point where the cutoff is, but it looks like it's around 1800 in this instance. I'm thinking it's something to do with sending all the user IDs to Twitter, but beyond that I'm in the dark. Is there some kind of workaround for following large numbers of users?

Multiple instance issue

Doesnt work for multiple instance. If one user search for any stream that works, but if any other user logs in and search for something it doesnt work.

unable to install twitter-node

I get this error when I try to install twitter using npm, any workarounds?

npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
npm info preinstall [email protected]
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for program gcc or cc : /usr/bin/gcc
Checking for gcc : ok
Checking for node path : not found
Checking for node prefix : ok /usr/local
'configure' finished successfully (0.418s)
Waf: Entering directory /usr/local/lib/node/.npm/twitter-node/0.0.2/package/vendor/node-base64/build' [1/2] cxx: base64.cc -> build/default/base64_1.o ../base64.cc: In function ‘v8::Handle<v8::Value> base64_encode_binding(const v8::Arguments&)’: ../base64.cc:178: error: ‘class node::Buffer’ has no member named ‘data’ ../base64.cc:178: error: ‘class node::Buffer’ has no member named ‘length’ ../base64.cc: In function ‘v8::Handle<v8::Value> base64_decode_binding(const v8::Arguments&)’: ../base64.cc:199: error: ‘class node::Buffer’ has no member named ‘data’ ../base64.cc:199: error: ‘class node::Buffer’ has no member named ‘length’ Waf: Leaving directory/usr/local/lib/node/.npm/twitter-node/0.0.2/package/vendor/node-base64/build'
Build failed: -> task failed (err #1):
{task: cxx base64.cc -> base64_1.o}
cp: cannot stat build/default/base64.node': No such file or directory npm info [email protected] Failed to exec preinstall script npm ERR! install failed Error: [email protected] preinstall:./build.sh npm ERR! install failedsh "-c" "./build.sh"failed with 1 npm ERR! install failed at ChildProcess.<anonymous> (/usr/local/lib/node/.npm/npm/0.3.13/package/lib/utils/exec.js:49:20) npm ERR! install failed at ChildProcess.emit (events.js:45:17) npm ERR! install failed at ChildProcess.onexit (child_process.js:171:12) npm info install failed rollback npm info uninstall [ '[email protected]' ] npm info preuninstall [email protected] npm info uninstall [email protected] npm info auto-deactive not symlink npm info postuninstall [email protected] npm info uninstall [email protected] complete npm info install failed rolled back npm ERR! Error: [email protected] preinstall:./build.sh npm ERR!sh "-c" "./build.sh"` failed with 1
npm ERR! at ChildProcess. (/usr/local/lib/node/.npm/npm/0.3.13/package/lib/utils/exec.js:49:20)
npm ERR! at ChildProcess.emit (events.js:45:17)
npm ERR! at ChildProcess.onexit (child_process.js:171:12)
npm ERR!
npm ERR! Failed at the [email protected] preinstall script.
npm ERR! This is most likely a problem with the twitter-node package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! ./build.sh
npm ERR! You can get their info via:
npm ERR! npm owner ls twitter-node
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 2.6.32-28-generic
npm ERR! argv { remain: [ 'twitter-node' ],
npm ERR! argv cooked: [ 'install', 'twitter-node' ],
npm ERR! argv original: [ 'install', 'twitter-node' ] }
npm not ok

npm install fails

$ sudo npm install twitter-node
npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
npm info fetch http://registry.npmjs.org/twitter-node/-/twitter-node-0.0.2.tgz
npm info calculating sha1 /usr/local/lib/node/.npm/.cache/twitter-node/0.0.2/package.tgz
npm info shasum 8d81570e4c18a4cf7b9d884224e8b6fada764a1b
npm info preinstall [email protected]
Checking for program g++ or c++          : /usr/bin/g++ 
Checking for program cpp                 : /usr/bin/cpp 
Checking for program ar                  : /usr/bin/ar 
Checking for program ranlib              : /usr/bin/ranlib 
Checking for g++                         : ok  
Checking for program gcc or cc           : /usr/bin/gcc 
Checking for gcc                         : ok  
Checking for node path                   : not found 
Checking for node prefix                 : ok /usr/local 
'configure' finished successfully (0.050s)
Waf: Entering directory `/usr/local/lib/node/.npm/twitter-node/0.0.2/package/vendor/node-base64/build'
[1/2] cxx: base64.cc -> build/default/base64_1.o
../base64.cc: In function ‘v8::Handle base64_encode_binding(const v8::Arguments&)’:
../base64.cc:178: error: ‘class node::Buffer’ has no member named ‘data’
../base64.cc:178: error: ‘class node::Buffer’ has no member named ‘length’
../base64.cc: In function ‘v8::Handle base64_decode_binding(const v8::Arguments&)’:
../base64.cc:199: error: ‘class node::Buffer’ has no member named ‘data’
../base64.cc:199: error: ‘class node::Buffer’ has no member named ‘length’
Waf: Leaving directory `/usr/local/lib/node/.npm/twitter-node/0.0.2/package/vendor/node-base64/build'
Build failed:  -> task failed (err #1): 
    {task: cxx base64.cc -> base64_1.o}
cp: cannot stat `build/default/base64.node': No such file or directory
npm info [email protected] Failed to exec preinstall script
npm ERR! install failed Error: [email protected] preinstall: `./build.sh`
npm ERR! install failed `sh "-c" "./build.sh"` failed with 1
npm ERR! install failed     at ChildProcess. (/usr/local/lib/node/.npm/npm/0.3.9/package/lib/utils/exec.js:49:20)
npm ERR! install failed     at ChildProcess.emit (events.js:45:17)
npm ERR! install failed     at ChildProcess.onexit (child_process.js:171:12)
npm info install failed rollback
npm info uninstall [ '[email protected]' ]
npm info preuninstall [email protected]
npm info uninstall [email protected]
npm info auto-deactive not symlink
npm info postuninstall [email protected]
npm info uninstall [email protected] complete
npm info install failed rolled back
npm ERR! Error: [email protected] preinstall: `./build.sh`
npm ERR! `sh "-c" "./build.sh"` failed with 1
npm ERR!     at ChildProcess. (/usr/local/lib/node/.npm/npm/0.3.9/package/lib/utils/exec.js:49:20)
npm ERR!     at ChildProcess.emit (events.js:45:17)
npm ERR!     at ChildProcess.onexit (child_process.js:171:12)
npm ERR! 
npm ERR! Failed at the [email protected] preinstall script.
npm ERR! This is most likely a problem with the twitter-node package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     ./build.sh
npm ERR! You can get their info via:
npm ERR!     npm owner ls twitter-node
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 2.6.35-25-generic
npm not ok

calling twit.stream(); from within a function gives a connection refusal

If I call twit.stream(); from within a function (any function) I get a connection refused error:

node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: ECONNREFUSED, Connection refused
at Client._onConnect (net.js:595:18)
at IOWatcher.onWritable as callback

Can anyone guess why this would be?

TypeError: Object #<Object> has no method 'createClient' after npm update

First due to the ssl change I've tried to update twitter-node using npm update but nothing have been updated because the version is always 0.0.2 ... I copied index.js manualy !

But after that, I got this error :

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
TypeError: Object #<Object> has no method 'createClient'
    at EventEmitter._createClient (/home/vrabah/node_modules/twitter-node/lib/twitter-node/index.js:160:15)
    at EventEmitter.stream (/home/vrabah/node_modules/twitter-node/lib/twitter-node/index.js:111:22)
    at Object.<anonymous> (/home/vrabah/Documents/projet/node.js/mydev/server3.js:34:2)
    at Module._compile (module.js:411:26)
    at Object..js (module.js:417:10)
    at Module.load (module.js:343:31)
    at Function._load (module.js:302:12)
    at Array.<anonymous> (module.js:430:10)
    at EventEmitter._tickCallback (node.js:126:26)

Any idea ?
Thank's

Error: process.mixin() has been removed.

Running your example code with the newest version of node.js gives

node user_server.js
Error: process.mixin() has been removed.
    at EventEmitter.mixin (node.js:11:11)
    at EventEmitter.stream (/var/www/paul.slowgeek.com/twitter2facebook/twitter-node/lib/index.js:92:24)
    at Object.<anonymous> (/var/www/paul.slowgeek.com/twitter2facebook/user_server.js:47:4)
    at Module._compile (module:385:23)
    at module:413:20
    at fs:52:23
    at node.js:164:9

.end() isn't always called on previous stream when .stream() is called

var TwitterNode = require('../lib').TwitterNode
  , sys         = require('sys')
  , assert      = require('assert');


var twit = new TwitterNode({
  user: 'username', 
  password: 'password',
  follow: [12345, 67890],                  // follow these random users
});


twit
  .addListener('tweet', function(tweet) {
    sys.puts("@" + tweet.user.screen_name + ": " + tweet.text);
  })  

  .addListener('limit', function(limit) {
    sys.puts("LIMIT: " + sys.inspect(limit));
  })  

  .addListener('delete', function(del) {
    sys.puts("DELETE: " + sys.inspect(del));
  })  

  .addListener('end', function(resp) {
    sys.puts("wave goodbye... " + resp.statusCode);
  })  

  .stream();

setInterval(function() {
  var user = Math.floor(Math.random() * 10000000);
  sys.puts('Following '+user);
  twit.follow(user);
  twit.stream();
}, 1000);

Gives:

Following 9177227
Following 8287550
Following 5989591
wave goodbye... 200
Following 2983167
Following 9713078
wave goodbye... 200
Following 9638683
wave goodbye... 200
Following 7539247
wave goodbye... 200
Following 8320555
wave goodbye... 200
Following 4808738
Following 1493951
wave goodbye... 200
Following 7033565
wave goodbye... 200
Following 8097783
wave goodbye... 200
Following 8631714
wave goodbye... 200

The problem with this, is that now I get duplicate "tweet" events called for each existing connection that wasn't closed.

example throws errors

the .count() throws "Parameter count not allowed in role status".

The first location throws "Bounding box is invalid: 122.75, 36.80, -121.75, 37.80. The first pair must be the SW corner of the boxBounding box is invalid: 122.75, 36.80, -121.75, 37.80. The first pair must be the SW corner of the box"

and leaving the .sample() gives me a weird 416.

install fails

npm info it worked if it ends with ok
npm info using [email protected]
npm info fetch http://registry.npmjs.org/twitter-node/-/twitter-node-0.0.2.tgz
npm info install [email protected]
npm info preinstall [email protected]
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for program gcc or cc : /usr/bin/gcc
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for gcc : ok
Checking for node path : not found
Checking for node prefix : ok /usr/local
'configure' finished successfully (0.080s)
Waf: Entering directory /usr/local/lib/node/.npm/twitter-node/0.0.2/package/vendor/node-base64/build' [1/2] cxx: base64.cc -> build/default/base64_1.o ../base64.cc: In function ‘v8::Handle<v8::Value> base64_encode_binding(const v8::Arguments&)’: ../base64.cc:178: error: ‘class node::Buffer’ has no member named ‘data’ ../base64.cc:178: error: ‘class node::Buffer’ has no member named ‘length’ ../base64.cc: In function ‘v8::Handle<v8::Value> base64_decode_binding(const v8::Arguments&)’: ../base64.cc:199: error: ‘class node::Buffer’ has no member named ‘data’ ../base64.cc:199: error: ‘class node::Buffer’ has no member named ‘length’ Waf: Leaving directory/usr/local/lib/node/.npm/twitter-node/0.0.2/package/vendor/node-base64/build'
Build failed: -> task failed (err #1):
{task: cxx base64.cc -> base64_1.o}
cp: cannot stat build/default/base64.node': No such file or directory npm info [email protected] Failed to exec preinstall script npm ERR! install failed Error: [email protected] preinstall:./build.sh npm ERR! install failedshfailed with 1 npm ERR! install failed at ChildProcess.<anonymous> (/usr/local/lib/node/.npm/npm/0.2.3-3/package/lib/utils/exec.js:24:18) npm ERR! install failed at ChildProcess.emit (events.js:67:17) npm ERR! install failed at ChildProcess.onexit (child_process.js:192:12) npm info install failed rollback npm info not installed twitter-node,0.0.2 npm info install failed rolled back npm ERR! Error: [email protected] preinstall:./build.sh npm ERR!sh` failed with 1
npm ERR! at ChildProcess. (/usr/local/lib/node/.npm/npm/0.2.3-3/package/lib/utils/exec.js:24:18)
npm ERR! at ChildProcess.emit (events.js:67:17)
npm ERR! at ChildProcess.onexit (child_process.js:192:12)
npm ERR! npm install
npm ERR! npm install
npm ERR! npm install
npm ERR! npm install
npm ERR! npm install @
npm ERR! npm install @
npm ERR! npm install @
npm ERR!
npm ERR! Can specify one or more: npm install ./foo.tgz bar@stable /some/folder
npm ERR! Installs '.' if no argument supplied
npm ERR! try running: 'npm help install'
npm ERR! Report this entire log at http://github.com/isaacs/npm/issues
npm ERR! or email it to [email protected]
npm not ok

Error: Cannot find module 'twitter-node'

Hi...I've tried install your module using npm and downloading the git and creating the symbolic link too but always get this error

node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Cannot find module 'twitter-node'
at Function._resolveFilename (module.js:320:11)
at Function._load (module.js:266:25)
at require (module.js:348:19)
at Object. (/home/yo/Descargas/libros/rm_proj/sinatra_auth/public/twino.js:2:19)
at Module._compile (module.js:404:26)
at Object..js (module.js:410:10)
at Module.load (module.js:336:31)
at Function._load (module.js:297:12)
at Array. (module.js:423:10)

i don't get any error or warning when I install this using npm...but this don't work...it's really weird :s

Connection refused

Hiya,

I tried throwing this in a quick app, but am getting connection refused and I can't figure out why. I read thru the module. I guess twitter is refusing the connection?

Error: ECONNREFUSED, Connection refused
    at Client._onConnect (net.js:601:18)
    at IOWatcher.onWritable [as callback] (net.js:186:12)

I've tried variously tracking and following. I also tried using the NPM version and a repo-cloned version, which I noticed had some big differences, one of them being http vs https. Niether worked, both returned the same error above. I also tried running the module on its own, not wrapped in an express app, as below. The app, mostly from your example:

var express = require('express');

var TwitterNode = require('twitter-node').TwitterNode
  , sys         = require('sys');

var app = module.exports = express.createServer();


var twit = new TwitterNode({
  user: 'astromanies', 
  password: '###',
  host: '74....',  
  port: 8000,                          
  track: ['baseball', 'football'],    
});

twit.addListener('error', function(error) {
  console.log(error.message);
});

twit
  .addListener('tweet', function(tweet) {
    sys.puts("@" + tweet.user.screen_name + ": " + tweet.text);
  })

  .addListener('limit', function(limit) {
    sys.puts("LIMIT: " + sys.inspect(limit));
  })

  .addListener('delete', function(del) {
    sys.puts("DELETE: " + sys.inspect(del));
  })

  .addListener('end', function(resp) {
    sys.puts("wave goodbye... " + resp.statusCode);
  })
  .stream();

// Configuration

app.configure(function(){
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(__dirname + '/public'));
});

app.configure('development', function(){
  app.use(express.errorHandler({ dumpExceptions: true, showStack: true })); 
});

app.configure('production', function(){
  app.use(express.errorHandler()); 
});

app.listen(8080);
console.log("Express server listening on port %d in %s mode", app.address().port, app.settings.env);

npm package

Hi,

what about future npm package updates? (the latest package is currently 230 days old, and is broken because of twitter stream has moved to ssl)

Cheers,
Nandor

Updating filter should follow Twitter's predicates

When updating the filter track mid stream, the old connection should not be dropped until the new connection is opened and collecting data to avoid tweet data loss as detailed here: http://dev.twitter.com/pages/streaming_api_concepts#updating-filter-predicates

"Connect with your new predicate, wait for the first response, then immediately disconnect the old connection. Keep the window where you are connected twice to an absolute minimum. Sometimes the Streaming API service will disconnect the old connection as it begins to feed the new connection. This step probably requires a multi-threaded development environment, or at the very least, inter-process-communication (IPC) of some sort. But, once this technique is working well, the lost tweets should be practically zero."

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.