GithubHelp home page GithubHelp logo

node-redis-pubsub's Introduction

NRP (Node Redis Pubsub)

This library is now mainly maintained by @rangermauve and @narcisoguillen

Simple pubsub for node using Redis. Why use NRP instead of Node's EventEmitter? It is useful when your Node application needs to share data with other applications. In that case EventEmitter will not help you, you need an external pubsub provider. Redis is pretty good at this, but its pubsub API is strange. So you use this wrapper.

Install and test

$ npm install node-redis-pubsub      # Install locally
$ npm install -g node-redis-pubsub   # Install globally
$
$ make test   # test (devDependencies need to be installed and a Redis server up)

Usage

Setup

for a trusted environment where Redis runs locally, unprotected on a port blocked by firewall.

var NRP    = require('node-redis-pubsub');
var config = {
  port  : 6379  , // Port of your locally running Redis server
  scope : 'demo'  // Use a scope to prevent two NRPs from sharing messages
};

var nrp = new NRP(config); // This is the NRP client

for a remote Redis server

var NRP = require('node-redis-pubsub');

var config = {
  port: 1234                        , // Port of your remote Redis server
  host: 'path.to.remote.redis.host' , // Redis server host, defaults to 127.0.0.1
  auth: 'password'                  , // Password
  scope: 'demo'                       // Use a scope to prevent two NRPs from sharing messages
};

var nrp = new NRP(config); // This is the NRP client

heroku and other services provide you with an environment variable REDIS_URL

var NRP = require('node-redis-pubsub');
var url = process.env.REDIS_URL;

var config = {
    url: url
};

var nrp = new NRP(config); // This is the NRP client

reuse existing redis pub and sub connections

var NRP = require('node-redis-pubsub');

var redisPub = redis.createClient();
var redisSub = redis.createClient();

var config = {
  emitter: redisPub,                      // Pass in an existing redis connection that should be used for pub
  receiver: redisSub,                     // Pass in an existing redis connection that should be used for sub
}

var nrp = new NRP(config); // This is the NRP client

Simple pubsub

nrp.on('say hello', function(data){
  console.log('Hello ' + data.name);
});

nrp.emit('say hello', { name: 'Louis' });   // Outputs 'Hello Louis'

// You can use patterns to capture all messages of a certain type
// The matched channel is given as a second parameter to the callback
nrp.on('city:*', (data, channel) => {
  console.log(data.city + ' is great');
});

nrp.emit('city:hello' , { city: 'Paris' });         // Outputs 'Paris is great'
nrp.emit('city:yeah'  , { city: 'San Francisco' }); // Outputs 'San Francisco is great'

Do something after subscribe finishes

nrp.on('mydata:sync', function(myData) {
  console.log(myData);
}, function() {
  nrp.emit('mydata:requestsync'); // request a sync of the data after the handler is registered, so there are no race conditions
});

Unsubscribe

var unsubscribe = nrp.on('say hello', function(data){
  // Never called
});

unsubscribe([Callback]);

Shut down connections

// Safely (connections will be closed properly once all commands are sent)
nrp.quit();

// Dangerously (connections will be immediately terminated)
nrp.end();

Listen for errors

nrp.on("error", function(){
  // Handle errors here
});

License

(The MIT License)

Copyright (c) 2012 tldr.io <[email protected]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

node-redis-pubsub's People

Contributors

af avatar assadvirgo avatar dotmilk avatar flybyme avatar gdelory avatar itamarwe avatar louischatriot avatar mafintosh avatar narcisoguillen avatar nerdbeere avatar nomadster avatar oliverwoodings avatar rangermauve avatar saintmac avatar seiyria avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-redis-pubsub's Issues

Issue with nested off() and on() calls

var program = require('commander')
var fs = require('fs')
var NRP = require('node-redis-pubsub');
var express = require('express')

program
  .version('0.0.1')
  .option("-p, --port [n]", parseInt)
  .option("-n, --number-of-hosts [m]", parseInt)
  .parse(process.argv)

var port = program.port || 80;
var hosts;
if(!program.numberOfHosts){
  console.warn("Number of hosts not set. Will run as single host")
  hosts = 1
} else {
  console.log("Will set number of host to " + program.numberOfHosts)
  hosts = program.numberOfHosts
}

var config = {
  port  : 6379  , // Port of your locally running Redis server
  scope : 'scope'  // Use a scope to prevent two NRPs from sharing messages
  //TODO, host : 'redis-host'
};
var masterPublishCannel = "Master has done publish"
var callbackChannelName = "Slave has done publish"

var exit = function(){ process.exit(1)}
var emitCallback = function(channel){
    console.log("Will emit on channel '" + channel + "'")
    nrp.emit(channel, {msg: "Done"});
}
var do_deploy = function(cb, channel){
  console.log("doing some fake actions...")
  setTimeout(function(){
    console.log("Fake actions done")
    cb(channel)
  }, '3000')
}
var SlaveHandler = function(data){
    console.log("Slave handler")
    do_deploy(emitCallback, callbackChannelName)
}
var sendResponse = function(res){
  console.log("Sending response")
  res.end("Thanks for you request")
}

var nrp = new NRP(config); // This is the NRP client
nrp.on(masterPublishCannel, SlaveHandler)

var app = express()
app.get("/doDeploy", function(req,res){
  nrp.off(masterPublishCannel, function(){
    console.log("Silenced channel '" + masterPublishCannel + "'")
    do_deploy(function(channel){
        console.log("Master Handler")
        var localNumberOfHosts = hosts
        console.log("localNumberOfHosts: " + localNumberOfHosts)

        if(localNumberOfHosts === 1){
          console.log("localNumberOfHosts is 1 exit")
          return sendResponse(res)
        }
        nrp.on(callbackChannelName, function(data){
          console.log("In handler of " + callbackChannelName)
          localNumberOfHosts--;
          console.log("# of hosts " + localNumberOfHosts)
          if(localNumberOfHosts === 1){
            console.log("All nodes published. Sending response");
            nrp.off(callbackChannelName, function(){
              console.log("Shutdown slave channel")
              console.log("Will listen again on '" + masterPublishCannel + "'")
              nrp.on(masterPublishCannel, SlaveHandler)
              //localNumberOfHosts = hosts
              sendResponse(res)
            });

          }
        })
        emitCallback(channel)
    }, masterPublishCannel);
  })
})


app.listen(port, function(){
  console.log("Service started on " + port)
})

Is it possible to expire listeners?

I want to use node-redis-pubsub in conjunction with websockets. Is it possible that the listeners expire after a defined period?

My usecase:
At the websocket server on each socket connection a unique token specified for that client will create some event listeners. These listeners listening on that token should be removed after a specified period.
Is it possible to add a expiration time? I don't want to have an increased amount of listeners stored for eternity, that never get executed, maybe due to a crash of my server instance.

This package is fairly new to me , but I know you can add an 'EX' (expiration time) to node-redis.

Thanks in advance!

redis 3 support

Redis 3 introduced support for rediss: protocol, for now I worked around this using a resolutions block:

"resolutions": {
  "node-redis-pubsub/redis": "^3.0.0"
},

and apparently there is no issue. I checked the breaking changes from redis 3 and I don't think it affects pubsub.

That would be nice if pubsub would natively migrate to redis 3, I'll submit a PR for this even if it's one line change

Using tls in redis client.

Hello!
Can you provide some example of using tls with the library? I'm trying to use tls options, but it doesn't work (looks so):
const client = new NRP({ host: 'aws_redis', auth: 'pass', tls: { servername: 'aws_redis' } })
What am I doing wrong here?
Thanks!

Allow for on('connect'

Hi,
Thank you for the awesome project.
Currently your code allows for us to provide an error handler if the clients cannot connect to redis.
The code here provides this functionality.
if(channel === "error"){ self.errorHandler = handler; this.emitter.on("error", handler); this.receiver.on("error", handler); callback(); return; }

It would be amazing if we had similar code that would let us know when the handler has connected something akin to
if(channel === "connect"){ self.connectHandler = handler; this.emitter.on("connect", handler); this.receiver.on("connect", handler); callback(); return; }

Right now my code looks like this
const handler = new NRP({ url: config.redis.url }); let connected = true; function connectError(error) { if (!connectError.loggedOnce) { logger.error(error.message); connected = false; connectError.loggedOnce = true; } } // This is never called function connectSuccess() { if (!connectSuccess.loggedOnce) { logger.info('Redis Connected: ', config.redis); connectSuccess.loggedOnce = true; } } handler.on('error', connectError); handler.on('connect', connectSuccess);

Because there is no connect handler, the code has to assume the connection is good and sets the connected variable to true by default. If I was unsure of my connection and I wanted to only set the variable once connected, or if I wanted to log that a connection was made I cannot do so.

In my local env I tried adding the code above to trigger the connection handler and it worked.

Thank you,
Jonathan

library should support non json messages too

with the current version, library test for the json messages and if its not a valid json it simply throws the error. However if we want to subscribe to events like "expired" they don't return a json object.

.quit hanging node on exit

I can see that this is calling quit on the redis library. I believe you also want to call .unref() on the connections, see npm redis module doc, in order to allow node to close.

Presently only .end() works.

Add maintainers

Though you might not be interested in supporting this library anymore, there might be people still interested in progressing it.

I volunteer to be a maintainer, and I think that @narcisoguillen might be interested too.

This'd just require collaborator status on here, and publishing rights on npm.

redis connection pooling support?

Hi,
Is there a connection pooling support for the pubsub ?
I tried to use generic pool of redis clients and it does not seem to work..

Want to know how we could get the pubsub working in case the redis connection is lost, without restarting the server.

Cluster support?

Hi there what about connecting to 10 cluster servers, there is any way to do it?

wait for subscription before doing an emit

hi, I'm working with this library and I have a situation in which I do a bunch of subscribes (to set up some data handling functions for a particular instance). after I do the subscribes, I do an emit which requests all of the data from another instance.

my problem is that the emit is firing before the subscriptions have happened. could they be made to return a promise when the subscription is complete?

thanks!

is this repo production ready?

I am about to use this in production and want to know if there are any known issues? is there a competing library that is more suitable at this time? thanks for this module, works great so far

Node crashes on "Unexpected token"

Hello,

I'm getting a strange error when subscribing to a channel. My script randomly crashes when receiving a message. I can't even log the message because it crashes before calling my handler. This does not happen everytime, but randomly on a message I couldn't identify yet. The crash report is :

events.js:68
        throw arguments[1]; // Unhandled 'error' event
                       ^
SyntaxError: Unexpected token #
    at Object.parse (native)
    at RedisClient.NodeRedisPubsub.on ({dir}\node_modules\node-redis-pubsub\lib\node-redis-pubsub.js:37:20)
    at RedisClient.EventEmitter.emit (events.js:123:20)
    at RedisClient.return_reply ({dir}\node_modules\node-redis-pubsub\node_modules\redis\index.js:641:22)
    at ReplyParser.RedisClient.init_parser ({dir}\node_modules\node-redis-pubsub\node_modules\redis\index.js:282:14)
    at ReplyParser.EventEmitter.emit (events.js:93:17)
    at ReplyParser.send_reply ({dir}\node_modules\node-redis-pubsub\node_modules\redis\lib\parser\javascript.js:300:10)
    at ReplyParser.execute ({dir}\node_modules\node-redis-pubsub\node_modules\redis\lib\parser\javascript.js:211:22)
    at RedisClient.on_data ({dir}\node_modules\node-redis-pubsub\node_modules\redis\index.js:504:27)
    at Socket.<anonymous> ({dir}\node_modules\node-redis-pubsub\node_modules\redis\index.js:82:14)

It indicates unexpected token #, but many of my messages contains a # . Sometimes it crashes right at the first message, sometimes it handles several messages before crashing. Any idea ?

Add ability to remove listeners

A thing that I like about EventEmitter is that you can remove listeners on different events to conserve resources, or even just list the number of listeners listening on an event. It'd be nice to have these features in NRP as well.

pub/sub upon update of key/value pair

my goal is to use Redis pub/sub (hopefully using this library) to send messages upon an update or insert of a Redis key.

Does this library go low-level enough for that? How can I invoke an nrp.emit upon a Redis key/value update?

Is the best way to do this using keyspace notifications?

http://redis.io/topics/notifications

Please let me know, thanks

Subscribe handler - channel argument

Hi,

I'm using the latest version.
I realised that you never have to take into consideration the 'scope' value (prefix) except in the handler.

When you subscribe and notify, you just give it the desired channel, but when you handle a message, you have to manually parse the channel.

I think you could improve this by simply parse the channel and removing the prefix since it's only intended as an internal setting.

Thanks!

Best regards,
Rui Freitas

On several occasions when I pass the same message key with with different payload I get old payload results.

Example:

nrp.on('api/*', function (data) {
console.log('result is: '+ data);
});

first time:
nrp.emit('api/', { data: 'some string' });
result "result is some string"

second time:
nrp.emit('api/', { data: 'some new string' });
result "result is some string"

The issue is intermittent. I will add more details if I manage to reproduce it consistently.
Could it be because my prefix is not ":" but "/" ?

PUBLISH from redis not captured

Hello!

So I was fooling around with REDIS and node-redis-pubsub. I started two redis-cli instances locally and ran two node scripts like the above:

var NRP = require('node-redis-pubsub');

var nrpConfig = {
  port: 6379,
  host: '127.0.0.1'
};

var nrp = new NRP(nrpConfig); // This is the NRP client

console.log("Listening to channel test");


nrp.on(
  'test',
  function(data){
    console.log('DATA RECEIVED: ' + data);
  }
);

In one redis-cli, I subscribed to channel test

From the other I publish to channel test and see the results ONLY IN THE REDIS CLI SUBSCRIBED INSTANCE, but nothing happens from the script. If I publish to the same channel from node environment, the script suddenly works and I see the published message.

Any clues why this might be happening ?

Uncaught error in handler causes incorrect "invalid JSON received" error

This block has both the JSON.parse() call and the handler() call in the try block. An uncaught error thrown within the handler will cause the catch block to be called, even though it's not a JSON parsing error. This confused me and I had to look up the error in the source as the logged JSON seemed fine.

It should be adjusted so only the JSON.parse() happens within this block. Otherwise, uncaught errors shouldn't be caught so they fall back to the debugger.

Unsubscribe function unsubscribes all the listeners for the same channel pattern

Just to point out a behavior of this library that is not obvious and that made me wrong.
If you subscribe for the same channel pattern multiple times, and then you call just the first unsubscribe function, then all the other listeners will never receive a message, since the redis client has been unsubscribed from that channel pattern.

In code words:

var unsub1 = nrp.on('say hello', function(data){
  console.log('First Hello ' + data.name);
});

var unsub2 = nrp.on('say hello', function(data){
  console.log('Second Hello ' + data.name);
});

// then at next tick or later
// ...

nrp.emit('say hello', { name: 'Louis' });   // Outputs 'First Hello Louis' and 'Second Hello Louis'

// let's unsubscribe just the first handler.
unsub1();

nrp.emit('say hello', { name: 'Louis' });   // It should output 'Second Hello Louis', but it won't

use redis:// urls

I was looking at using this lib, but saw that it doesn't support the redis:// protocol urls. Not that it's a dealbreaker to have to parse it myself, but it would be a lot easier if I didn't have to.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.