socketcluster / socketcluster

Highly scalable realtime pub/sub and RPC framework

SocketCluster

Toolset and boilerplate for quickly creating systems using SocketCluster. See the client and server repos for documentation:

• https://github.com/SocketCluster/socketcluster-client
• https://github.com/SocketCluster/socketcluster-server

Documentation for SCC (horizontally scalable cluster) is available at https://github.com/SocketCluster/socketcluster/blob/master/scc-guide.md

Installation

Setup the socketcluster command:

npm install -g socketcluster

or:

sudo npm install -g socketcluster

then:

socketcluster create myapp

Once it's installed, go to your new myapp/ directory and launch with:

node server

Access at URL http://localhost:8000/

Node.js v10.0.0 or above is recommended but you can also use SocketCluster with older Node.js versions if you use while loops instead of for-await-of loops.

Compatibility mode

For compatibility with existing SocketCluster clients, set the protocolVersion to 1 and make sure that the path matches your old client path:

let agServer = socketClusterServer.attach(httpServer, {
  protocolVersion: 1,
  path: '/socketcluster/'
});

Change log

See the 'releases' section for changes: https://github.com/SocketCluster/socketcluster/releases

License

(The MIT License)

Copyright (c) 2013-2023 SocketCluster.io

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

socketcluster's Issues

How to perform zero downtime deploy?

Am about to use SocketCluster in production.

I wondering if SocketCluster implement something like Naught internally?

If not, do you have any tool recommendation for this usage?

Would love to see a socketcluster client sample using nodejs

I am planning to run nodejs scripts on embedded controllers, which propagate realtime data to socketcluster channels. Current samples show in detail how to do that from browser script.

a simple sample what to require and write to channels would do the jump start here.

thanks

Simple HTTP server for debug mode

It would useful to have a simple HTTP server: only 1 process (master) for debug purpose.

We could run a test version alongside the production version with:

node server_debug.js --debug

Maybe this feature already exist?

Current Plan for SocketCluster V2

I just thought I should clarify the current plan for SC v2.
If you have any comments or suggestions, feel free to post them here: #48

As of now, the idea is that V2 will be almost entirely backwards compatible with v1.
Also VERY IMPORTANT to note is that v2 will be optional! Both v1 and v2 will be supported at the same time until WebSockets becomes adopted by more browsers (right now it appears to have about 85% penetration: http://caniuse.com/#search=WebSocket).

The idea is that Developers will be able to seamlessly move their project directly from v1 to v2 without having to change any code (assuming that you don't use the session API). The session API ( http://socketcluster.io/#!/docs/api-session and ) will be marked as deprecated.

The only changes planned right now are these:

• Removing the session object on sockets - If you want to send messages to a user, (across all open tabs for example) you can just send a message to one of their personal channels.
• The BIG one: v2 will use plain WebSockets (no fallback) - The efficiency improvements of using WebSockets is huge - The engine.io long-polling handshake is VERY expensive in a clustered environment like SC.
• Not having to deal with long-polling fallbacks means that we can get rid of load balancer processes entirely (because WebSockets ensure that a user will stay bound to the same worker - We don't have to keep routing EVERY poll request to the correct worker).

The Pros:

• Huge performance gains - I'm hoping for at least a 5x performance improvement over the current version of SC (particularly when it comes to handling new connections). Load balancers currently consume a lot of CPU power so getting rid of them gives a huge speedup.
• Lighter architecture with more focus around pub/sub - Encourages good design.
• SC itself will have simpler source code

The cons (only one I can think of right now - A big one though):

• No WebSocket fallback - You should stick to the current version of SC for a bit longer if you need to support older browsers or proxies (which can be a problem if you don't use SSL - See http://www.infoq.com/articles/Web-Sockets-Proxy-Servers).

need to have more control over clients reconnect events

In my app, users are authenticated before they are able subscribe to a private channel. The sequence for achieving this is be having the client emit("auth",token) upon establishing a connection. Only after the server responds with a greeting (after verifying the provided token and tagging the session), users get to subscribe to the privateChannel channel.

// on the client side:
socket.on('connect', function () {
        socket.emit('auth', {uId: 1412, token: 'secretToken'});
     });

socket.on('greeting', function (data) {
        socket.subscribe("privateChannel");
})

The problem I have is when the server goes down or crashes, the client attempts to reconnect and re-subscribes to its old channels. In my app, the authentication sequence gets broken since the socket should only attempt to subscribe to privateChannel after receiving the greeting message.

What's the best way for handling the above problem ? Is there a way to disable auto re-subscribe upon re-connection ? or at least have more control over it ?

Cyclic timeout

I create one test app and detect cyclic timeout on client side PING event.
Main test is always timeout ping event after 10 item success.

Server run localhost ubuntu14 64bit, client use firefox 33 NodeJs 0.10.29 SC update 2014.11.10.

Client:

<script type="text/javascript"> var ChatApp = function(params){ var self = this; // Initiate the connection to the server self.socket = socketCluster.connect(); self.authObj = new Object({ username:'anonymus', password:'secret', roomid:'123456789' }); self.socket.on('error', function (err) { throw 'Socket error - ' + err; }); self.socket.on('connect', function (socketId) { console.debug('Connected:'+socketId); self.commObj = new Object({ token:0, sid:0, channel:0 }); self.socket.emit('auth',self.authObj,function(err,res){ if (err) console.log('Auth error:'+err); if (res.status == 'success'){ console.log('Auth successfull'); self.commObj.token = res.token; self.commObj.sid = self.socket.ssid; self.socket.subscribe(self.authObj.roomid, function (err) { if (err) { console.log('Failed to subscribe to room event channel:'+self.authObj.roomid); }else{ self.commObj.channel = self.authObj.roomid; } }); }else{ console.log('Auth failed, status:'+res.status); } }); }); self.socket.on('rand', function (num,res) { res.end(); console.log('RANDOM:', num); if (num.rand) { document.body.innerHTML = 'RANDOM: ' + num.rand + '
'; } }); self.socket.on('pong', function (num,res) { res.end(); console.log('PONG:', num); }); setInterval(function () { if (self.commObj.channel){ self.socket.emit('ping',self.commObj,function(err){ if (err){ console.log('Emit error:'+err); if (err.err == 'unauthorizedSession'){ self.socket = socketCluster.connect(); } } }); } }, 10000); }; var chat1 = new ChatApp(); </script>

Server:
var fs = require('fs');
var express = require('express');
var serveStatic = require('serve-static');
var log4js = require('log4js');
log4js.configure('log4js_configuration.json',{});

module.exports.run = function (worker) {
var log = log4js.getLogger('SCW_'+process.pid);

log.info('>> Worker PID:', process.pid);
var app = require('express')();

// Get a reference to our raw Node HTTP server
var httpServer = worker.getHTTPServer();
// Get a reference to our realtime SocketCluster server
var scServer = worker.getSCServer();

app.use(serveStatic(__dirname + '/public'));

httpServer.on('req', app);

var activeSessions = {};

var count = 0;
//Mielőtt átadódik a socket.connection-ra itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_HANDSHAKE, function (req, next) {
    log.debug('Handshake');
    next();
});
//Mielőtt átadódik a socket.subscribe-ra itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_SUBSCRIBE,
    function (socket, channel, next) {
        socket.session.get('isUserAuthorized', function (err, value) {
            if (err){
                log.warn('Session error:'+JSON.stringify(err));
            }
            if (value) {
                next();// Allow
            } else {
                next({err:'unauthorizedSession',sid:socket.session.id,channel:channel}); // Block
            }
        });
    }
);
//Mielőtt átadódik a socket.publish-ra itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_PUBLISH,
    function (socket, event, data, next) {
        socket.session.get('isUserAuthorized', function (err, value) {
            if (err){
                log.warn('Session error:'+JSON.stringify(err));
            }
            if (value) {
                next();// Allow
            } else {
                next({err:'unauthorizedSession',sid:socket.session.id,event:event}); // Block
            }
        });
    }
);
//Mielőtt átadódik a socket.emit-re itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_EMIT,
    function (socket, event, data, next) {
        log.debug('MIDDLEWARE_EMIT:'+event+JSON.stringify(data));
        if (event == 'auth'){
            //use unauthorized emits
            log.debug('Enable AUTH event');
            next();
        }else{
            //authorize required
            socket.session.get('isUserAuthorized', function (err, value) {
                if (err){
                    log.warn('Session error:'+JSON.stringify(err));
                }
                if (value) {
                    log.debug('Session authorized, enable '+event+' event');
                    next();// Allow
                } else {
                    log.debug('MIDDLEWARE_EMIT'+JSON.stringify({err:'unauthorizedSession',sid:socket.session.id,event:event}));
                    next({err:'unauthorizedSession',sid:socket.session.id,event:event}); // Block
                }
            });
        }
    }
);

scServer.on('connection', function (socket) {
    var doLog = new Object({
        clientIp:socket.clientAddress,
        socketId:socket.id,
        clientSessionId:socket.ssid,
        cookie:socket.cookie,
        header:socket.request.headers
    });
    log.info('Connected:'+JSON.stringify(doLog));

    socket.on('auth',function(data,res){
        log.debug('Auth:'+JSON.stringify(data));
        if (data.username && data.password && data.roomid){
            var token = 'TOKEN123456';
            res.end({status:'success',token:token});
            log.debug('Set session auth to '+socket.session.id+', token:'+token);
            socket.session.set('isUserAuthorized', true, function(err){
                if (err){
                    log.warn('Session error:'+JSON.stringify(err));
                }
            });
            activeSessions[socket.session.id] = socket.session;
        }else{
            res.end({status:'failed'});
        }
    });


    socket.on('ping', function (data,req) {
        //req.end();
        count++;
        log.debug('PING'+JSON.stringify(data));
        scServer.global.publish('pong', count);
    });
});

scServer.on('sessionend', function (ssid) {
delete activeSessions[ssid];
});

setInterval(function () {
/*
Emit a 'rand' event on each active session.
Note that in this case the random number emitted will be the same across all sockets which
belong to the same session (I.e. All open tabs within the same browser).
*/
for (var i in activeSessions) {
activeSessions[i].emit('rand', {rand: Math.floor(Math.random() * 100)});
}
}, 1000);
};

Server log:
vargab@vargab-laptop:~/munka/webdav/nodeserver$ node server.js
[Busy] Launching SocketCluster

Store PID: 7096
[2014-11-13 08:28:14.924] [INFO] SCW_7099 - >> Worker PID: 7099
Balancer PID: 7109
[Active] SocketCluster started
Port: 8000
Master PID: 7093
Balancer count: 1
Worker count: 1
Store count: 1

[2014-11-13 08:28:16.410] [DEBUG] SCW_7099 - Handshake
[2014-11-13 08:28:16.523] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:auth{"username":"anonymus","password":"secret","roomid":"123456789"}
[2014-11-13 08:28:16.523] [DEBUG] SCW_7099 - Enable AUTH event
[2014-11-13 08:28:21.968] [DEBUG] SCW_7099 - Handshake
[2014-11-13 08:28:22.444] [INFO] SCW_7099 - Connected:{"clientIp":"127.0.0.1","socketId":"_w0_8000_0_gu1LzinLIZxbJqMbAAAB","clientSessionId":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","cookie":{"n/app/io":"_w0_8000_0_vVoA-uKPs3w_llLbAAAA","ga":"GA1.1.366504261.1415259035","n/app/ssid":"w0_8000_0_EUFeHXRWgRgwBQeHAAAA"},"header":{"host":"nodeserver:8000","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0 FirePHP/0.7.4","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8","accept-language":"hu-hu,hu;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate","dnt":"1","referer":"http://nodeserver:8000/","cookie":"n/app/io=_w0_8000_0_vVoA-uKPs3w_llLbAAAA; _ga=GA1.1.366504261.1415259035; n/app/ssid=_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","x-insight":"activate","connection":"close","x-forwarded-for":"127.0.0.1","x-forwarded-port":"8000","x-forwarded-proto":"http"}}
[2014-11-13 08:28:22.515] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:auth{"username":"anonymus","password":"secret","roomid":"123456789"}
[2014-11-13 08:28:22.515] [DEBUG] SCW_7099 - Enable AUTH event
[2014-11-13 08:28:22.515] [DEBUG] SCW_7099 - Auth:{"username":"anonymus","password":"secret","roomid":"123456789"}
[2014-11-13 08:28:22.515] [DEBUG] SCW_7099 - Set session auth to _w0_8000_0_EUFeHXRWgRgwBQeHAAAA, token:TOKEN123456
[2014-11-13 08:28:31.967] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:28:31.969] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:28:31.970] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:28:41.973] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:28:41.974] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:28:41.974] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:28:51.968] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:28:51.969] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:28:51.969] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:01.973] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:01.974] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:29:01.974] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:11.972] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:11.973] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:29:11.973] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:21.968] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:21.969] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:29:21.969] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:31.967] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:31.970] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:29:31.970] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:41.971] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:41.973] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:29:41.974] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:51.967] [DEBUG] SCW_7099 - MIDDLEWARE_EMIT:ping{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}
[2014-11-13 08:29:51.980] [DEBUG] SCW_7099 - Session authorized, enable ping event
[2014-11-13 08:29:51.980] [DEBUG] SCW_7099 - PING{"token":"TOKEN123456","sid":"_w0_8000_0_EUFeHXRWgRgwBQeHAAAA","channel":"123456789"}

Implement Redis plugin to sync internal SocketCluster Pub/Sub Channels with external Redis channels

This will allow external systems to interact with SC pub/sub via Redis

Session destroyed or disconnect

Hello!
I testing SC and have a problem:
Main client is bind 'click' and emiting event on Socket.
If binding next click and emiting again, the client is disconnect.
On server side call sessiondestroy and disconnection.
Client is resonnect and begining again this loop.

Client:
$('#users-list').on('click','.action>a',function(){
var o = auth;
o.subClientId = $(this).parent().attr('data-id');
console.log(o);
if ($(this).hasClass('av-on')){
o.sub = 'webcamOn';
}
socket.emit('new-action',o,function(cb){console.log(cb);} );
});
//this concole.log is logging: EventResponseTimeout after sending event

Server:
var ChatServer = function(options){
var self = this;
self.io = options.io;
self.activeSessions = {}; //global session storage
self.socketData = {};
self.debug = 1;
self.init = function() {
console.log(options.w+'->Init chat server instance...');
//REDIS init
self.redisStore = redis.createClient();
self.redisStore.select(0); //select DB
self.redisStore.on("error", function (err) {
console.log("Redis error " + err);
});

    // Fired upon a connection
    self.io.on("connection", function(socket) {
        self.handleConnection(socket);
    });

    self.io.on('disconnection', function (socket) {
        console.log('Socket ' + socket.id + ' was disconnected, destroy redis');
        self.socketData[socket.id].destroyRedis();
        delete self.socketData[socket.id];
    });

    self.io.on('sessiondestroy', function (ssid,socket) {
        console.log('Client leave room, session destroyed SSID:'+ssid);
        self.redisStore.srem('confserver:rooms:'+currentRoom.id+':node_sessions',ssid);
        self.redisStore.del('confserver:rooms:'+currentRoom.id+':node:'+ssid);

        delete self.activeSessions[ssid];
    });
    console.log('ChatServer:'+options.w+'->instance inited');
}

On server console put disconnect after put sessiondestroy.

Howto debuging this problem?
Wher is a problem, clientside or server side?

I use currently SC 0.9.53

Balazs

Handling HTTP & HTTPS

How I can handle/allow 2 connections (https&http) to my SC server?

Multi socket cluster

How would you handle scalability passed one big multicore server?

Accessing sessions, passport

In this ticket (Express-session and PassportJS) you mentioned using socketclusters session store all the way, but how would I go about doing that?

So my use case is pretty much the same as in that ticket, using passport to authenticate users (with google, facebook etc).

One socket channel (admin) will need users to be authenticated, the rest of the channels won't have that demand. The auth-part works fine if I use mongodb as a session store, but then socketcluster isn't able to read the sessions.

When using mongo to keep track of sessions I can find them in a req.user-object. However, I can't seem to find a way to access express' req-object from inside a socket (socket.on('admin', function () {...}))

Now I'm trying to use socketclusters ndata, as you wrote in the other ticket, to have the same shared session store both for http and real time communications.

Which store do I tell the session-middleware to use?

var app = require('express')();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(methodOverride());
app.use(cookieParser());
app.use(session({
    secret: 'secret',
    store: scServer.???? //using mongodb store here works great
}));
app.use(passport.initialize());
app.use(passport.session());

Or do I skip the express-session stuff completely? In that case, how do I access the socketclusters sessions store from the auth-callback from ie google

app.get('/auth/google/callback',
    passport.authenticate('google', { session: true, failureRedirect: '/login'}),
        function(req, res) {
            //works with mongo, but what should I do to access the correct socketcluster sessionsstore
            req.session.user = req.user;
            res.redirect('/');
            return;
        }
    );

Or am I way off here? Been sitting with this problem way too long now =)
#18

XHR access-controll

Hello!

I try latest version and have a problem this:

XMLHttpRequest cannot load http://maindomain.eu:8000/engine.io/?EIO=2&transport=polling&t=1407669750102-0&b64=1. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://maindomain.eu' is therefore not allowed access. create:1
Chat server disconnect!

I use sample app. configure, and try change origins:maindomain.eu

not working. Tested Chrome 36.0.1985.125

previous version is run good..., unfortunatly i droped oldest version.

The connection is not success.

HTTP Server support for SPDY

I was wondering if it's possible to support the SPDY protocol in scWorker.js with node-spdy

I don't see any caveats?

Can't open more than x sockets with server from a single client...

Thanks to Cyrus for notifying me of this issue:

Basically, when trying to stress test SocketCluster with a single client, you might run into the issue of new sockets not connecting past a certain number (e.g. 20).

Tiny server

Hello,

I discovered that my managed server have only 2 cores (cmd: nproc: 2)

Does using socketcluster with 2 workers will be beneficial for this server?

Problem terminating

Hi again, I'm trying to create an init script for my SC server. The problem I've encountered is when I want to stop the server I send an interrupt signal to the master process it terminates itself but not its child processes.

It terminates as it should when i run the server on the command line and send the signal with ctrl + c.

RFC: API changes suggestions

Please, feel free to add your suggestions here! Also, comments are requested.

Even if your suggestion is extremely small and even if you want to send just '+1' or '-1', please, do it.

Rooms question

Hi,

I'm actually using socket.io right now and I use rooms/private channels a lot in my application.
It works fine until now that I want to scale it, therefore I'm looking for replacement.
I think socketcluster should fit my needs but I'm wondering if it supports rooms feature?

Thanks.
Samuel

How to get updates from an external (php) application ?

My application is written in php. I experimented with Ratchet (http://socketo.me/) for running a websocket server. However, I am finding SocketCluster much more attractive mainly due to its scalability features. With Ratchet, I used ZeroMQ message for communication between my php application and Ratchet. They have put up a simple guide here http://socketo.me/docs/push on how to do so. It was very simple to achieve.

Is it possible to have a similar implementation with SocketCluster ? What is the best way for communicating between an external app and SocketCluster ?

Migration from Socket.io

I am looking to migrate to SocketCluster from Socket.io and I was wondering how to do authentication or implementing such middleware(s) is supported.

I'm not sure if sending ArrayBuffers is a socket.io or a engine.io feature, but is sending ArrayBuffers supported? When I tried to send via SocketCluster, I am getting an empty response.

Sending data on connection

Have feathure sending data on connection process or only separate event?

eg. client:
var socket = socketCluster.connect({custom-dta:{username:'bob',password:'secret',foo:bar}});

on SC:
scServer.addMiddleware(scServer.MIDDLEWARE_HANDSHAKE,function(req,next){
if (req.custom-data){
if (req.custom-data.username == ... && req.custom-data.password == .... ) next();
else next('Unauthorized user');
}else next('Not receive custom-data');
});

Or custom-data parsing on scServer.on('connection',function(socket,data){
if (data.custom-data)....
});

Or none this feathure and requiresending separate event after connection is success eg.: socket.emit('auth',data)...

messages fail to deliver in v2.1

I just upgraded to sc2.1.4 and noticed that for some reason messages published in channels don't get delivered to subscribed clients anymore. The problem is not present in 2.0.28 and is present in 2.1.0.

1428570453888 - Origin: Worker (PID 26338)
    [Notice] Failed to deliver message 'a42d83e2-1061-4143-8fe9-d9372758fe6a' to 1 subscriber on the '["chat","onlinelist","courses/chocolate/Cohorts/ClassOf2014"]' channel

SC load monitoring, clients listening

Hello!

Howto listing, monitoring SC on live?

eg.: listing global clients data as live(ip,sid,ssid,cookie,browser,custom-data) and collate/group as worker, namespace...etc, and monitoring process time, load, memory....etc.

This is good feathure to administrative monitoring main system. I use one very old chatroom server and good plan the admin monitor (listing and parsing clients eg.: error finding, service supporting to clients...etc).

Sockets storing and accessing global

Hello!
I require many minor fethures:

I use socketcluster to multichatroom. Problem, the socket.session is common a many socket in browser (eg.: teil1: i create on room and login moderator and attendee separate tab, common session created. If socket.session.emit is send any. This correct: i send socket.emit. Teil2: I create many room in separate tab, all session is common, tabs/rooms not separated.). Problem: no common labeled store the socket identifying.

Require a external memory store, have accessing all worker to do. this store provide common data to workers/sockets communications. eg.: Client create room, save a key-val into globalStore. Many clients socket id is connect to this (simple object model). If get a new event to socket, find the globalStore all subscribed socket and broadcasting to, or get individual socket and emitting to eg.: private chat.

I try subscribe/unsubscribe undefined now.

I find sessionId customizing, not allowed now.

Or how can identifying simultane all socket and recalling sockat as socket-id?
(Try storing sockets-id in redis and parsing subscribeing, try reloading subscreibed sockets to broadcast...etc)?

Pleas help me creating this feathures!

Regards Balazs

socket publish and broadcast

Hello,

Here is my question description:
If user1 subscribe channel "test", and then publish message to "test", does user1 receive message that he just sent ? If the answer is "YES", can I have any method to make user1 not to receive any data that he just published ?

After read documents and issue #15, it seems that I can not filter data on
Pub/Sub feature.

Thanks for your help :)

btw, socketCluster is really awesome 👍

Connection problems with SocketRocket iOS

Hello guys, okay?

I'm doing a iOS app that will communicate by websockets, I'm using SocketCluster as my server and SocketRocket as the client in my application.

I managed to connect to the server and receive data from it, as follows:

Connecting to websocket at: Optional(ws://localhost:8000/socketcluster/?transport=websocket)
Websocket connected!!
DidReceiveMessage: 0{"sid":"_w0_8000_0_WvzMBaK_FpjIF4hLAAAG","upgrades":[],"pingInterval":25000,"pingTimeout":60000}
DidReceiveMessage: 4{"event":"connect","data":{"soid":"_w0_8000_0_WvzMBaK_FpjIF4hLAAAG","appName":"app"},"cid":1}

However the server does not fire the event: 'connection'.
Could someone help me?

RFC: Additional API for both sides. Channel entity on the server-side (socketcluster-v2)

moved to #48

How To Scale SocketCluster Across Multiple Machines?

Hi there. I'm looking to scale SocketCluster in production across multiple machines horizontally.
I read your response to another user (#12) on the role that NData plays in SC, but am not sure how to scale.

1. Would SocketCluster be scaled the same way socket.io is scaled across multiple machines using redis pub/sub,MessageQueue, HAProxy?

2. Would the 'SocketCluster' cluster object need to be configured in any way for multiple servers, for example I notice the sample code has the following settings:

balancers: Number(argv.b) || 1,
workers: Number(argv.w) || 1,
stores: Number(argv.s) || 1,

But what would determine the number '1' here? The number of cpu cores on each machine? Or how many servers we are trying to scale horizontally with?

3. Also does SocketCluster still require the built-in node cluster module (http://nodejs.org/api/cluster.html) to be used, to access multiple cpu cores?

Thanks.

Socket session, sessionEnd,kikOut problem

I try SC and have a many session problem.
If user leave room (close browser) scServ. detect disconnect and sessionEnd. Main sessionEnd only logging this. disconnect event set one timed function. This timed func is store disconnected socket object->load socket.session->get custom data (channelIduserId) and send socket.session.kickOut(customData.channel....)

Err1: The another connected client is not get always onKickOut event
Err2: After client is closed, the other connected client session broken, connecting again automaticaly and loggedIn 5-10 item/user..
Err3: After login again success, get a onKickOut event the client, in session data is a more logged in data and not a previous normal closed client.

The session datas is mixed on SC server?

Pleas recheck this mechanism: clients connecting->subscribeing to channel and client(s) leave channel (eg.: browser close) the online clients is get OnKickOut.
Pleas checking session and connection stabilization. In main test app detect a more socket disconnecting and fail on reconnecting.

I use now SC nData 2.6.5, SC 1.1.3, SCClient 1.0.6

there is java client for android

can you use it for android using java?

New label

Please make a new label for SocketCluster v2 and mark all issues with existing labels.

engine-io parser bug fixes

Hi,
Looks like engine-io parser package has few useful bug fixes recently wrt to usage of utf8 encoding. Since socektcluster package uses fixed version(1.0.6) of it, the latest developments in that package are missing. can you please looks at this?

Thanks,
BLN

Remove node_modules from the repo

Handshake middleware authentication

I'm currently developing an application that makes use of the handshake middleware to authenticate my users when they open up a socket connection. I'm using the module passport.socketcluster (https://github.com/artema/passport.socketcluster) to help out, and here's a sample of the code I'm using:

sc.addMiddleware(sc.MIDDLEWARE_HANDSHAKE, passportSocketCluster.handshake({
    cookieParser: cookieParser,
    key: cookieKey,
    secret: cookieSecret,
    store: store,
    passport: passport,
    callback: function(err, req, user, next){
        if(!err){
            req.socket.setAuthToken({ //authenticates socket only if user information was loaded
                profile: {
                    name: user._json.personaname,
                    avatar: user._json.avatar
                }
            });
        }
        next();
    }
}));
//Later on...
sc.on('connection', function(socket){
    var authToken = socket.getAuthToken();

    socket.on('state request', function(data){
        var state = {};

        if(authToken){
            state.profile = authToken.profile;
        }

        socket.emit('state', state);
    });
});

Basically, if a user has logged in, the server sends a 'state' event with profile information, but if the user hasn't logged in, then profile information isn't sent. Everything works when the user hasn't logged in, but when they have, the authToken is still null.

I've had a hard time finding documentation about the req parameter for the middleware handshake, and I was wondering if req.socket is a reference to the actual socket and if how I'm using req.socket.setAuthToken(...) is correct. If not, what are my other options for passing information from the handshake middleware to sockets now that socket.session is no longer available? Any advice will be appreciated.

tool performance test socketcluster

Hello,
I'm building applications based on nodejs, I'm resolving implementation problems handling multiple connections from the client. I've found you "socketcluster", I tried but I can not test yourself socketcluster performance. Please give me one tool that can check the performance of socketcluster on 200k connection.

Sorry for my english.

Thanks !

Socket session id parsing is bug?

Hello, i try socket-server 0.9.43

Detect a different sessionId-s

Method:
1: Client get connection, logging SID from onconnect event server and client side.
2: Client send an auth event to sc, sc logging,parsing this
3: Client receive auth response, and set communication object into main.

In logging have different session Id-s!

Server log:

handshake
connected:_w0_8000_0_EUFeHXRWgRgwBQeHAAAA
MidEmit:auth{"username":"anonymus","password":"secret","roomid":"123456789","sid":"_w0_8000_0_6pUFFKLIWx67Wm_0AAAA"}

!!!! Problem: The connected:w0......HAAAA is a connection SID. Log out this on client side, is correct. The server receive a MidEmit:auth{} an oldest sessin data, is ok, not use old sid, do enable parsing auth event:

Enable emit parsing
Auth:{"username":"anonymus","password":"secret","roomid":"123456789","sid":"_w0_8000_0_6pUFFKLIWx67Wm_0AAAA"}
Set session auth to _w0_8000_0_EUFeHXRWgRgwBQeHAAAA, token:TOKEN123456

!!!Problem: this last line log logging out socket.session.id on AUTH event parsing end.

The on.connection(SID) != on.auth(...socket.session.id...) and not understand why!

MidEmit:ping{"token":"TOKEN123456","sid":"_w0_8000_0_6pUFFKLIWx67Wm_0AAAA","channel":"123456789"}
Session authorized, allow emit parsing
PING { token: 'TOKEN123456',
sid: '_w0_8000_0_6pUFFKLIWx67Wm_0AAAA',
channel: '123456789' }
The client sending periodicaly PING with data, use SID from on(connect(SID)), the server is check all event as middlevare sessionIsAuthorized, detecting good this.

Why a difference the SESSION ID-s?

And if node is broken/restart all sessions is destroyed on server side. The clients trys reconnect and sending oldest session data. The server is detect bad session data, send response 'unauthorizedSession'. Client try reconnect and auth again.

Now server/client is freezing in auth procedure...

Client:

<script type="text/javascript"> var ChatApp = function(params){ var self = this; // Initiate the connection to the server self.socket = socketCluster.connect(); self.authObj = new Object({ username:'anonymus', password:'secret', roomid:'123456789', sid:0 }); self.socket.on('error', function (err) { throw 'Socket error - ' + err; }); self.socket.on('connect', function (SID) { console.debug('Connected:'+SID); self.authObj.sid = SID; self.commObj = new Object({ token:0, sid:0, channel:0 }); self.socket.emit('auth',self.authObj,function(err,res){ if (err) console.log('Auth error:'+err); if (res.status == 'success'){ console.log('Auth successfull'); self.commObj.token = res.token; self.commObj.sid = SID; self.socket.subscribe(self.authObj.roomid, function (err) { if (err) { console.log('Failed to subscribe to room event channel:'+self.authObj.roomid); }else{ self.commObj.channel = self.authObj.roomid; } }); }else{ console.log('Auth failed, status:'+res.status); } }); }); self.socket.on('rand', function (num,res) { res.end(); console.log('RANDOM:', num); if (num.rand) { document.body.innerHTML = 'RANDOM: ' + num.rand + '
'; } }); self.socket.on('pong', function (num,res) { res.end(); console.log('PONG:', num); }); setInterval(function () { if (self.commObj.channel){ self.socket.emit('ping',self.commObj,function(err){ if (err){ console.log('Emit error:'+err); if (err.err == 'unauthorizedSession'){ self.socket = socketCluster.connect(); } } }); } }, 10000); }; var chat1 = new ChatApp(); </script>

Server:
scServer.addMiddleware(scServer.MIDDLEWARE_HANDSHAKE, function (req, next) {
console.log('handshake');
next();
});
//Mielőtt átadódik a socket.subscribe-ra itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_SUBSCRIBE,
function (socket, channel, next) {
socket.session.get('isUserAuthorized', function (err, value) {
if (err){
console.log('Session error:');
console.log(err);
}
if (value) {
next();// Allow
} else {
next({err:'unauthorizedSession',sid:socket.session.id,channel:channel}); // Block
}
});
}
);
//Mielőtt átadódik a socket.publish-ra itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_PUBLISH,
function (socket, event, data, next) {
socket.session.get('isUserAuthorized', function (err, value) {
if (err){
console.log('Session error:');
console.log(err);
}
if (value) {
next();// Allow
} else {
next({err:'unauthorizedSession',sid:socket.session.id,event:event}); // Block
}
});
}
);
//Mielőtt átadódik a socket.emit-re itt lehet vizsgálni dolgokat
scServer.addMiddleware(scServer.MIDDLEWARE_EMIT,
function (socket, event, data, next) {
console.log('MidEmit:'+event+JSON.stringify(data));
if (event == 'auth'){
//use unauthorized emits
console.log('Enable emit parsing');
next();
}else{
//authorize required
socket.session.get('isUserAuthorized', function (err, value) {
if (err){
console.log('Session error:');
console.log(err);
}
if (value) {
console.log('Session authorized, allow emit parsing');
next();// Allow
} else {
console.log('MiddlevareEmit'+JSON.stringify({err:'unauthorizedSession',sid:socket.session.id,event:event}));
next({err:'unauthorizedSession',sid:socket.session.id,event:event}); // Block
}
});
}
}
);

scServer.on('connection', function (socket) {
    console.log('connected:'+socket.session.id);
    /*
      Store that socket's session for later use.
      We will emit events on it later - Those events will
      affect all sockets which belong to that session.
    */
    socket.on('auth',function(data,res){
        console.log('Auth:'+JSON.stringify(data));
        if (data.username && data.password && data.roomid){
            var token = 'TOKEN123456';
            res.end({status:'success',token:token});
            console.log('Set session auth to '+socket.session.id+', token:'+token);
            socket.session.set('isUserAuthorized', true, function(err){
                if(err)
                    console.log(err)
            });
            activeSessions[socket.session.id] = socket.session;
        }else{
            res.end({status:'failed'});
        }
    });


    socket.on('ping', function (data,req) {
        //req.end();
        count++;
        console.log('PING', data);
        scServer.global.publish('pong', count);
    });
});

socketcluster client options

I could not find how to make my socketcluster client connect to another ip address and port. I am using the demo page and i have added a parameter to the connect call like so:

var socket = socketCluster.connect('...*:8000');

Using the ip to my remote socketcluster server but it's not connecting. Firewall on the remote server is opened si i'm sure it's not that. How do i make it connect to the remote websocket server and are there other options that can be passed into the connect method ? Thanks !

Hapi instead of express

Hi i know this might be a little outside the scope, but is there a way to use hapi instead of express? it seems like express exposes a method var app = require('express')(); which i can use in conjunction with the worker httpServer.on('req', app) it doesnt seem that hapi.js does the same.

thanks
-David

How to get the remote IP address

I want to be able to retrieve the IP address of an incoming connection.

I tried socket.request.connection.remoteAddress but is not working.

Any idea?

I understand the midleware...

I sorry, if is a stupid question!

Client connecting to server -> Check authorization -> set session.isAuthorized -> setMidlewareSubscribe to room.
Client side is send unauthorized session or unauthorized room, can receive midleware denied.

I see github and socketcluster.io samples.... don't understand what is a client and server side code, where is a function entry point in workflow... on exaples use scServer, wsServer...

I use now simple session auth set/check and try expanding this from midleware preparsing.

Pleas write a simple, completly example howto use midleware handshake, subscribe...etc.

Accessing an array from multiple workers

Currently, I have one single worker running with a global array "ActiveUsers = []". Inside that array, I store a list of the ids for all logged in users that subscribed to some private channels.

My question is: when spawning two workers say worker1 and worker2, what is the best way for accessing worker1.ActiveUsers from inside worker2 ?

Deeper description of store needed

Hey! My company is about to build a larger application for collecting and displaying real-time data. This looks like the perfect framework for just that. I've tried the example and it works perfectly, both on single core machines and on multicore VMs.

I am, however, a little confused as to what part the "store" module plays in the system. Could you provide an example on when and how to use the store. It would also be nice to know the ratio of store to worker that is needed in general, is it a 1:1 relationship? When is the store activated?

Much appriciated if you also could provide some insight into how socketcluster could handle the case below.
"Thousands of users can in realtime vote on an event in a certain 'room' through sockets. In a different 'room' moderators can see these votes in a sliding-window-graph in real time."
My plan was to use redis in memory db so the moderators could get their results without the backend having to query the filesystem. Could the store be used for this or should this be done in the worker on-event? And then maybe in a setInterval sent batched through the socket to the moderator room? Any ideas on this?

Publish data to channel by socketCluster object

after var sc = new SocketCluster(opts);, I want to use sc object to publish data to channel, but can not find a way.

while I can use worker or socket to complete this job, but I want to use redis pub/sub to trigger sending data action, I don't want to do subscribing to redis on worker because I have to sub only once. So the sc creation is the best place to do redis pub/sub.

Sending to all clients except sender

I have an event listener:
var events = require('events');
var eventEmitter = new events.EventEmitter();
eventEmitter.on('doorOpen', function ringBell()
{
console.log('ring ring ring');
});

I have also private variables inside my workers, like:
var ClientsMap[data.userID] = {
SocketHandler: socket.id
};

The problem is when I emit an event (coming from an external request):
eventEmitter.emit('doorOpen');

I don't have access to my variable 'ClientsMap' because the variable is inside another worker.
Example: I add a user on a process PID 6910 and the event is sent to another worker 6913, so 6913 he don't have any reference to my local variable ClientsMap...

How I can send a general event between my workers (like req maybe) ?

Express-session and PassportJS

As I'm trying to migrate my old Socket.IO based project to Socketcluster I noticed PassportJS is not saving sessions into the store.
Probably because Socketcluster interferes the current session solution from Express that I've used? Which also mean I can not use a Redis store for Express only?

Old version would give me a user id if they are authenticated:

req.session.passport = { user: 'userID' }

But now after serialization it just stays empty:

req.session.passport = {}

Although the req.session.passport object is still there.

I don't know what I should do, neither am I sure what is causing this, any ideas?

Store usage example. Push data from php.

Hi. I'm just starting to develop my socketcluster based app and i was wondering if there is any way in which i can push data to the socketcluster app from outside. My main app is running on php and from that i would like to send information into socketcluster. Could you outline how to set up such a thing on the socketcluster side ? Is it best to implement this at a store level or somewhere else ?

Uncaught exception handler

Does it exists an uncaught exception handler?

I need one to log what happened in case of fire.

Can we move the reference to SocketCluster server outside of the workercontroller?

We have a nodejs based application with huge code base and we're currently using socketCluster.
We need to publish/emit from other part of the code, not just from workercontroller. Is this possible?
Pardon my ignorance, this may be a silly question :)

Error: Cannot traverse circular structure

Hello,

Having problem with circular structures in a object I'm sending with

var items = {};
items.obj = items;
socket.emit('post', { data: items });

It gives me the following error:

1412974000965 - Error: Cannot traverse circular structure at convertBuffersToBase64 (/home/willeponken/projects/test/node_modules/socketcluster/node_modules/socketcluster-server/formatter.js:23:11)

Should I sanitize the object myself, or is this something SC should do?
Really not a big deal to fix, but maybe it could be interesting to see.

How to run socketcluster on startup

Hi! Im currently deploying a socketcluster server. Is there any reason to start it with a process manager like forever or pm2? Or should it be sufficient to start is as a regular background application?