xpush / node-xpush

XPUSH is an realtime communication channel server for quickly, easily adding scalable functionality to web and mobile apps.

visit XPUSH (for version 0.0.23)

We are preparing the documents for version 0.1.x

xpush

XPUSH (eXtensional PUSH) is a real-time communication server and a programmable library that supports websocket, GCM and APN. It is suitable for implementing components such as messengers and push system.

Features

You can easily build real-time services with HTML5 Websocket and socket.io, the module on node.js.

But, there are a lot of things to consider on building the real-time services. You need to consider various things such as distributed server configuration in consideration of the users who are rapidly increasing, network traffic processing of large capacity and scale-out strategy.

XPUSH is a server platform that provides such as real-time message communication, message storage, user and device management and Mobile Push Notification. Everyone will be able to install and use after you downloaded. And, XPUSH is a service platform for developing a wide range of real-time services and applications over a single XPUSH server platform.

Without having to directly implement the real-time communication servers, After you install XPUSH platform, try adding a real-time data communication features of your service.

1. Real-time Web Communication Platform

" We help you easily build real-time applications. XPUSH was designed for developers. "

XPUSH consists of real-time communication servers that has been developed as a Web technology. You can build a variety of real-time messaging applications such as your own messenger and chat services

Currently, XPUSH developers are provided Javascript libraries for developing Web services and JAVA libraries for developing JAVA applications and Android applications. We are continually developing additional libraries.

Real-time data communication, while maintaining the network connection between the Client and the Server, can transmit and receive data bidirectionally. If the connection to the server is lost, or even if some servers fails, it guarantees the service availability.

2. Works Everywhere

" At the core of XPUSH is the HTML5 WebSocket protocol, but we also have fallback mechanisms so that XPUSH just works anywhere, anytime. "

XPUSH was developed by node.js to implement high-performance servers. It uses Socket.IO, one of the modules for web-based real-time messaging. And we continue to update the modules to use the latest version.

Socket.IO enables real-time bidirectional event-based communication. It works on every platform, browser or device, focusing equally on reliability and speed.

3. Scalable Web Architecture

" XPUSH was designed to work with commodity servers, an elastic virtualised environments saving you money and headaches. A scalable web application can handle growth – of users or work – without requiring changes to the source code and stoping existed servers "

Real time server platform has to be designed to be able to handle a large amount of network traffic to a sharply rising splice. It has to run a large number of servers for load balancing, needs to designed to be non-disruptive expansion. XPUSH developers designed the Scalable Web Architecture for a long time, and continue to optimize the architecture design.

XPUSH manages the real-time status of the distributed servers through a zookeeper and use Redis to store connection information of the visitors and meta datas in the memory. And XPUSH stores various types of unstructured messages sent or received in MongoDB.

XPUSH server platform run each in the Session server and Channel server. Channel server is responsible to authenticate users, managing user and device information, and assigning distribution server for load balancing. Since relatively Channel servers are easy to increase the load on the network traffic, so that it can be added separately as an expansion only Channel server

1. Prepare

To use the XPUSH is, nodejs, zookeeper, redis is required .

The following is how to install 64bit linux. Please install to suit your environment. If you have already been installed, please skip these preparation steps.

Install nodejs

nodejs installation by referring to Download and unzip the nodejs.

cd $HOME/xpush
wget http://nodejs.org/dist/v5.0.0/node-v5.0.0-linux-x64.tar.gz
tar zvf node-v5.0.0-linux-x64.tar.gz
cd node-v5.0.0-linux-x64
./configure
make
sudo make install

Install zookeeper

Install and run zookeeper with reference Zookeeper installation.

The following is the code to install and run the zookeeper3.4.9.

cd $HOME/xpush
wget http://apache.mirror.cdnetworks.com/zookeeper/stable/zookeeper-3.4.9.tar.gz
tar xvf zookeeper-3.4.9.tar.gz
cp zookeeper-3.4.9/conf/zoo_sample.cfg zookeeper-3.4.9/conf/zoo.cfg
cd zookeeper-3.4.9/bin
./zkServer.sh start

Install redis

Install and run redis with reference Redis installation.

The follow is the code to install and run redis 3.2.6.

cd $HOME/xpush
wget http://download.redis.io/releases/redis-3.2.6.tar.gz
tar xzf redis-3.2.6.tar.gz
cd redis-3.2.6
make
src/redis-server
(daemon : $ nohup src/redis-server & )

2. Run your application with push module

clone xpush repository

git clone https://github.com/xpush/node-xpush.git
cd xpush
npm install

run session server

bin/start --session

run channel server

bin/start --channel

with config.json

bin/start ---config config.json

Config Options

- host : bind ip
- port : bind port

node-xpush's Issues

Where is the channel name??

Channel 신규 생성할때
shortId 만 생성해서 저장하고
원래 channel 이름은 저장 안하는거 같은뎅,
맞나요? 필요할거 같은데..ㅋ

Add rhe authorization for socket connection.

Reported from @notdol.

PUT으로 application생성후 GET하면 서버가 죽는 것 같습니다

위 스크린샷은 도커로 생성후 (daemon옵션을 빼고 -it를 넣음)

[GET] :8000/app
[PUT] :8000/app/demo

했을때 출력된 오류입니다

channel server의 auto scaling은 어떻게 처리해야 할까요?

session server는 load balancer를 앞단에 붙혀 부하에 따른 auto sacaling을 처리하면 되는데 channel server의 경우 auto scaling을 어떻게 처리해야 할까요?
아키텍쳐를 보면 session server에는 load balancer가 있고, channel server는 session server에서 load balancing을 해주고 있는 것으로 파악됩니다. 그럼 channel server는 auto scaling역할을 해주고 있는 instance가 없는 상태입니다. 이상태에서 auto scaling을 어떻게 해줘야 하는지 궁긍해서 여쭙습니다.
제가 생각하는 시나리오는

1. channel server를 넉넉히 확보한다.
2. auto scaling 하는 instance를 따로 만든다. 예를 들면channel server를 watching 하는 인스턴스를 만들고 connection 갯수(??) 에 따라 channel server를 add 및 delete 하는역할을 하게한다. channel server가 delete될시 client에서 session server에 재요청하여, channel server를 다시 할당 받는다. channel server가 add 될시에는 그냥 사용한다.(add는 최대한 스무스하게 되도록 규칙을 넣는다.)

두가지 시나리오를 생각해보고 있는데 제가 맞게 생각하고 있는지 궁금합니다. 만약 틀리다면 xpush에 가장 걸맞는 auto scaling시나리오가 있는지 알고 싶습니다!!

multi oauth for multi applications

Must create multi passports.

jaredhanson/passport-facebook#5

socket.io-java-client handshake issue

socket.io java client 사용하여 socket 연결(handshake)할 때,
get 방식으로 query string을 넘기지 못하는 문제가 있습니다.
Gottox/socket.io-java-client#75

해결 방법은 크게 두가지입니다.

1. 인증(handshake)를 위한 데이터를 헤더에 넘김
   • socket.io java client에서 헤더를 포함하는 기능을 지원함.
   • 채널 서버에 handshake 할때 header 정보를 분기하여 체크하는 로직을 추가해야함
2. query string 을 같이 넘기도록 socket.io java 소스를 수정하여 사용
   • 소스 수정하여 테스트 성공
   • socket.io master의 소스 변경시 version up 에 대한 문제가 있음
     or contribution 해야함

sync the connnection infos with redis

• install redis
• implements session manager.

Exception: CONNECTION_LOSS

서버를 Ctr+C로 강제로 죽이면, zookeeper 의 노드를 삭제하면서, 아래와 같은 에러가 나는 경우가 많습니다.

Failed to remove node: /xpush/meta/session/192.168.219.140:8000 due to: Exception: CONNECTION_LOSS[-4]

왜 그럴까요?
왜 Connection 을 읽어버리는거죠?

maintaining the meta data object from zookeeper.

Create the gateway server path on zookeeper while gateway server is alive.
(The app paths will be created by xpush manager server)

The server paths were completed. Be ignore.

Once started, check whether zookeeper and redis are running.

If zookeeper and redis are not running or accessible, send out error messages on the terminal console and exit.

Create the application, channel, user, message source code skeleton.

동일 서버에서 session, channel socket 이 동시에 접속 할 수 없습니다.(web)

socket.io client 가 동일 서버로는 handshake의 query 를 변경 할 수 없습니다.

socket.io client 를 변경 하든지 아니면 서버에서 session 과 channel socket 은 포트를 달리 띄워야 합니다.

mongoose.createConnection VS mongoose.connection

mongoose.createConnection 사용시엔 mongoose.model 을 사용할 수가 없어 db 조회가 안됩니다.
createConnection 사용시엔 createConnection 결과로 반환된 connection.model로 바꿔야해서 connection을 인자로 들고다녀야하는 문제.

http://stackoverflow.com/questions/12806559/mongoose-model-vs-connection-model-vs-model-model

단, mongoose.createConnection 사용시엔 mocha 테스트시 여러개의 컨넥션을 동시에 맺을 수 없습니다. mocha가 한번 맺은 커넥션은 재사용이 가능하므로 해당 부분 처리는 가능할 것으로 보입니다.

New API : userUpdate

POST /user/update

Parameters
app : application ID mandatory
userId : User ID mandatory
deviceId : Unique client device ID
password : password
notiId : Notification ID
users : Addional Datas ( JSON Object )

Logic
update notiId & users data

Result
return user Object( updated )

Create SessionSocket Server API

Get SessionSocketServer location , Use applicationKey & userId infomation.

Create application api code ( no mongodb version )

Create application api code ( no mongodb version )

Make code easier to read from callback hell.

Asynchronous javascript, or javascript that uses callbacks, is hard to get right intuitively. A lot of code ends up looking like this

• node-xpush / lib / node-manager / node-manager.js

    self._initPath('', function () {
    self._initPath(constants.SERVERS_PATH, function () {
    self._initPath(constants.META_PATH, function () {
      if(isWatching) self._watchServerNodes();
      self.ready = true;

      if(callback) callback();
    });
    });
    });

Makes code easier to read !!!

receive all of infomation in Channel Data ?? ( consider key data )

When client library listen Channel data , use all of data or use only key data.
How do we handle on this situations??

Create the mongodb persister and API logics.

I gonna do this tonight. :)

Channel problem when restart the server

channel에 포함된 유저의 리스트가 memory에 있다보니..

채널을 신규 생성을 하고, 메세지를 보내면 문제가 없지만..

서버를 재시작하고, 기존에 생성된 채널로 메세지를 보내면,
채널에 포함된 유저 리스트가 undefined 여서 exception이 나면서 서버가 종료됩니다.

메모리에 없으면, 몽고에서 가져오는 방식으로 바꾸면 될까요?

/home/james/node-xpush/lib/server/channel-server.js:677
for(var x=0; x<users.length; x++){
^
TypeError: Cannot read property 'length' of undefined
at ChannelServer.send (/home/james/node-xpush/lib/server/channel-server.js:677:23)

Complete implementing APIs (only for mongodb)

except APIs for application resource.

This is just current source profiling(test)