mrjoes / tornadio2

Python socket.io server implementation on top of Tornado framework

TornadIO2

Disclaimer

Unfortunately, Socket.IO 0.8 branch is abandoned, there are huge amount of bugs and nothing getting fixed. I would suggest to look at alternative options, for example SockJS <http://github.com/sockjs/>. There's sockjs-tornado <http://github.com/mrjoes/sockjs-tornado/> too. I don't plan to support TornadIO2 anymore, assuming that parent library is no longer maintained.

If you still want to try Socket.IO 0.8.x with Python, TornadIO2 is quite stable and used in production.

Contributors

• Serge S. Koval

Introduction

This is python server implementation of the Socket.IO realtime transport library on top of the Tornado framework.

TornadIO2 is compatible with 0.7+ version of the Socket.IO and implements most of the features found in original Socket.IO server software.

Key features:

• Supports Socket.IO 0.8 protocol and related features
• Full unicode support
• Support for generator-based asynchronous code (tornado.gen API)
• Statistics capture (packets per second, etc)

What is Socket.IO?

Socket.IO aims to make realtime apps possible in every browser and mobile device, blurring the differences between the different transport mechanisms. It's care-free realtime 100% in JavaScript.

You can use it to build push service, games, etc. Socket.IO will adapt to the clients browser and will use most effective transport protocol available.

Getting Started

In order to start working with the TornadIO2 library, you have to have some basic Tornado knowledge. If you don't know how to use it, please read Tornado tutorial, which can be found here.

If you're familiar with Tornado, do following to add support for Socket.IO to your application:

1. Derive from tornadio2.SocketConnection class and override on_message method (on_open/on_close are optional):

   class MyConnection(tornadio2.SocketConnection):
       def on_message(self, message):
           pass

2. Create TornadIO2 server for your connection:

   MyRouter = tornadio2.TornadioRouter(MyConnection)

3. Add your handler routes to the Tornado application:

   application = tornado.web.Application(
       MyRouter.urls,
       socket_io_port = 8000)

4. Start your application
5. You have your socket.io server running at port 8000. Simple, right?

Starting Up

We provide customized version (shamelessly borrowed from the SocketTornad.IO library) of the HttpServer, which simplifies start of your TornadIO server.

To start it, do following (assuming you created application object before):

if __name__ == "__main__":
    socketio_server = SocketServer(application)

SocketServer will automatically start Flash policy server, if required.

If you don't want to start IOLoop immediately, pass auto_start = False as one of the constructor options and then manually start IOLoop.

More information

For more information, check TornadIO2 documentation and sample applications.

Examples

Acknowledgment

Ping sample which shows how to use events to work in request-response mode. It is in the examples/ackping directory.

Cross site

Chat sample which demonstrates how cross-site communication works (chat server is running on port 8002, while HTTP server runs on port 8001). It is in the examples/crosssite directory.

Events and generator-based async API

Example which shows how to use events and generator-based API to work with asynchronous code. It is in the examples/gen directory.

Multiplexed

Ping and chat demo running through one connection. You can see it in examples/multiplexed directory.

Stats

TornadIO2 collects some counters that you can use to troubleshoot your application performance. Example in examples/stats directory gives an idea how you can use these stats to plot realtime graph.

RPC ping

Ping which works through socket.io events. It is in the examples/rpcping directory.

Transports

Simple ping/pong example with chat-like interface with selectable transports. It is in the examples/transports directory.

tornadio2's Issues

python client for tornadio2

hey, this is more a feature request than an issue, but is it possible to include a python client to interact with the server? there are use cases where we need to to push some data to tornadio server programmatically and a python client woud come handy. other use cases been testing, load testing, etc.

i tried the code hosted here: http://pypi.python.org/pypi/websocket-client/0.5.0a, but i don't think it handles the handshaking protocol implemented in tornadio2.

thx.

«on_event() got multiple values for keyword argument 'name'»

If key "name" in json data, then:

ERROR:root:on_event() got multiple values for keyword argument 'name'
Traceback (most recent call last):
  File "/Users/yurtaev/PycharmProjects/tornado_bag/tornadio2/session.py", line 404, in raw_message
    ack_response = conn.on_event(event['name'], **str_args)
TypeError: on_event() got multiple values for keyword argument 'name'

because:

def on_event(self, name, _args, *_kwargs):
via. https://github.com/MrJoes/tornadio2/blob/master/tornadio2/conn.py#L154

python

@event
def echo(self, **kwargs):
    return kwargs

javascript

conn.emit("echo", {name:"test"}, function(data){console.dir(data)})

ERROR - Error in periodic callback

I have some problems running tornadio2 in production. I get "Error in periodic callback".

Any ideas what the problem could be?

Many thanks for any hint!

Traceback (most recent call last):
File "build/bdist.freebsd-9.1-RELEASE-amd64/egg/tornadio2/periodic.py", line 82, in _run
next_call = self.callback()
File "build/bdist.freebsd-9.1-RELEASE-amd64/egg/tornadio2/session.py", line 274, in _heartbeat
self.send_message(proto.heartbeat())
File "build/bdist.freebsd-9.1-RELEASE-amd64/egg/tornadio2/session.py", line 192, in send_message
self.flush()
File "build/bdist.freebsd-9.1-RELEASE-amd64/egg/tornadio2/session.py", line 202, in flush
self.handler.send_messages(self.send_queue)
File "build/bdist.freebsd-9.1-RELEASE-amd64/egg/tornadio2/persistent.py", line 160, in send_messages
self.write_message(m)
File "lib/python2.7/site-packages/tornado-3.0.1-py2.7.egg/tornado/websocket.py", line 164, in write_message
self.ws_connection.write_message(message, binary=binary)
AttributeError: 'NoneType' object has no attribute 'write_message'

Support engine.io

Socket.io 1.0 is around the corner , with its new core transport component engine.io (which also can be used standalone) , and actually many developers are shifting to use engine.io , because it is lightweight and more scalable than the full stack socket.io

I think that natively supporting engine.io is essential feature

AttributeError in logs when client drops connection

I'm seeing this error in my logs when a client dies.

ERROR:root:Error in periodic callback
Traceback (most recent call last):
File "/Library/Python/2.6/site-packages/tornado-2.2-py2.6.egg/tornado/ioloop.py", line 479, in _run
self.callback()
File "/Users/progan/Projects/Workspaces/Aptana/python/cb_tornado/src/multiplexed/multiplexed.py", line 64, in broadcast_all_num
p.send(len(cls.my_messge_buffer))
File "build/bdist.macosx-10.6-universal/egg/tornadio2/conn.py", line 238, in send
self.session.send_message(msg)
File "build/bdist.macosx-10.6-universal/egg/tornadio2/session.py", line 192, in send_message
self.flush()
File "build/bdist.macosx-10.6-universal/egg/tornadio2/session.py", line 202, in flush
self.handler.send_messages(self.send_queue)
File "build/bdist.macosx-10.6-universal/egg/tornadio2/persistent.py", line 149, in send_messages
if self.client_terminated:
AttributeError: 'TornadioWebSocketHandler' object has no attribute 'client_terminated'

THANK YOU SO MUCH FOR WORKING WHEN PLAIN WEBSOCKETS TORNADO EXAMPLE DID NOT WORK!!

YOU ARE THE BEST THANK YOU FOR YOUR CODE AND SERVICE TO HUMANITY EVERYWHERE.

Hugs,
D

Running TornadIO2 with Tornado using Supervisord and different ports throws "socket.error: [Errno 98] Address already in use"

I'm writing a Tornado app and one of the aspects of it is that it provides real-time updates to the client via SocketIO.

The Tornado documentation suggests using Supervisor to run multiple processes of Tornado (one per core) behind NginX.

I've set it up so that the Tornado and TornadIO applications bind to different ports for each process, when the server starts up. However it looks like the TornadIO2 implementation can't bind to '0.0.0.0' more than once?

Maybe I'm doing something wrong - I'm setting the variables in the applications settings as such:

project_settings = {
"debug": True,
"xheaders": True,
"socket_io_port": 8081,
"socket_io_address": "0.0.0.0"
}

When unsetting the socketio_address field it still raises the same error.

Thanks,
Martin

@event decorator limitation?

I have a set of classes like this:

class DistributeHandler(SocketConnection):
@event
def join(self, _args, *_kargs):
# Join a channel based on kargs
print "Joining %s" % (kargs['group'], )

on_message(self, msg):
    pass

class MainRouter(SocketConnection):
endpoints : { '/realtime': DistributeHandler, '/other': OtherHandler}
on_open(self, *args):
print "Connected"

On the client side, I have this:
var socket = io.connect("ws://mysite.com/realtime");
socket.emit('join', {group: 5}, function() { console.log('joined'); });

However, the server informs me that 'join' is an invalid event. Is this because the @event decorator is on the endpoint SocketConnection class and not on the MainRouter object? Is @event supposed to be that limited or this a bug? I'd love to be able to put the @event decorator on the endpoint classes, as that would make my system far more flexible. I just wanted to make sure I understood what is going on.

using tornado.web.RequestHandler along with tornadio2.SocketConnection

Hi

Is there a way to subclass both of the classes from topic?

A reason for this is to use @tornado.web.authenticated decorator with on_message() and on_open()

It would also help me to get current user. A little more details:

I have a class UserHandler
class UserHandler(tornado.web.RequestHandler):
with a method get_current_user() that is based on secret cookies and mongo db.
I need a way to use this method form my SocketConnection's on_message() and on_open()

Right now it is not possible because UserHandler's constructor is never called if I define it like this:
class MySocketConnection(SocketConnection, UserHandler):

How would I identify the user otherwise?

Thanks.

Documentation on Session usage

From what I can tell, the current documentation is a little scant on how to properly implement session handling from within a router. The structure of the code makes it a little difficult to understand how sessions are instantiated.

The chat included with the codebase gives a detailed explanation of how to persist multiple users on a singleton connection object, but does go into detail about persisting sessions or creating channels. A practical howto would be great.

PyPi

Please upgrade PyPi repositories for package "tornadio2"

When i install "pip install tornadio2" that installed old version package without some fixed bugs :(
its very discomfortable, also when i install from github repositories from master
"-e git+git://github.com/mrjoes/tornadio2/#egg=TornadIO2" its not working correctly, because TornadIO2-0.0.3-py2.7.egg-info has incorrect link to source for examle that included "../tornadio2/"
but source installed in venv/src/tornadio2/tornadio2/...

Do you know why pip do that? It is correctly behavior for pip ?

session_check_interval is in milliseconds instead of seconds

Hi,

I've noticed that the session check interval is taken directly from the settings (where it's said to be in seconds and default value is 15) and supplied to the PeriodicCallback constructor. However the constructor expects milliseconds, rather than seconds. It should be fixed by multiplying check_interval by 1000 in the following line:

self._sessions_cleanup = ioloop.PeriodicCallback(self._sessions.expire,
check_interval,
self.io_loop)

Dmitry

Preflight handshake broken for credentialed CORS

Client handshakes under Cross-origin Request Sharing are not working properly if the following conditions are met

1. Origin site and Tornadio service are running on different ports
2. The origin site has a cookie set

Only condition 1 is guaranteed to be met for the crosssite example.

When both of these conditions are present, a CORS-compliant browser will attempt to use the withCredentials property of the XMLHttpRequest object. However, this requires that the following response header are set.

• Access-Control-Allow-Origin: exactOrigin
• Access-Control-Allow-Credentials: true

where exactOrigin is self.request.headers['origin'].

Expected results:

• Access-Control-Allow-Origin: exactOrigin
• Access-Control-Allow-Credentials: true

Actual results:

• Access-Control-Allow-Origin: exactOrigin
• Access-Control-Allow-Credentials: True

Browsers are strictly enforcing a case-sensitive header value, making True invalid and true valid.

This results in a broken handshake process for any socket.io-client later than 0.8.6.

Related documentation:
http://www.w3.org/TR/cors/#access-control-allow-credentials-respons
https://developer.mozilla.org/En/HTTP_Access_Control#Requests_with_credentials

Related issue in socket.io-client:
socketio/socket.io-client#333

In socket.io-client 0.9.1 client side heartbeat was implemented, but tornadio2 doesn't send it in xhr-polling transport

In socket.io-client 0.9.1 client side hearbeat was implmeneted, see commit socketio/socket.io-client@0bcd25f

It took heartbeat invetrval and waiting for packet. If not heartbeat -- it triggers reconnect.

But i found, that tornadio2 doesn't start hearbeat process when use xhr-transport.

So we got regular reconnection process.
It's not problem, i can use 0.9.0, but need to fix (may be in clientside library?)

on_event fails with unicode dict keys

I was overriding on_event, and sending keyword arguments from the client:

socket.emit('channel1', {msg: 'hello'});

I kept getting the error:

File "/ws_env/src/tornadio2/tornadio2/session.py", line 386, in raw_message
conn.on_event(event['name'], **args[0])
TypeError: on_event() keywords must be strings

I'm using Chrome, and after putting a print statement in the Tornadio source, it seems that somewhere along the line, (possibly in the Socket.io library, although I'm not entirely sure) both the key and value are being converted to unicode strings, which can't be accepted as kwargs.

I patched the source to create a new dict, and convert all the keys and values into plain strings, and this seems to have fixed the issue (around line 384 in session.py):

if len(args) == 1 and isinstance(args[0], dict):
    str_args = {}
    for k, v in args[0].iteritems():
        str_args[str(k)] = str(v)
    conn.on_event(event['name'], **str_args)
else:
    conn.on_event(event['name'], *args)

Perhaps there's a more elegant way of solving this, and it might also affect other parts of the raw message handling.

Thanks

Stuart

how to figure out the current transport type / protocol?

Hey,

in the old version of tornadio, we used to use the _protocol (private, i know) attribute on a SocketConnection object to check whether a connection was websockets or flashsocket or xhr-polling or whatever.

We can't find a similar way of doing it in the new API. Any suggestions?

Logo

I created logo for myself (used to speed dial in chrome). It may be useful... =)

png: http://dl.dropbox.com/u/714800/tornadio2_logo.png

pxd (pixlr.com): http://dl.dropbox.com/u/714800/tornadio2_logo.pxd

uncaught exception breaks connection - poss due to heartbeat breaking chatty connection?

Hey,

we've noticed several Uncaught exception in /socket.io/1/websocket/ types of errors in our logs. We think it tends to happen when there's a lot of small bits data being transmitted between the client and server in quick succession.

We find that, with every error, an incoming message is received at the same time as a heartbeat... so it looks like that's probably what's causing the issue?

Here's a minimal repro, which should crash within a minute or so... :

from os import path as op
import logging
from tornado import web
from tornadio2 import SocketConnection, TornadioRouter, SocketServer
ROOT = op.normpath(op.dirname(__file__))

class IndexHandler(web.RequestHandler):
    """Regular HTTP handler to serve the ping page"""
    def get(self):
        self.render('index.html')

class SocketIOHandler(web.RequestHandler):
    def get(self):
        self.render('../socket.io.js')

class PongConnection(SocketConnection):
    def on_message(self, message):
        print message
        self.send('pong')

PingRouter = TornadioRouter(PongConnection,
                            dict(enabled_protocols=['websocket', 'xhr-polling',
                                                    'jsonp-polling', 'htmlfile']))
application = web.Application(
    PingRouter.apply_routes([
        (r"/", IndexHandler),
        (r"/socket.io.js", SocketIOHandler)
    ]),
    flash_policy_port = 843,
    flash_policy_file = op.join(ROOT, 'flashpolicy.xml'),
    socket_io_port = 8001
)

if __name__ == "__main__":
    # watch logs for error
    logging.basicConfig(filename='repro.log', level=logging.DEBUG, format='%(asctime)s %(levelname)s:%(message)s')
    SocketServer(application)

<!DOCTYPE html>
<html>
<head>
  <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
  <script src="socket.io.js"></script>
  <script>
    $(function() {
        var socketio = new io.connect('http://' + window.location.host);

        socketio.on('disconnect', function() {
            console.log('disconnected');
        });
        socketio.on('error', function() {
            console.log('error');
        });

        socketio.on('connect', function() {
          console.log('connected');
          for (var i=0; i< 100000; i++){
            socketio.send('ping' + i);
          }
        });
    });
</script>
</head>
<body><h3>Minimal repro</h3></body>
</html>

and here is what we find in the logs -- the <<<3 is a normal socketio message, and if we understand it correctly, the <<<2 is a heartbeat.

2012-03-07 15:49:22,313 DEBUG:>>> 3:::ping10717
2012-03-07 15:49:22,314 DEBUG:<<< 3:::pong
2012-03-07 15:49:22,317 DEBUG:>>> 3:::ping10718
2012-03-07 15:49:22,317 DEBUG:<<< 3:::pong
2012-03-07 15:49:22,323 DEBUG:>>> 3:::ping10719
2012-03-07 15:49:22,324 DEBUG:<<< 3:::pong
2012-03-07 15:49:22,326 DEBUG:<<< 2::
2012-03-07 15:49:22,326 DEBUG:<<< 0::
2012-03-07 15:49:22,327 ERROR:Uncaught exception in /socket.io/1/websocket/9292bc6ecf30cab96d548ae0b87e4e18
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 254, in wrapper
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornadio2/persistent.py", line 136, in on_message
    self.session.close()
AttributeError: 'NoneType' object has no attribute 'close'

Tornadio and AppEngine

if running on Windows:

tornadio2\router.py in ()
22 """
23
=> 24 from tornado import ioloop, version_info

//////////////////////////////////////////////////////////////
\tornado\ioloop.py in ()
47 signal = None
48
=> 49 from tornado.platform.auto import set_close_exec, Waker

//////////////////////////////////////////////////////////////
tornado\platform\auto.py in ()
30 if os.name == 'nt':
31 from tornado.platform.common import Waker
=> 32 from tornado.platform.windows import set_close_exec

//////////////////////////////////////////////////////////////
tornado\platform\windows.py in ()
4
5 from future import absolute_import, division, with_statement
=> 6 import ctypes

//////////////////////////////////////////////////////////////
-- \Python27\lib\ctypes__init__.py in ()
8 version = "1.1.0"
9
=> 10 from _ctypes import Union, Structure, Array
/////////////////////////////////////////////////////////////
<type 'exceptions.ImportError'>: No module named _ctypes
args = ('No module named _ctypes',)
message = 'No module named _ctypes'

if deploy:
Error: Server Error

asd

Access to Application settings

It would be helpful to get access to the core application. For example then items like get_secure_cookie could be set up as you'd have access to the settings. ie: cookie_secret

The context of the request is that I'm working to try and add tornadio2 support to AsyncMongo-Sessions. My library uses set_secure_cookie and get_secure_cookie to manage the tokens.

SocketConnection,on_open(),get_cookie(),always returns None.

Hi,
when I uses get_cookie() on the "on_open()" method,it always returns None although I have set the cookie.Could you help me?
Thank you very much!

Testing best practices

Testing best practices for server + web ?

HandshakeHandler should set Access-Control-Allow-Origin

I know it is still work in progress, but I just want to drop this here.

Also, thanks for your hard work!

Typo in sessioncontainer.SessionContainer.remove

As I understand here https://github.com/MrJoes/tornadio2/blob/master/tornadio2/sessioncontainer.py#L116 should be:

self._items[session_id].promoted = -1

Sorry, have no time to make pull request.

asd

Multiple connects to same endpoint result in connection leakage

In the course of writing my own socket.io client, I discovered that a "misbehaving" client that connects to the same endpoint more than once causes any existing connection to that endpoint to never be disconnected in the server (this may be specific to the multiplexed example as it keeps an internal reference to connection and may be interacting poorly with expected garbage collection behavior).

My fix was this:

diff --git a/tornadio2/session.py b/tornadio2/session.py
index c55b5c7..48aadc9 100644
--- a/tornadio2/session.py
+++ b/tornadio2/session.py
@@ -296,6 +296,9 @@ class Session(sessioncontainer.SessionBase):
             return

         conn = conn_class(self, endpoint)
+        old_conn = self.endpoints.get(endpoint, None)
+        if old_conn is not None:
+            self.disconnect_endpoint(endpoint)
         self.endpoints[endpoint] = conn

         self.send_message(proto.connect(endpoint))

Any thoughts on what the proper fix should be?

JSONP double json encoding

I'm getting unparsable data submitted by the jsonp transport. The reason for this seems to be this code in the client:

    // we temporarily stringify until we figure out how to prevent
    // browsers from turning `\n` into `\r\n` in form inputs
    this.area.value = io.JSON.stringify(data);

I'm not sure if the following change is the correct way to fix this in tornadio2/polling.py:

         if data.startswith(u'"'):
-            data = data.strip(u'"')
+            data = proto.json_load(data)

Multiplexed connection's on_open method

For example we have two multiplexed connections:

class RouterConnection(tornadio2.conn.SocketConnection):
    __endpoints__ = {
        '/admin': AdminConnection,
        '/user': UserConnection
    }

We need authentication only for AdminConnection, butAdminConnection.on_open does not affect connection process. I mean returning False does not abort connection.

Server logging and slow server work.

Hello! We develop a web application based on tornadio2. In the main the application works stable but it gives some mistakes rather often. The list of the mistakes is below.
As we see there are no references to the our code and we can’t understand what give that mistakes.
Sometimes (rhythmically) the application gives as a lot of such mistakes (frequency of mistakes increases in times) that makes server work slowly.
We think that the main operation that makes server slower is logging. Can you tell us if we think correct and the logging slows the server?
Can you give us an advise how to avoid or reduce such mistakes?
We looked for the answer at thematic forums and among the issues to the libraries tornado and tornadio2 and had found some statements. The statements tell us that such mistakes are not critical, don’t affect the client-server communication and don’t give any negative effect. Is it true?
If it’s true how can we suppress or inhibit the logging process?

ERROR:tornado.application:Exception in callback 
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 453, in _run_callback
    callback()
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 241, in wrapped
    callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/iostream.py", line 316, in wrapper
    callback(*args)
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 241, in wrapped
    callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/httpserver.py", line 217, in _on_connection_close
    callback()
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 241, in wrapped
    callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 256, in on_connection_close
    self.on_close()
  File "/usr/local/lib/python2.7/dist-packages/tornadio2/persistent.py", line 151, in on_close
    self._detach()
  File "/usr/local/lib/python2.7/dist-packages/tornadio2/persistent.py", line 117, in _detach
    self.session.remove_handler(self)
  File "/usr/local/lib/python2.7/dist-packages/tornadio2/session.py", line 173, in remove_handler
    raise Exception('Attempted to remove invalid handler')
Exception: Attempted to remove invalid handler

ERROR:tornado.application:Uncaught exception in /socket.io/1/websocket/ce83bfa8b8109109f5d4a278204558ec
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 285, in wrapper
    return callback(_args, *_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornadio2/persistent.py", line 87, in open
    raise HTTPError(401, "Invalid Session")
HTTPError: HTTP 401: Unauthorized (Invalid Session)

ERROR:tornado.application:Exception in callback 
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 453, in _run_callback
    callback()
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 241, in wrapped
    callback(_args, *_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/iostream.py", line 316, in wrapper
    callback(_args)
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 241, in wrapped
    callback(_args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/httpserver.py", line 320, in _on_headers
    self.request_callback(self._request)
  File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1479, in __call__
    handler._execute(transforms, _args, *_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornadio2/persistent.py", line 81, in _execute
    super(TornadioWebSocketHandler, self)._execute(transforms, _args, *_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 143, in _execute
    self.ws_connection.accept_connection()
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 492, in accept_connection
    self._accept_connection()
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 541, in _accept_connection
    self._receive_frame()
  File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 582, in _receive_frame
    self.stream.read_bytes(2, self._on_frame_start)
  File "/usr/local/lib/python2.7/dist-packages/tornado/iostream.py", line 162, in read_bytes
    self._try_inline_read()
  File "/usr/local/lib/python2.7/dist-packages/tornado/iostream.py", line 393, in _try_inline_read
    self._check_closed()
  File "/usr/local/lib/python2.7/dist-packages/tornado/iostream.py", line 543, in _check_closed
    raise StreamClosedError("Stream is closed")
StreamClosedError: Stream is closed

Access-Control-Allow-Credentials

In file tornadio2/preflight.py
there is a check

if 'Cookie' in self.request.headers:
self.set_header('Access-Control-Allow-Credentials', 'true')

If I set up new server and I have no any cookie it in not working.
If I commented out this check socketio works fine.

Maybe server should add this header to response in any case if it got cookie or not.

Multiplexed connection does not receive query string arguments

For example:

class UserConnection(tornadio2.conn.SocketConnection):

    def on_open(self, info):
        print  info.arguments  # {}

class RouterConnection(tornadio2.conn.SocketConnection):
    __endpoints__ = {
        '/user': UserConnection
    }

Connection:

var s = new io.connect('http://' + window.location.hostname + ':8002/user?sid=100500', {
    rememberTransport: false
});

Did I miss something?
I fount one way: save request.arguments in tornadio2.session.Session.__init__ and add them to ConnectionInfo in tornadio2.session.Session.connect_endpoint method.

Access-Control-Allow-Credentials header case

in file preflight.py
self.set_header('Access-Control-Allow-Credentials', True)
should be replaced with
self.set_header('Access-Control-Allow-Credentials', 'true')

Because boolean True serialized as 'True' and will not accepted by browser.
Tested in firefox and chrome.
It should be in lowercase.
When I replaced boolean true with string value it works fine.

WebSocket transport doesn't work in Safari

Tornado 2.2 changed their WebSocketHandler implementation so that it implements RFC 6455 [0] as the default, and only, implementation.

However, the latest Safari 5.1 still only supports draft 76 and there is no planned support for RFC 6455 in Safari 6 either [1].

tornado.websocket.WebSocketHandler can be configured to still support outdated clients by overriding allow_draft76 [2].

I would suggest that an application setting is added that allows you to enable draft 76 support, as Tornadio2 users cannot override the default WebSocketHandler.

[0]
http://tools.ietf.org/html/rfc6455

[1]
http://caniuse.com/websockets

[2]
http://www.tornadoweb.org/documentation/websocket.html#tornado.websocket.WebSocketHandler.allow_draft76

global connection for database

Hello,

I can in tornado make global connect to mongo ( self.db in tornado.web.Application ) and use as self.application.db in handlers ( with tornado.web.RequestHandler)

How can I do this in TornadIO2?

Issues with Chrome and Opera

Just tested examples that are in the source.
This is common for all examples, but I used ackping to post these logs.

My OS is OSX lion (if that's relevant).

Opera 11.60: websockets wont work, nor will flashsocket

DEBUG:root:<<< 1::
INFO:root:200 GET /socket.io/1/?t=1327023775575 (127.0.0.1) 0.92ms
WARNING:root:404 GET /socket.io/static/flashsocket/WebSocketMain.swf (127.0.0.1) 0.66ms
DEBUG:root:Initializing xhr-polling transport.

And goes on with xhr-polling.

Chrome 16.0.912.75: websockets wont work, flashsocket is not even tested

INFO:root:304 GET / (127.0.0.1) 18.03ms
INFO:root:304 GET /socket.io.js (127.0.0.1) 33.66ms
DEBUG:root:<<< 1::
INFO:root:200 GET /socket.io/1/?t=1327024036562 (127.0.0.1) 2.12ms
WARNING:root:404 GET /favicon.ico (127.0.0.1) 0.60ms
DEBUG:root:Initializing websocket handler.
DEBUG:root:Initializing xhr-polling transport.
INFO:root:200 GET /socket.io/1/xhr-polling/c828c0c6e97d475bcc0f02ffbd14c630?t=1327024046839 (127.0.0.1) 1.34ms
DEBUG:root:Initializing xhr-polling transport.

Tests run on localhost, port 80, no cross domain content, root privileges.

Safari tested: works - immediately goes to websockets.
Firefox 9 (with websockets enabled) - works as well, havent tried disabled websockets (which is by default).

I also tried node socket.io examples - these work normally with Chrome (websockets transport as well).

Here is what is happening on client-side with Chrome and Firebug.
GET http://localhost/socket.io/1/?t=1327024253984 returns 200 OK:
74cd3f19e29a2d86035ab15f0bfa6dea:12:25:websocket,flashsocket,xhr-polling,jsonp-polling,htmlfile

Then:
GET ws://localhost/socket.io/1/websocket/74cd3f19e29a2d86035ab15f0bfa6dea HTTP/1.1
Origin: http://localhost
Connection: Upgrade
Host: localhost
Sec-WebSocket-Key: hNDhwJlWrIbONI27pRBX5Q==
Upgrade: websocket
Sec-WebSocket-Version: 13

No response.
No error at all as well.

example gen.py not working

On run example gen.py on tornado 2.3 raised exception

ERROR:root:Failed to handle message: Traceback (most recent call last):
  File "/home/pooh/works/testtornado/lib/python2.6/site-packages/tornadio2/persistent.py", line 142, in on_message
    self.session.raw_message(message)
  File "/home/pooh/works/testtornado/lib/python2.6/site-packages/tornadio2/session.py", line 410, in raw_message
    ack_response = conn.on_event(event['name'], args=args)
  File "/home/pooh/works/testtornado/lib/python2.6/site-packages/tornadio2/gen.py", line 113, in wrapper
    run(args, kwargs)
  File "/home/pooh/works/testtornado/lib/python2.6/site-packages/tornadio2/gen.py", line 84, in run
    data.runner = SyncRunner(gen, finished)
  File "/home/pooh/works/testtornado/lib/python2.6/site-packages/tornadio2/gen.py", line 45, in __init__
    super(SyncRunner, self).__init__(gen)
TypeError: __init__() takes exactly 3 arguments (2 given)

I replace in file tornadio2/gen.py line 84
from

data.runner = SyncRunner(gen, finished)

on

data.runner = Runner(gen, finished)

I think class SyncRunner no more need

Room support

Are there any plans for adding room support into tornadio2?

https://github.com/LearnBoost/socket.io/wiki/Rooms

Endpoints dont get info.arguments

Прежде всего спасибо за вашу работу.

Я не уверен, что это баг, но мне такое поведение кажется странным.

Если определены SockectConenction вида:

class ChatConnection(SocketConnection):
    def on_open(self, info):
        self.send('Chat open connection')
        self.send(info.arguments)


class RouterConnection(SocketConnection):
    def on_open(self, info):
        self.send('Router open connection')
        self.send(info.arguments)

    __endpoints__ = {'/chat': ChatConnection,
                    '/ping': PingConnection}

и осуществляется подключение с аргументами

    var chat = new io.connect('http://localhost:8001/chat?test=chat');

То при подключении к chat каналу , сначала создается объект типа RouterConnection и вызывается его метод on_open (здесь query аргументы от сервера приходят), потом создается объект ChatConnection и вызывается его метод on_open, но здесь уже не приходят аргументы от сервера.

Вопрос: Так и должно быть? Я ожидал что при подключении на localhost/chat будет создан только подключение ChatConnection и до него должны дойти аргументы запроса (request.arguments).

Scalability (sessions)

I'm looking into ways of scaling our tornadio2 setup. At this moment our stack is like

stud > haproxy > nginx (static) / tornadio2 (websockets/longpolling)

and this works wonderful, but adding another tornadio2 server to the setup and use haproxy to loadbalance it I get "HTTPError: HTTP 401: Unauthorized (Invalid Session)" which make sense, because one instance doesn't know about the sessions from the other instance.

In the documentation (http://tornadio2.readthedocs.org/en/latest/deployment/#scalability) I read about a memcached/redis backed session store, which was the first thing i was thinking about, is there any example around how to implement this in tornadio2? I found some examples for tornado but they look really different from the way tornado2 works.

The other option, using a transport mechanism to communicate between servers sounds interesting to because we already use rabbitmq to get the client statements to workers and the results back. Any tips or examples how to start with this?

The best way to implement authentication

Maybe you thought about authentication or have made it in projects. I'd be grateful for some hints about authentication in Tornadio2.

The most easy way I see is to check request in tornadio2.router.HandshakeHandler.get and prevent session create for inappropriate users. So for any try to send messages not authenticated user will receive HTTPError(401, 'Invalid session').

tornadio cluster behind a load balancer

Hi I am trying to set up a cluster of tornado servers behind a load balancer and noticed that SessionContainer is just a dict. This means I MUST send clients to the same server every request. Maybe you already thought about this? What I want to do is implement a memcached session backend so it doesn't matter on which server you end up for each request.

Or maybe you've got some other ideas?

SessionContainer.remove does not remove session from self._queue

Way to reproduce:

1. add session#1
2. remove session#1
3. create session#2 with session#1.session_id
4. call SessionContainer.expire, our new session is removed.

I understand that it is not common situation, but in my opinion SessionContainer.remove should totally remove remove session.

None of the examples work

when starting any of the example servers, this occurs:

tornadio2/examples/transports >>> python transports.py
INFO:root:Starting up tornadio server on port '8001'
Traceback (most recent call last):
File "transports.py", line 55, in
tornadio2.server.SocketServer(application)
File "/usr/lib64/python2.6/site-packages/TornadIO2-0.0.1-py2.6.egg/tornadio2/server.py", line 58, in init
self.listen(socket_io_port, socket_io_address)
File "/usr/lib64/python2.6/site-packages/tornado/netutil.py", line 100, in listen
sockets = bind_sockets(port, address=address)
File "/usr/lib64/python2.6/site-packages/tornado/netutil.py", line 265, in bind_sockets
sock.bind(sockaddr)
File "", line 1, in bind
socket.error: getsockaddrarg: bad family
tornadio2/examples/transports >>> python --version
Python 2.6.7
tornadio2/examples/transports >>>

---

Cloned from git today, Tornado version 2.1.1

---

EDIT: Upon further investiagtion, the issue seems to be that session.py (specifically line 65) seems to think that tornado.httpserver.HTTPRequest has a 'cookies' attribute. It doesn't :P

Safari wss Invalid certificate chain

For example whe have:

• example.com <- simple site with socket.io
• example.com:8080 <- tornadio2 with ssl mode

In Safari when js connect to the server on host wss://example.com:8080/socket.io/1/websocket/121a6ae3852addac4c159379a585a27a
that return error "WebSocket network error: OSStatus Error -9807: Invalid certificate chain"

but in Chrom, Firefox working correctly.

My server config:
Nginx

nginx version: nginx/1.3.6
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.3.6/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.3.6/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.3.6/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.3.6/debian/modules/nginx-dav-ext-module

example.com

server {
    listen 80;
    server_name example.com www.example.com;

    rewrite ^(.*)$ https://example.com$1 permanent;
}

server {
    listen 443 ssl;

    server_name example.com www.example.com;

    ssl on;
    ssl_certificate /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;

    access_log /var/log/nginx/example.access.log;
    error_log /var/log/nginx/example.error.log;

    root  /var/www/example/htdocs/;
    expires +1M;

    location / {
            include uwsgi_params;
            uwsgi_pass unix:///run/uwsgi/app/example/socket;
    }

    location /websocket_status {
        check_status;
    }

}

example.com:8080

upstream cluster {
    server 127.0.0.1:8001;

    check interval=3000 rise=2 fall=5 timeout=1000;
}

server {
    listen 8080;

    ssl on;
    ssl_certificate /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;

    proxy_pass cluster;
}

Thanks.

can't get transports demo to work

seems to serve the main page OK, as well as socket.io.js. when I click "connect" though, it logs "connecting" but doesn't get any further. same in firefox & chromium.

in the server logs, I see:

2011-11-07 16:35:44,642 INFO:Starting up tornadio server on port '8001'                                       
2011-11-07 16:35:44,643 INFO:Starting Flash policy server on port '843'                                       
2011-11-07 16:35:44,643 INFO:Entering IOLoop...                                                               
2011-11-07 16:36:29,634 INFO:200 GET / (192.168.0.181) 4.91ms                                                 
2011-11-07 16:36:29,729 INFO:200 GET /socket.io.js (192.168.0.181) 21.00ms

On the client-side, if I open up the firebug console, I see it tries to make a bunch of ajax GET requests to, eg:

http://localhost:8001/socket.io/1/?t=1320683800628

but it doesn't receive a response.

enabling & disabling the various different transport types doesn't seem to make a difference

python 2.6 on debian, FF 7.0.1, Chromium 14.0.835.202 (Developer Build 103287 Linux)

I realise it's early days! any ideas what might be going wrong?

Python3

Hi,

Please add support for Python3.

I tried packaging tornadio2 for Python 3, for Arch Linux, by using 2to3. This resolved several issues, but there are a few minor issues left.

Thanks.

Authentication

So I store a secure cookie upon verifying the user. When connecting to multiplexed socked connection the cookies are not passed, just an empty cookie is passed. Is there anyway you would suggest to access the cookie or the connectioninfo object of the 'root' socket since that gets the cookie? Also have you had any success in passing the application instance, that would solve all the difficulties I have been running into.

TornadIO2 should allow the user to override message type (Message of Json)

TornadIO2 now chooses the type of message to be sent according to the type of the data. string generates simple messages (type 3) while dict and arrays generate JSON messages (type 4). This is generally true except when the string already represent the JSON encoded version of the object.

TornadIO2 should allow the user to override this behavior.

10s Delay in CrossSite Example

When I run the crosssite example, I'm usually having a 10-second delay between loading the web page (the chat form appearing) and seeing the welcome message ("Welcome from the server. New participant joined"). I experienced this delay in Safari, Chrome, Firefox (8 and 11), and IE9. I've tried viewing on Mac and Windows 7, serving from Mac or Ubuntu.

I've timed the delay several times and it seems to be consistently about 10 seconds long. Sometimes, one of the browsers will consistently have no delay, but then another day when I try again there is a delay again. Chrome and IE9 have always given me a delay 100% of the time. I haven't seen the delay in Firefox 9 (on Windows 7) at all.

Any ideas about this issue?