anateus / asyncboto Goto Github PK

AsyncBoto

Thomas Parslow 2011
[email protected]
http://almostobsolete.net/
twitter: @almostobsolete

Make Boto operate in an Asynchronous way using Tornado.

I've only tested with SimpleDB so far (and then only a little bit)
but you should be able to do the same with all the other types of
connection in Boto.

****THIS REALLY ISN'T SUITABLE FOR PRODUCTION****

DON'T BE STUPID AND USE THIS FOR SOMETHING IMPORTANT. THANKS :)

Please check out Josh Haas's version using Python Greenlets:

http://blog.joshhaas.com/2011/06/marrying-boto-to-tornado-greenlets-bring-them-together/

How to use it
=============

Create an asynchronous version of a Boto connection object my adding
in the mixin, which needs to be supplied as the first base class so
that it overrides the methods in the second. For example, to make an
asynchronous version of the SimpleDB connection you would do this:

 import boto.sdb.connection
 import asyncboto
 class AsyncSDBConnection(asyncboto.AsyncConnectionMixin, boto.sdb.connection.SDBConnection):
   pass

The connection will act just like a normal version of the connection
except it now has a new method you can call. The async_call method
takes a function containing the stuff you want to do and a callback.
It returns immediately (ish), the result will be sent back to you
later via the callback.

The "do stuff" function can make calls on the connection and the
objects it returns as normal. 

WARNING: The function you pass in will be called multiple times as
part of the way this works, so don't do anything with side effects in
their! In fact, you're best of just confining yourself to Boto calls
within that function, save the rest for the callback.

Here's an example of a call:

  sdb_conn = AsyncSDBConnection(aws_access_key_id, aws_access_key_secret)
  def callback(ret):
      print "create_domain returned:", ret
  sdb_conn.call_async(lambda : sdb_conn.create_domain("mytest"), callback=callback)

Of course you'd need to start the Tornado IOLoop to actually see it run:

  tornado.ioloop.IOLoop.instance().start()

Known Problems
==============

Request retries won't work. In fact time the same HTTP request is made
in a loop it won't work.

Performance, a lot of the code in a Boto call will get executed
multiple times. I have no idea how much of an effect this actually has
on performance right now.




That's it for now. This is really early (I only just wrote this) but
I'm interested in what people think about the approach. Please do get
in touch with any comments, suggestions or whatever. I'm on Twitter as
@almostobsolete or my email is [email protected]