Asynchronous api? about rabbitmq-c

[carun]

//Since the existing functions seem to require the calling thread to block in many cases, I don't see a way to do this without introducing a new thread.
Please check this comment by Alan. I've done similar implementations in my application when I din't want my high priority threads to get blocked.

In addition, if you can have your design in such a way that the connections (amqp_connection_state_t) are not shared across threads, you are safe (just expressing the possibility, albeit you already mentioned that you prefer not to have multiple threads in your app).

[cjcarlino]

Thanks for the reference. I had read that before. Let's call that a semi-synchronous solution. I was wondering if anyone was thinking/planning a more fully asynchronous api: i.e. one that supported O_NONBLOCK sockets. If we were to propose a good one, would there be much chance of acceptance? Or have folks pretty much decided already that it's not needed and/or not worth the code complexity and/or not portable.... To be clear, we have not yet designed or implemented anything, we're just considering our options.

[alanxz]

Currently there are no concrete plan for an async API.

That said, I have seen increasing interest in rabbitmq-c supporting features that require an asynchronous API (such as support for heartbeat, better support for handing returns of messages when a basic.publish fails, consumer cancellation, etc).

I've thought quite a bit about the different forms that an async API can take, and I think the way I would go initially would be to provide an API similar to the Java and .NET APIs that have a thread per connection servicing the async bits of the AMQP spec, with call backs happening for async events.

This I believe would require a good deal of work at least with way the library is currently structured. The hardest part in my estimation would be building a cross-platform thread shim, which at best is a hard thing to do.

[cjcarlino]

Thanks for the reply.

I guess I was hoping for an interface that would allow the app to remain thread-free (more light-weight to my way of thinking). I expect that the same set of call backs would happen, but via some other function that the app calls when select indicates a connection's fd. I also expect that you've considered this and probably are not happy with what that would do to the code structure. If so, I agree.

But is the multi-threaded approach too heavy for some applications (memory, cpu)? I have no experience with embedded applications, so I don't have much of a feel for this. For now, this would not be much of a problem for us, but there's some chance we will need to operate in a much more limited environment in the future (I have no details about that, but this possibility along with my lack of experience makes me want to be conservative).

Would the call backs occur in the per-connection threads so that the app would need some concurrency control? Again, for us this is not much of a problem right now, since we have only 1 connection, and we already have a thread-safe event delivery mechanism built into our select loop. There may be more connections in our future, but again this is very fuzzy.

[cjcarlino]

One other question that just occurred to me. If the async API uses threads that emit events via call backs, would the fd still be exposed to the app?

[alanxz]

Part of the AMQP protocol is async, so in order to immediately act on async events, something has to execute code to deal with the data received. The traditional AMQP client (the official Java and .NET clients) have a single thread servicing async event call backs for each connection. In this case the callback would execute in the context of the connection thread, so it would be the responsibility of the application not to do anything too heavy in the call back, to pass things taking a long time to another thread.

I don't see this as the only way of doing this: I could also imagine servicing an AMQP connection as a part of an external event loop. That said not everyone has an event loop as a part of their application, so I think the way to develop something where you can plug your own event loop is to develop it as a traditional client so you know where the bits need to plug in, then extend is so someone can provide their own.

To answer your first question: I could see providing a method that tries to service callbacks (by looking at the result of a select() call) with the current code.

If you things happening in other threads you will at some point need some way to communicate between threads, thus some form of concurrency control is required (mutual exclusion most likely).

If I were going to build an AMQP library with a thread handling the connection, I would not expose the socket. It would result in too many difficult to track bugs if someone was screwing with the socket while the library was trying to do things to it from the connection thread.

If having a thread sitting mostly idle is too resource intensive, AMQP probably isn't the right technology. Something like STOMP might be a better alternative if you're looking to use the RabbitMQ broker. That said if you're operating on something like a smart phone, threading is usually something that is encouraged for responsiveness.

Hope this answers your question.

[cjcarlino]

"I don't see this as the only way of doing this: I could also imagine servicing an AMQP connection as a part of an external event loop. That said not everyone has an event loop as a part of their application, so I think the way to develop something where you can plug your own event loop is to develop it as a traditional client so you know where the bits need to plug in, then extend is so someone can provide their own."

This sounds interesting, but I want to be sure I understand. Are you saying that we start with the existing library and write a thread-per-connection client? From that we morph the client to the select/event model to show us how to change the library. Probably what we gain from that is insight to a (hopefully) reasonable set of callbacks, right? If I have the right idea here, then I think we're at least part of the way to the "traditional client" already. We have a single-connection client that just runs synchronously right now, and we'll be adding (for the short term) a thread dedicated to that single connection so the other parts of the process can meet their latency requirements.

[alanxz]

This sounds interesting, but I want to be sure I understand. Are you saying that we start with the existing library and write a thread-per-connection client?

Yes I would use the current rabbitmq-c client as a starting point. Many of the internal APIs dealing with sending and receiving data would likely change radically, its hard to comment on how the public APIs would change, but I think there would necessarily be some backwards incompatible changes.

From that we morph the client to the select/event model to show us how to change the library. Probably what we gain from that is insight to a (hopefully) reasonable set of callbacks, right?

Yes. The connection-thread essentially runs its own event loop. Developing it this way first would allow us to develop an API for plugging loops without having to expose the internal details how how we plug the event loop into things. Once we're comfortable with how we've done the internal event loop we can expose a publicly stable event-loop API that developers can develop against. The async callback APIs would necessarily be made public with the release of the initial threaded client.

If I have the right idea here, then I think we're at least part of the way to the "traditional client" already. We have a single-connection client that just runs synchronously right now, and we'll be adding (for the short term) a thread dedicated to that single connection so the other parts of the process can meet their latency requirements.

Yes.