michaelklishin / langohr

A small, feature complete Clojure client for RabbitMQ that embraces AMQP 0.9.1 model

Langohr, a feature-rich Clojure RabbitMQ client

Langohr is a Clojure RabbitMQ client that embraces AMQP 0.9.1 Model.

Project Goals

• Embrace AMQP 0.9.1 Model. Follow Java client's API conventions instead of inventing new overly opinionated ones
• Be well documented. Use Ruby amqp gem guides as a foundation.
• Be well tested.
• Error handling and recovery should be well covered
• Support all of the RabbitMQ features, include extensions to AMQP 0.9.1.
• Make error handling and recovery easier

We've learned a lot from over 6 years history of the Ruby amqp gem, Bunny, and RabbitMQ Java client development and try to apply this experience to Langohr design.

Project Anti-Goals

Here is what Langohr does not try to be:

• A replacement for the RabbitMQ Java client
• Sugar-coated API for task queues that hides all the protocol machinery from the developer
• A port of Bunny to Clojure

Artifacts

Langohr artifacts are released to Clojars. If you are using Maven, add the following repository definition to your pom.xml:

<repository>
  <id>clojars.org</id>
  <url>http://clojars.org/repo</url>
</repository>

The Most Recent Release

With Leiningen:

With Maven:

<dependency>
  <groupId>com.novemberain</groupId>
  <artifactId>langohr</artifactId>
  <version>5.4.0</version>
</dependency>

Documentation & Examples

If you are only starting out, please see our Getting Started guide.

Documentation guides:

• AMQP 0.9.1 Concepts
• Connecting To The Broker
• Queues and Consumers
• Exchanges and Publishing
• Bindings
• Durability
• TLS/SSL

API Reference

For existing users, there is API reference.

Code Examples

Several code examples used in the guides are kept in a separate Git repository.

Our test suite also can be used for code examples.

Supported Clojure Versions

Langohr requires Clojure 1.6+. The most recent stable release is highly recommended.

Supported RabbitMQ Versions

Langohr depends on RabbitMQ Java client 5.x and requires a supported RabbitMQ version.

Project Maturity

Langohr has been around since 2011. The API is stable.

Community

Langohr has a mailing list. Feel free to join it and ask any questions you may have.

To subscribe for announcements of releases, important changes and so on, please follow @ClojureWerkz on Twitter.

Langohr Is a ClojureWerkz Project

Langohr is part of the group of libraries known as ClojureWerkz, together with

• Elastisch
• Cassaforte
• Monger
• Welle
• Neocons
• Quartzite
• Titanium

Continuous Integration

Development

See CONTRIBUTING.md.

License

Copyright (C) 2011-2022 Michael S. Klishin and the ClojureWerkz Team.

Double licensed under the Eclipse Public License (the same as Clojure) or the Apache Public License 2.0.

langohr's Issues

Exception during manual topology recovery

Hi!

We are attempting to harden some code against restarts of RabbitMQ, and haven't had much luck so far.

We would like to tolerate the case where the RabbitMQ server is down briefly, e.g. in the case of a restart (sudo service rabbitmq-server restart).

We adapted the example clojurewerkz.langohr.examples.recovery.example1 at http://clojurerabbitmq.info/articles/error_handling.html (Gist: https://gist.github.com/msszczep/11266360). Slight differences include connection parameters and the removal of the lb/publish call (since we're filling the data with an external source).

We can successfully consume data from the queue and wait for more messages. However, when we restart RMQ, the following exception is thrown:

Caught an exception during connection recovery!
java.io.IOException
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:106)
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:102)
    at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:378)
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:516)
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:545)
    at com.novemberain.langohr.Connection.recoverConnection(Connection.java:166)
    at com.novemberain.langohr.Connection.beginAutomaticRecovery(Connection.java:115)
    at com.novemberain.langohr.Connection.access$000(Connection.java:18)
    at com.novemberain.langohr.Connection$1.shutdownCompleted(Connection.java:93)
    at com.rabbitmq.client.impl.ShutdownNotifierComponent.notifyListeners(ShutdownNotifierComponent.java:75)
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:573)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:67)
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:33)
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:343)
    at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:321)
    ... 8 more
Caused by: java.io.EOFException
    at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:273)
    at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
    at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:533)

Are the documented restart semantics the correct way to go here? Should they work in this case, and if so, could they be made to work easily? Alternatively, if we were to implement our own reconnection semantics, what would be the simplest way to monitor the connection? Any suggestions for how to best implement a tolerance for brief (but "hard") outages would be most welcome.

Thanks!

Server-initiated channel close w/ error should raise an exception

I've recently had a bug in my code where I called basic/ack more than once on the same message. Modern versions of RabbitMQ respond to this by closing the channel.

However -- langohr did not report event with an exception, but silently ate it; the only exception which bubbled up was later, on future attempts to use the channel, at which point it was already closed -- and at which point the exception given was a generic attempt-to-use-closed-channel error, not mentioning the original message given by the RabbitMQ server pointing to the duplicate-acks bug.

typo in langohr.shutdown: sort-error? should probably be soft-error?

I guess sort-error? should be renamed to soft-error?

Upgrade RabbitMQ Java client to 3.5.4

Consider providing serialization-aware consumer API

Many scenarios in message consumer applications are very common. For example, JSON or MsgPack used for serialization. We should investigate a convenient way to plug common serialization formats into consumer API. Then payload attribute can be a data structure instead of a bag of bytes.

ThreadFactory and other GAE-related customizations

RabbitMQ Java client 3.3.0 allows the user to provide a custom thread factory that will be used to instantiate various threads the client uses. This is needed for the client to work on GAE, which has certain restrictions.

Langohr should expose these features.

Design connection error handling

RabbitMQ Java client provides an opinionated way of detecting errors but not recovering from them. amqp gem experience shows what can be done (I can't say it is an idea approach, but we did well given all the backwards compatibility constraints). Langohr needs something similar that fits functional/Clojuric way of doing things.

recoverConnection does not use :executor

Connection.init reads

  public Connection init() throws IOException {
    ExecutorService es = (ExecutorService)this.options.valAt(EXECUTOR_KEYWORD);
    this.delegate = cf.newConnection(es);

while reoverConnection forgets about :executor

  private void recoverConnection() throws IOException {
    this.delegate = this.cf.newConnection();
  }

Channel shutdown hooks are not recovered

Langohr's Connection should delegate toString

Default values for core/connect automatically-* flags is not always "true"

Documentation states that following options for langohr.core/connect have default value of "true":

:automatically-recover
:automatically-recover-topology

But it is correct only when no parameters are passed to langohr.core/connect, due to the way Connection.java handles default options.

Proof: add this failing test

(deftest t-default-recovery
  (let [conn (lc/connect {:host "localhost"})]
    (is (= true (lc/automatic-topology-recovery-enabled? conn)))))

3.2.0 java client incompatible

javac error:

/langohr/src/java/com/novemberain/langohr/Connection.java:18: error: com.novemberain.langohr.Connection is not abstract and does not override abstract method clearBlockedListeners() in com.rabbitmq.client.Connection
public class Connection implements com.rabbitmq.client.Connection, Recoverable {
       ^

Passing :automatically-recover false or nil to langohr.core/connect has no effect

Line 177 and 188 as seen here:
https://github.com/michaelklishin/langohr/blob/master/src/java/com/novemberain/langohr/Connection.java#L177

should not check containsKey but rather valAt and convert nulls to false.

Recovery of consumer and threading semantics

I'm running into some issues around managing the state of the connection to RabbitMQ. In a nutshell, in the case of an exception during consumer processing, I need to tear down the Rabbit connection and establish a new one. However, this is difficult because basicConsume appears to run the actual consumer fn in a thread pool. This means that exceptions thrown in the consumer cannot be caught & handled by the code which sets up the connection, which means it can't get the signal that it needs to reconnect.

I've dealt with Rabbit before and haven't run into this issue, so I'm not sure if I'm going crazy or what. This example code seems to assume that langocr.consumers/subscribe blocks: https://github.com/clojurewerkz/langohr.examples/blob/master/src/clojure/clojurewerkz/langohr/examples/blabbr.clj

The docs also indicate that it will block:
“We use langohr.queue/subscribe to start a blocking consumer. Because the consumer will loop waiting for messages forever, we start it in a separate thread…”

However, this is not what I'm seeing. I'm using langohr 1.0.0-beta13 & amqp-client 3.0.2.

Here's my modified example code:

(defn handler [username ch metadata ^bytes payload]
  (println (format "[consumer %s] %s received %s"
                   (Thread/currentThread)
                   username (String. payload "UTF-8"))))

(defn start-consumer
  "Starts a consumer bound to the given topic exchange in a separate thread"
  [ch topic-name username]
  (let [queue-name (format "nba.newsfeeds.%s" username)]
    (lq/declare ch queue-name :exclusive false :auto-delete true)
    (lq/bind    ch queue-name topic-name)
    (println (format "Consumer tag: %s"
                     (lc/subscribe ch queue-name (partial handler username)
                                   :auto-ack true)))))

(defn -main
  [& args]
  (let [conn  (rmq/connect)
        ch    (lch/open conn)
        ex    "nba.scores"
        users ["joe" "aaron" "bob"]]
    (le/declare ch ex "fanout" :durable false :auto-delete true)
    (doseq [u users]
      (start-consumer ch "nba.scores" u))
    (println "publishing")
    (lb/publish ch ex "" "BOS 101, NYK 89" :content-type "text/plain" :type "scores.update")
    (lb/publish ch ex "" "ORL 85, ALT 88"  :content-type "text/plain" :type "scores.update")
    (println (format "[coordinator %s] sleeeeping" (Thread/currentThread)))
    (Thread/sleep 10000)
    (rmq/close ch)
    (rmq/close conn)))

And its output:

Consumer tag: amq.ctag-e05_-YsOoow_y38uyZc_6A
Consumer tag: amq.ctag-ZQ3Tkuo8r3DivLd6tzfYnA
Consumer tag: amq.ctag-A_sKoUOFBgs-ubKQaNFP2Q
publishing
[coordinator Thread[main,5,main]] sleeeeping
[consumer Thread[pool-1-thread-4,5,main]] aaron received BOS 101, NYK 89
[consumer Thread[pool-1-thread-4,5,main]] aaron received ORL 85, ALT 88
[consumer Thread[pool-1-thread-5,5,main]] bob received BOS 101, NYK 89
[consumer Thread[pool-1-thread-5,5,main]] bob received ORL 85, ALT 88
[consumer Thread[pool-1-thread-5,5,main]] joe received BOS 101, NYK 89
[consumer Thread[pool-1-thread-5,5,main]] joe received ORL 85, ALT 88

I honestly have no idea why this would be happening. I noticed the issue because some code I wrote that reconnects when the langohr.consumers/subscribe function returns or throws and exception was flapping — messages were getting processed, but it was disconnecting and reconnecting in a tight loop.

What on earth is going on here?

Issue running tests locally

Hi!

I'm trying to run the tests locally. When I run lein all test, I see the following exception:

Performing task 'test' with profile(s): 'dev'
Reflection warning, langohr/core.clj:191:3 - call to method addQueueRecoveryListener can't be resolved (target class is unknown).
Using Clojure version {:major 1, :minor 6, :incremental 0, :qualifier nil}
Reflection warning, clj_http/multipart.clj:26:4 - call to org.apache.http.entity.mime.content.FileBody ctor can't be resolved.
Reflection warning, cheshire/core.clj:63:3 - call to method createJsonGenerator on com.fasterxml.jackson.core.JsonFactory can't be resolved (argument types: unknown).

lein test langohr.test.basic-test

lein test langohr.test.channel-test

lein test langohr.test.confirm-test

lein test langohr.test.consumers-test

lein test langohr.test.core-test

lein test :only langohr.test.core-test/test-connection-with-multiple-hosts

ERROR in (test-connection-with-multiple-hosts) (AMQConnection.java:340)
Uncaught exception, not in assertion.
expected: nil
  actual: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
 at com.rabbitmq.client.impl.AMQConnection.start (AMQConnection.java:340)
    com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection (RecoveryAwareAMQConnectionFactory.java:36)
    com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init (AutorecoveringConnection.java:83)
    com.rabbitmq.client.ConnectionFactory.newConnection (ConnectionFactory.java:609)
    com.novemberain.langohr.Connection.init (Connection.java:90)
    langohr.core$connect.invoke (core.clj:93)
    langohr.test.core_test/fn (core_test.clj:42)
    clojure.test$test_var$fn__7187.invoke (test.clj:704)
    clojure.test$test_var.invoke (test.clj:704)
    clojure.test$test_vars$fn__7209$fn__7214.invoke (test.clj:722)
    clojure.test$default_fixture.invoke (test.clj:674)

The rabbitmq logs appear normal:

=INFO REPORT==== 23-Dec-2014::21:54:29 ===
Server startup complete; 10 plugins started.
 * rabbitmq_management_visualiser
 * rabbitmq_management
 * rabbitmq_web_dispatch
 * webmachine
 * mochiweb
 * rabbitmq_mqtt
 * rabbitmq_stomp
 * rabbitmq_management_agent
 * rabbitmq_amqp1_0
 * amqp_client

=INFO REPORT==== 23-Dec-2014::21:55:07 ===
accepting AMQP connection <0.397.0> (127.0.0.1:60179 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
accepting AMQP connection <0.407.0> (127.0.0.1:60180 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
closing AMQP connection <0.407.0> (127.0.0.1:60180 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
accepting AMQP connection <0.423.0> (127.0.0.1:60181 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
closing AMQP connection <0.423.0> (127.0.0.1:60181 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
accepting AMQP connection <0.436.0> (127.0.0.1:60182 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
closing AMQP connection <0.436.0> (127.0.0.1:60182 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
accepting AMQP connection <0.452.0> (127.0.0.1:60183 -> 127.0.0.1:5672)

=INFO REPORT==== 23-Dec-2014::21:55:15 ===
closing AMQP connection <0.452.0> (127.0.0.1:60183 -> 127.0.0.1:5672)

My environment is OS X 10.10.1, installed rabbitmq version 3.4.1 using Homebrew, and I have leiningen version 2.50. Has anyone else come across this? Did I miss a setup step?

Thanks,
Bill

langohr's version is still set to 1.0.0-beta13-SNAPSHOT

langohr.core/version is set to 1.0.0-beta13-SNAPSHOT.

Exchange/queue deletion via any channel should result in the exchange/queue not being recovered

Langhor's new topology recovery tracks exchange/queue creation per channel. Ideally this should be specific to the vhost (even across connections), so that if an exchange or queue is created via channel A and deleted via channel B, it will not be recovered if channel A's recovery occurs.

Connection recovery tests fail on JDK 8

Support lists of node addresses

We used to support it but no longer do after connection recovery was introduced. Not only the Java client does but it also uses the list of nodes during recovery, which makes total sense.

Upgrade RabbitMQ Java client to 3.3

Exception handler support

RabbitMQ Java client 3.3.0 allows the user to provide an exception handler. Langohr should support this functionality.

add ability to specify a thread pool for consumers

there's currently no easy way to set a custom threadpool/executor to be used when creating a connection. I've been using the following hackish solution with langohr 1.0:

(defn connect-with-thread-pool
  [rmq-settings thread-pool]
  (let [cf (#'rmq/create-connection-factory rmq-settings)]
    (.newConnection ^ConnectionFactory cf thread-pool)))

using this connection object to open a channel fails with langohr 1.1:

java.lang.ClassCastException: com.rabbitmq.client.impl.AMQConnection cannot be cast to com.novemberain.langohr.Connection
 at langohr.channel$open.invoke (channel.clj:23)

Shouldn't I be able to use that Connection object? (yes, I've seen the Connection object implementation in Connection.java that delegates to a real Connection object)

Can we add the functionality to pass in a thread pool/custom executor?

documentation incorrect for http://reference.clojurerabbitmq.info/langohr.basic.html#var-publish

publish does not use varagrs for properties however example in docs show that it does.

Should Langohr docs warn against sharing channels between threads?

In https://www.rabbitmq.com/api-guide.html the docs state clearly that channel instances shouldn't be shared across threads. We recently experienced a deadlock that looks likely to be caused by failure to abide by this. Should a warning about this be included in the Langohr docs somewhere since these may be a significant source of understanding of RMQ semantics for many people (as they were for us)? The supplied examples in the Langohr docs actually violate this recommendation / requirement as well (see http://clojurerabbitmq.info/articles/getting_started.html).

The execution hangs when nested publish inside consumer handler function

This issue could be just be me doing something weird that shouldn't be doing. Please let me know if that's the case.

This issue happens on more complicated code but I simplified it to this. Consuming from a queue name in then republishing to a queue name out. It works fine if there are little number of messages in the in queue. But if there are a bit more messages in the queue (more than 10k messages on my machine), when I evaluate the snippet below, the execution will hang at the out queue declaration expression (lq/declare out-ch "out")

(let [in-ch (lch/open conn)]
  (lq/declare in-ch "in")
  (lcon/subscribe in-ch "in" (fn [channel meta message]
                               (let [out-ch (lch/open conn)]
                                 (lq/declare out-ch "out")
                                 (lb/publish out-ch "" "out" (str message))
                                 (lcore/close out-ch))
                               (lb/ack channel (:delivery-tag meta)))))

There are a few messages successfully published to the out queue. A few thousand messages are Unpacked and the remaining messages are Ready in the in queue. Note that the rabbitmq server doesn't reach memory or disk limited.

If I move the queue declaration to outside the consumer function like below, it changes to hang at close channel line (lcore/close out-ch)

(let [in-ch (lch/open conn)]
  (lq/declare in-ch "in")
  (lq/declare in-ch "out")
  (lcon/subscribe in-ch "in" (fn [channel meta message]
                               (let [out-ch (lch/open conn)]
                                 (lb/publish out-ch "" "out" (str message))
                                 (lcore/close out-ch))
                               (lb/ack channel (:delivery-tag meta)))))

Any idea what have I done incorrectly?

Consumer API improvements

Today if I want to manually ack and nack messages in a subscription I have to do something like this:

(lhcons/subscribe 
  channel queue (fn [delivery headers payload] 
                  (let [envelope (.getEnvelope delivery)
                        delivery-tag (.getDeliveryTag envelope)]
                    (try 
                      (message-handler delivery headers payload)
                      (lhb/ack channel delivery-tag)
                      (catch Exception e
                        (prn (.getMessage e) (.printStackTrace e))
                        (lhb/nack channel delivery-tag false true))))))

which isn't very pretty, but it's because you don't have access to the channel in the message-handler. Should the channel be sent to the message-handler? How do we do that without breaking back-compatibility? As a new function maybe..

Also I think that delivery should be replaced with envelope as it's the only thing which isn't extracted from delivery already (http://www.rabbitmq.com/releases/rabbitmq-java-client/v2.8.1/rabbitmq-java-client-javadoc-2.8.1/com/rabbitmq/client/QueueingConsumer.Delivery.html).

I also find it weird that auto-ack is false by default, now when it's so cumbersome to manually ack..

responses from langohr.http showing HTML when no result found

Hello,
I've noticed that when there is no result or error in using the langohr.http functions, we get the html response from the management plugin. It'd be nice to handle some things like 404 and return nil or throw an appropriate exception. I'm happy to contribute a patch if you have a direction you prefer.

Arguments argument to publish

The documentation seems to refer to a keyword arguments :arguments for basic/publish (http://clojurerabbitmq.info/articles/exchanges.html#toc_29). I'm having difficulty understanding how this works and there doesn't seem to be any reference to this field in the source code. It's not in the docstring and passing values in on that key seems to have no effect.

Am I missing something here or is the documentation wrong?

Thanks in advance for your patience if I'm missing something terribly obvious.

langohr.queue/declare doesn't behave as the docstring

Execute (doc langohr.queue/declare) shows a docstring that gives a usage example that says

  ;; creates named non-durable, exclusive, autodelete queue
  (lhq/declare channel queue-name :durable false :exclusive true :auto-delete true)

But If I execute the code (with the ns, channel and queue-name prepared), the REPL throws an ArityException.

Digging into the code, it seems

  (lhq/declare channel queue-name {:durable false :exclusive true :auto-delete true})

is the right way to do this.

I notice the latter form is used in the hello world example here, but the former is used in the API reference here.

how to check for the consumer tokens of a particular queue

when the consume method is called, it returns a token that represents the consumer

However, if I just have a channel and a queue name, how would I be able to access all tokens that are currently avaliable on the queue?

Basic get (pulling API) always sends acks

I tried to use langohr.basic/get with explicit ack, but it seems to not work as expected. Acks are sent anyway.

I looked for a test of the function and I found this code, but I didn't understand its idea. Shouldn't it send some explicit ack?

Discordance in documentation about `lcons/subscribe`

In Getting Started (section start a consumer), it's said that the function starts a new thread; in Working with queues and consumers (section Convenience method), it's said that the calling thread is blocked.

Switch to forgiving exception handler by default

See RabbitMQ 3.5.4 release notes. This depends on #75.

:automatically-recover fails with java.lang.ClassCastException: com.novemberain.langohr.Connection$1 cannot be cast to java.lang.Comparable

I see ClassCastException (stacktrace below) when I enable automatically-recover (using v1.4.0):

     (let [conn (rmq/connect 
                        {:uri "amqp://..."
                         Connection/AUTOMATICALLY_RECOVER_KEYWORD true})] ...)

If I comment out the Connection/AUTOMATICALLY_RECOVER_KEYWORD line, it works fine.

Also, it seems to me that the default option Connection/AUTOMATICALLY_RECOVER_KEYWORD true will almost never be used. The default is only used if no options are specified at all. Except for the most trivial case, I expect there will always be at least some combination of user/pass/uri/host/port. I also thought of merging my options in with (Connection/buildDefaultOptions), but can't do that because that method is private.

The stack trace is...

java.lang.ClassCastException: com.novemberain.langohr.Connection$1 cannot be cast to java.lang.Comparable
at java.util.concurrent.ConcurrentSkipListMap.comparable(ConcurrentSkipListMap.java:621)
at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:862)
at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:202)
at com.novemberain.langohr.Connection.addAutomaticRecoveryHook(Connection.java:89)
at com.novemberain.langohr.Connection.init(Connection.java:66)
at langohr.core$connect.invoke(core.clj:72)
at com.climate.queue.common$connect.doInvoke(common.clj:48)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:601)
at wb_calvin.queue.mq_listener$amqp_conn.invoke(mq_listener.clj:100)

Channel recovery exception fn arity

Should handle-channel-recovery-exception-fn take 2 arguments here?

langohr/src/clojure/langohr/core.clj

Line 267 in eb0ae43

(handle-channel-recovery-exception-fn )))

Not urgent for me at all; just happened to notice when looking for something else.

Name change to "-fn" style was not made everywhere

We had the problem that this didn't do anything in response to a rabbitmqctl stop:

    (let [thread (Thread. #(lc/subscribe ch (:queue configuration) handler :auto-ack true
                                         :handle-shutdown-signal (fn [& args] (prn "shutdown caught" args))
                                         :handle-recover-ok (fn [& args] (prn "recover caught" args))))]

The problem is this:

subscribe in consumers.clj has this code:

        consumer  (create-default channel
                                  :handle-delivery-fn f
                                  :handle-consume-ok      (get cons-opts :handle-consume-ok)
                                  :handle-cancel-ok       (get cons-opts :handle-cancel-ok)
                                  :handle-cancel          (get cons-opts :handle-cancel)
                                  :handle-recover-ok      (get cons-opts :handle-recover-ok)
                                  :handle-shutdown-signal (get cons-opts :handle-shutdown-signal))]

Notice that, except for the most important handle-delivery-fn, there's no -fn suffix.

Now consider the declaration of create-default. It looks like this:

(defn ^Consumer create-default
  "Instantiates and returns a new consumer that handles various consumer life cycle events. See also langohr.basic/consume."
  [^Channel channel &{:keys [handle-consume-ok-fn
                             handle-cancel-fn
                             handle-cancel-ok-fn
                             handle-shutdown-signal-fn
                             handle-recover-ok-fn
                             handle-delivery-fn]}]

Notice that it looks for keys ending in -fn. So the :handle-shutdown-signal value is ignored. As a result, here's what happens when an actual shutdown signal comes in:

    (handleShutdownSignal [^String consumer-tag ^ShutdownSignalException sig]
      (when handle-shutdown-signal-fn
        (handle-shutdown-signal-fn consumer-tag sig)))

The when turns the function into a no-op.

I've confirmed that making the following change to subscribe correctly calls the handler function:

                                   :handle-recover-ok      (get cons-opts :handle-recover-ok)
-                                  :handle-shutdown-signal (get cons-opts :handle-shutdown-signal))]
+                                  :handle-shutdown-signal-fn (get cons-opts :handle-shutdown-signal))]

langohr.consumers/blocking-subscribe should invoke langohr.basic/consume directly

langohr.consumers/blocking-subscribe should invoke langohr.basic/consume directly, without using clojure.core/apply, as it now fails with a function arity exception.

Retrying publishers

RabbitMQ provides publisher confirms but it is a really low level feature that is hard to use. Langohr should try providing a more convenient way of doing things.

Upgrade RabbitMQ Java client to 3.3.4

lc/create-queueing parity with lc/create-default in terms of handlers

I believe langohr.consumers/create-queueing should have parity with langohr.consumers/create-default in terms of handlers.

The former appears to be missing:
handle-cancel-fn, handle-shutdown-signal-fn and possibly handle-delivery-fn.

Although, AFAIK, they are supported by QueuingConsumer (assuming latest java rabbitmq lib: http://www.rabbitmq.com/releases/rabbitmq-java-client/v3.2.2/rabbitmq-java-client-javadoc-3.2.2/com/rabbitmq/client/QueueingConsumer.html).

When connection has failed, error message is not easy to grasp

Whenever Langohr can't connect to the server (e.q. server's down), I get an error:

Exception in thread "main" java.lang.ClassCastException: clojure.lang.Var$Unbound cannot be cast to com.rabbitmq.client.Connection, compiling:(commands.clj:26)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3342)
    at clojure.lang.Compiler$DefExpr.eval(Compiler.java:375)
    at clojure.lang.Compiler.eval(Compiler.java:6470)
    at clojure.lang.Compiler.load(Compiler.java:6902)
    at clojure.lang.RT.loadResourceScript(RT.java:357)
    at clojure.lang.RT.loadResourceScript(RT.java:348)
    at clojure.lang.RT.load(RT.java:427)
    at clojure.lang.RT.load(RT.java:398)
    at clojure.core$load$fn__4610.invoke(core.clj:5386)
    at clojure.core$load.doInvoke(core.clj:5385)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5200)
    at clojure.core$load_lib.doInvoke(core.clj:5237)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:602)
    at clojure.core$load_libs.doInvoke(core.clj:5271)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:602)
    at clojure.core$require.doInvoke(core.clj:5352)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:602)
    at user$eval29.invoke(NO_SOURCE_FILE:1)
    at clojure.lang.Compiler.eval(Compiler.java:6465)
    at clojure.lang.Compiler.eval(Compiler.java:6454)
    at clojure.lang.Compiler.eval(Compiler.java:6455)
    at clojure.lang.Compiler.eval(Compiler.java:6431)
    at clojure.core$eval.invoke(core.clj:2795)
    at clojure.main$eval_opt.invoke(main.clj:296)
    at clojure.main$initialize.invoke(main.clj:315)
    at clojure.main$null_opt.invoke(main.clj:348)
    at clojure.main$main.doInvoke(main.clj:426)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.lang.Var.invoke(Var.java:405)
    at clojure.lang.AFn.applyToHelper(AFn.java:163)
    at clojure.lang.Var.applyTo(Var.java:518)
    at clojure.main.main(main.java:37)
Caused by: java.lang.ClassCastException: clojure.lang.Var$Unbound cannot be cast to com.rabbitmq.client.Connection
    at langohr.channel$open.invoke(channel.clj:21)
    at clojure.lang.AFn.applyToHelper(AFn.java:161)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.core$apply.invoke(core.clj:600)
    at langohr.core$create_channel.doInvoke(core.clj:69)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3337)
    ... 35 more

I think it should rather be something more readable.

URI-based configuration should be publicly accessible

At present, the env-config function in langohr.core is private.

For applications which wish to accept URI-formatted configuration values, this is needlessly restrictive.

Sleep in consumer

More of a question than an issue; seems like (Thread/sleep 10000) in a subscription handler returns immediately, is that right? In that case, why/how?

EXTERNAL for sasl-config

I'm the one who made this PR ruby-amqp/rubybunny.info#27, but when it comes to langohr, the story is different.

We had an error when trying to make a rabbbitmq connection to rabbitmq broker that has ssl authentication plugin enabled. We figured through looking at Java client source code that we have to set DefaultSaslConfig/EXTERNAL to sasl-config to make it works, as in the snippet below.

(rmq/connect { ...
               :sasl-config DefaultSaslConfig/External }

I need your thought and help on these.

• What's an appropriate place to put this example in the langohr.docs?
• Should we wrap DefaultSaslConfig/External in a more friendly fashion? Like for example

{:sasl-config "EXTERNAL"}

I can try to come up with a change PR if you think it's a good way to go.

Recovery callback do not use channel delegate

The code that calls callbacks upon the recovery of a Channel does not use the delegate but the raw Channel implementation from the java library: code. The langohr.consumers/subscribe function expects an instance of com.novemberain.langohr.Channel. That means, if I try to reconnect consumers via these kind of callbacks, I can't use the langohr API. Is this a bug or should I reconnect consumers in some other way?

Connections that take a long time to recover can overflow the stack

Connection.java:recoverConnection calls itself. This can lead to StackOverflowErrors when connection is not recovered for a long time:

java.lang.Thread.State: TIMED_WAITING (sleeping)
    at java.lang.Thread.sleep(Native Method)
    at com.novemberain.langohr.Connection.recoverConnection(Connection.java:157)
    at com.novemberain.langohr.Connection.recoverConnection(Connection.java:158)
    at com.novemberain.langohr.Connection.recoverConnection(Connection.java:158)
    at com.novemberain.langohr.Connection.recoverConnection(Connection.java:158)
    ......
    at com.novemberain.langohr.Connection.beginAutomaticRecovery(Connection.java:102)
    at com.novemberain.langohr.Connection.access$000(Connection.java:20)
    at com.novemberain.langohr.Connection$1.shutdownCompleted(Connection.java:79)
    at com.rabbitmq.client.impl.ShutdownNotifierComponent.notifyListeners(ShutdownNotifierComponent.java:75)
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:573)

Port RabbitMQ Java client integration tests

We need to port RabbitMQ Java client integration tests. Langohr has decent coverage but integration tests is where you really try out feature you may or may not use actively day to day and they also can use as a basis for documentation examples.

Let the Java client perform connection recovery

This will both reduce the amount of code we maintain and may solve #54 (as Java client tests pass on JDK 8).

Consider adding a function that checks for exchange existence and is a predicate

If langohr-exchange/declare-passive is used to find out about an non existing exchange it will throw an IO Error, if there is no such exchange. This is cumbersome, a falsy return value would appear to me more useful.
At any rate, it will also close the used channel and renders it useless:
com.rabbitmq.client.AlreadyClosedException: channel is already closed due to channel error; protocol ...
This behavior is very annoying; in particular in the context of automatic connection recovery etc.

This is true for langohr 3.0.0-rc3.

Re-declaring a queue with new arguments succeeds but doesn't change arguments

Not sure if this is in langohr or in the Java library - if it's in Java, it might be worth at least documenting the behaviour.

If you re-declare a queue with new arguments, such as setting x-dead-letter-exchange, the call succeeds, but the new arguments are not applied. For example:

(lq/declare ch "my.queue" :exclusive false :durable true :auto-delete false})

... some time later:

(lq/declare ch "my.queue" :exclusive false :durable true :auto-delete false :arguments {"x-dead-letter-exchange" "my.dlx"})

The second call will succeed without exceptions, but the dead letter exchange will not have been set.