barryp / py-amqplib Goto Github PK

Tests, examples, etc are missing from source tarball.

Which version of amqplib are you using?
0.6.0, 0.6.1, hg commit 174:aa7a2b480166

Which broker are you using (RabbitMQ?) which version?
QPid 0.6 (Java version)

Which version of Python?
2.6.4 (distributed by python.org), 2.6.5 (distributed by Cygwin)

What steps will reproduce the problem?
1. Start Qpid using default plain-text passwd file. 
2. Run this python code:
    from amqplib import client_0_8 as amqp
    amqp.Connection()

What is the expected output? What do you see instead?
I expect the Connection constructor to return. It never returns. Past issues 
suggest this was fixed between 0.6.0 and 0.6.1. I confirmed that this works on 
my linux machine. On Windows XP it still hangs.

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.2 with some custom patches

Which version of Python?
2.7

What steps will reproduce the problem?
1. Have a client on Windows XP connect to a RabbitMQ broker via a ssh tunnel 
from another host.
2. Let the client sit idle for around a day eventually

What is the expected output? What do you see instead?
Eventually, the RabbitMQ (Linux) side will not see the connection anymore. 
However, the Windows end would still assume it is connected.

Even netstat on the Windows end shows the connection as ESTABLISHED. This is 
true even when the ssh tunnel itself has been taken down.


Please provide any additional information below.
Although the problem might be due to the FIN/RST packets being lost somewhere, 
enabling TCP keepalives seems to be an effective workaround.
It would involve adding the following to _AbstractTransport.__init__(), right 
after the socket is created:

        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)

I'll be happy to submit this as a proper patch if required.

Which version of amqplib are you using?

0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

yes

Which broker are you using (RabbitMQ?) which version?

rabbitmq 1.7.1

Which version of Python?

2.5.4

What steps will reproduce the problem?

>>> import amqplib.client_0_8 as amqp
>>> import cPickle
>>> msg = amqp.Message("hello")
>>> cPickle.loads(cPickle.dumps(msg))

What is the expected output? What do you see instead?

  File "build/bdist.macosx-10.5-
i386/egg/amqplib/client_0_8/serialization.py", line 447, in __getattr__
RuntimeError: maximum recursion depth exceeded


Please provide any additional information below.

In GenericContent.__getattr__ check for __setstate__

        if name == '__setstate__':
            raise AttributeError('__setstate__ not found')

I am using amqplib v1.0.2 (with timeout patch) on Python 3.2.

While the lib is in channel.wait() waiting for a message (which, of course, it 
is most of the time):

- Remove power to the WLAN router of which the computer is connected.
- Wait a few seconds
- Plug power back in

Observe: NOTHING happens. The amqplib client is now just waiting indefinitely, 
will never exit and will never receive a message.

Which version of amqplib are you using?

0.6.1

Which broker are you using (RabbitMQ?) which version?

RabbitMQ

Which version of Python?

Python 2.5.2

What steps will reproduce the problem?

#!/usr/bin/python2.5

from amqplib import client_0_8 as amqp

conn = amqp.Connection(host="localhost:5672", userid="guest",
password="guest", virtual_host="/", insist=False)
chan = conn.channel()

chan.queue_declare(queue="po_box", durable=True, exclusive=False,
auto_delete=False)
chan.exchange_declare(exchange="sorting_room", type="direct", durable=True,
auto_delete=False,)
chan.queue_bind(queue="po_box", exchange="sorting_room", routing_key="jason")

def recv_callback(msg):
  msg.channel.basic_reject(msg.delivery_tag, requeue=False)

chan.basic_consume(queue='po_box', no_ack=True, callback=recv_callback,
consumer_tag="testtag")
while True:
    chan.wait()
chan.basic_cancel("testtag")

chan.close()
conn.close()


What is the expected output? What do you see instead?

Traceback (most recent call last):
  File "amqp_consumer.py", line 17, in <module>
    chan.wait()
  File
"/usr/lib/python2.5/site-packages/amqplib/client_0_8/abstract_channel.py",
line 89, in wait
    self.channel_id, allowed_methods)
  File "/usr/lib/python2.5/site-packages/amqplib/client_0_8/connection.py",
line 218, in _wait_method
    self.wait()
  File
"/usr/lib/python2.5/site-packages/amqplib/client_0_8/abstract_channel.py",
line 105, in wait
    return amqp_method(self, args)
  File "/usr/lib/python2.5/site-packages/amqplib/client_0_8/connection.py",
line 367, in _close
    raise AMQPConnectionException(reply_code, reply_text, (class_id,
method_id))
amqplib.client_0_8.exceptions.AMQPConnectionException: (503,
u'COMMAND_INVALID - unimplemented method', (60, 90), 'Channel.basic_reject')


Please provide any additional information below.

Method appears to be implemented in amqplib/client_0_8/channel.py and the
inline comment/documentation has no mention of this being unusable or
unimplemented? (missing METHOD_MAP entry?)

I have clone the repo with a fix to speeding up the breaking up of the body.

patch: 
http://code.google.com/r/vbabiy-packing-speed/source/detail?r=39cb634d935395964a
8282b649cb52f855aec59d#

Explain:

# Before patch
from amqplib import client_0_8 as amqp
import time

conn = amqp.connection.Connection()
chan = conn.channel()
msg = amqp.Message("1"*(1048576*100))
s = time.time(); chan.basic_publish(msg); print time.time() - s
54.5186219215


# After Patch
from amqplib import client_0_8 as amqp
import time

conn = amqp.connection.Connection()
chan = conn.channel()
msg = amqp.Message("1"*(1048576*100))
s = time.time(); chan.basic_publish(msg); print time.time() - s
0.23295211792

After making a connection to rabbitmq with connect_timeout enabled, amqplib 
does settimeout(None).  This means the connection can hang later on.

Is there any reason to clear the timeout?

hi,
sth from http://www.rabbitmq.com/ha.html#genesis,
amqplib can not support list type? in serialization.py?

Which version of amqplib are you using?
latest

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.2  and/also 2.2

Which version of Python?
2.6.5

What steps will reproduce the problem?
Not all messages are text and/or Unicode.  It is straight forward to wrap 
basic_consume with your own to add some serialization (json, yaml, pickle, et 
al)  However, the interface to basic_consumer does not afford the same chance 
to deserialize the message body prior to passing the Message to the callback 
function.  This means the callback function needs to know how to deserialize 
the message body (if/when required).

Yes, I could write a wrapper/decorator for the callback function but that seems 
so sloppy and unDRY.  However, I see you not wanting to deal with the 
support/insanity that this may bring.  However, if you/we could shim 
basic_consumer to allow for a deserialization function.  Then it would be up to 
the user/developer to specify and implement their own.

Does this feature request interest you?  

Which version of amqplib are you using?

0.6

Which broker are you using (RabbitMQ?) which version?

RabbitMQ 1.5.0

Which version of Python?
2.6.2

What steps will reproduce the problem?
Connect to a RabbitMQ server/vhost with a username/pass that does not have 
permissions.

What is the expected output? What do you see instead?
On py-amqplib 0.5, you'd get this error:

unpack requires a string argument of length 1

Which at least was trappable.

On 0.6 you hang indefinitely...connect never returns.


Please provide any additional information below.

It looks like the client is sending a the broker a heartbeat value, but,
being single threaded, I'm presuming that it's not maintaining actual
heartbeats.

What's the status of this feature? Should heartbeats always be 0?

As a side question, in the absence of heartbeats, what's a good/cheap way
to actively test the broker connection to confirm we can still talk to the
broker? E.g. as a hack (or maybe good design, shrug :) I'm declaring a
non-durable queue whose sole purpose is to let me communicate with the broker.

Page 34 of AMQP 0.8: "Short strings, used to hold short text properties. Short 
strings are limited 
to 255 octets and can be parsed with no risk of buffer overflows."

amqplib-0.6.1, transport.py:

    def read_shortstr(self):
        """
        Read a utf-8 encoded string that's stored in up to
        255 bytes.  Return it decoded as a Python unicode object.

        """
        self.bitcount = self.bits = 0
        slen = unpack('B', self.input.read(1))[0]
        return self.input.read(slen).decode('utf-8')

I'm not convinced you can assume shortstrs will be encoded at all -- returning 
the appropriate 
str object might be a better fit for the protocol. write_shortstr does the 
right thing, though, by 
passing str objects through unchanged.

The error below is raised in a number of situations where the connection to
the broker fails. One of the more easy to recreate is when the
authentication to the broker fails.

Another scenario is when the broker closes unexpectedly.

Ideally, one could handle both cases by catching a specific exception type.

Traceback (most recent call last):
  ...
  File "../src/amqplib/client_0_8/connection.py", line 140, in __init__
    (10, 30), # tune
  File "../src/amqplib/client_0_8/abstract_channel.py", line 89, in wait
    self.channel_id, allowed_methods)
  File "../src/amqplib/client_0_8/connection.py", line 203, in _wait_method
    self.method_reader.read_method()
TypeError: 'NoneType' object is not iterable

I'm not going to use the template, because it seems to be almost entirely 
inapplicable.

We want to connect to our RabbitMQ server with Carrot using IPv6. 
Unfortunately, transport.py hardcodes AF_INET, which means it will always use 
IPv4.

I'm fairly new to Python, but there must be some frontend to getaddrinfo(), 
which allows you to connect in a protocol-neutral way, right?

When an amqplib 1.0.2 connection is asked to close:

1. It sends message (10, 60).
2. It receives message (10, 61).
3. This invokes ._close_ok().
4. Which invokes ._do_close().
5. Which calls .transport.close().
6. Which attempts self.sock.shutdown(...).

This creates an exciting race condition. Which will happen first? Will the 
socket .shutdown() method be successfully called on a still-open socket? Or 
will the FIN packet from the server (RabbitMQ, in my case) arrive fast enough 
to mark the socket as closed before Python gets around to invoking the socket's 
shutdown() system call?

On Linux and Windows, this is an entirely safe race condition — both 
operating systems are very forgiving about calling shutdown() on a closed 
socket, so the above sequence always works.

But on Mac OS X — a BSD variant — the above code only succeeds if the 
operating system thinks that the socket is still open when shutdown() is 
invoked. If Python is too slow and the FIN packet arrives before that statement 
can be reached, then OS X kills the self.sock.shutdown() statement with:

socket.error: [Errno 57] Socket is not connected

As there seems to be no way to check whether a socket is closed — nor, I 
guess, would it help, since the socket could close between such a check and the 
actual shutdown() call! So there are two possibilities here:

1. Protect shutdown() with a try…except that catches the socket.error, tests 
to make sure Errno is right, and ignores it if Errno matches.
2. Ditch the shutdown() altogether. Don't all modern OS's perform a shutdown() 
on a closed socket anyway? I am suspicious of the claim in the transport.py 
comment that outgoing data could be lost; I would want to see evidence that any 
modern TCP/IP stack behaves in that way. I cannot find a recommendation to use 
shutdown() to avoid data loss on Stack Overflow, for example.

Thanks so much for amqplib — I am using it with greenlet/eventlet and the 
project is going very well!

Python 2.6 (32bit) with amqplib 0.6.1 (used through kombu 1.1.6) and RabbitMQ 
2.4.1

I have exactly this problem when sending a message (receiving seems to work):

http://comments.gmane.org/gmane.comp.networking.rabbitmq.general/2265

The code is running on a Win2008 64bit VM running inside Virtual Box on Ubuntu 
10.10.  The VM firewall is disabled.

If I switch the kombu backend from amqplib to pika it all works.

It would be nice to have a PKG-INFO file inside the release tarball. The 
sdist command would automatically do this:

  $ python setup.py sdist

PKG-INFO helps in analyzing the static metadata of any package using 
several tools including `pkginfo` - http://pypi.python.org/pypi/pkginfo

Additionally, PyPM too requires this file to be present in order to build 
a package. This is why amqplib is not installable via PyPM - http://
pypm.activestate.com/list-a.html#amplee

Several files like TODO, CHANGES, ... and directories like docs, demo, ... are 
missing from the source distribution. I've sent a patch to add a MANIFEST.in 
file to include those missing parts of the source code in the distribution 
tarball.

Which version of amqplib are you using?

Best match: amqplib 0.6

Which broker are you using (RabbitMQ?) which version?

rabbitmq-server-1.6.0-1

Which version of Python?

Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39) 
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin

What steps will reproduce the problem?

1. Create or reset a RabbitMQ node.
2. Run the following:

>>> from amqplib import client_0_8 as amqp
>>> con = amqp.Connection(host='localhost', userid='guest', password='garbage')

3. Note that this expression never completes unless interrupted or the server 
goes down.

What is the expected output? What do you see instead?

Ideally, the Connection constructor should raise an exception that indicates 
that the supplied 
credentials were rejected by the server. I THINK, reading the AMQP 0.8 spec, 
that the server 
should be sending back another Secure method frame or an exception frame, but I 
don't have 
protocol diagnostics in front of me...

I am considering donating to py-amqplib, since my project in the future might 
depend on it. This way i would like to encourage to at least to continue 
support this package.

Any ideas how to do that?

Amqplib 1.0.2 with timeout patch. Python 3.2. RabbitMQ 2.7.1.

I have two workers on the same server processing incoming messages and replying 
RPC-style (i.e. preserving correlation_id). Also, 
chan.basic_qos(prefetch_size=0,prefetch_count=1,a_global=False) and 
chan.basic_consume(queue=q, no_ack=False, callback=data_received)

Each work item takes 200-2000ms depending on content. There is plenty of time 
between work items, they arrive 1-20 per second. The two workers are mainly for 
redundancy (being able to upgrade and restart without downtime).

I have logging, with timestamps, in my own code both on submitting work to 
RabbitMQ and on processing it.

Occasionally, I will see work packets being delayed maybe 25 seconds from being 
given to RabbitMQ before arriving at the client. This happens randomly and 
about once every 5K-10K work items. After 20 seconds the client times out so 
while the work is eventually done, no one is interested in the result.

RabbitMQ and the workers are separated by one network switch, the servers are 
just next to eachother. The Rabbit queue setup is very simple with just a 
default/direct exchange.

I am unsure where the problem lies, but my own logging indicates it is either 
RabbitMQ or amqplib. I would be happy to log more, share code, share data or 
otherwise do work to follow up on people's hunches.

If anyone is interested, the site in question is boardword.com

Using v1.0.0 of amqplib, Python 2.7.1, Mac OS X 10.7. 

If network connectivity is lost while a Connection is open, and the Connection 
is then closed, its _do_close() method first attempts to close its transport. 
When that fails, an exception is raised, and the rest of the cleanup code in 
_do_close() is not run.  This leaves Channel objects with open callback 
references.  Since the object that created the Connection and Channel is likely 
to provide the callback for that Channel, a circular reference results and the 
objects are never deleted.

I've worked around this problem by wrapping the self.transport.close() call in 
Connection's _do_close in a try: finally block, so the cleanup code is run 
whether the transport.close() works or not. If this is an inappropriate 
solution, or I'm missing something, I'd like to know. Thanks!

I'm enclosing a patch both in plain and in hg bundle format, which adds:

1. UnitTests for sequence serialization/deserialization
2. Sequence serialization/deserialization support

While under 1.0 amqplib raises a sensible exception while trying to
publish a message whose application headers contain a sequence:

>>> from  amqplib import client_0_8 as amqp
>>> cn=amqp.Connection(userid='test', password='*********', 
virtual_host='*******')
>>> ch=cn.channel()
>>> msg = amqp.Message('hello world', application_headers = 
{'hdr1':['val1','val2']})
>>> ch.basic_publish(msg,routing_key='tests', exchange='hdrXc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "amqplib/client_0_8/channel.py", line 2310, in basic_publish
    self._send_method((60, 40), args, msg)
  File "amqplib/client_0_8/abstract_channel.py", line 76, in _send_method
    method_sig, args, content)
  File "amqplib/client_0_8/method_framing.py", line 250, in write_method
    properties = content._serialize_properties()
  File "amqplib/client_0_8/serialization.py", line 531, in _serialize_properties
    getattr(raw_bytes, 'write_' + proptype)(val)
  File "amqplib/client_0_8/serialization.py", line 397, in write_table
    raise ValueError('%s not serializable in AMQP' % repr(v))
ValueError: ['val1', 'val2'] not serializable in AMQP


in the 0.6.1 case the ch.basic_publish seems to return, but  on subsequent 
publish() tries I get a socket error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pymodules/python2.6/amqplib/client_0_8/channel.py", line 2223, in basic_publish
    self._send_method((60, 40), args, msg)
  File "/usr/lib/pymodules/python2.6/amqplib/client_0_8/abstract_channel.py", line 70, in _send_method
    method_sig, args, content)
  File "/usr/lib/pymodules/python2.6/amqplib/client_0_8/method_framing.py", line 240, in write_method
    self.dest.write_frame(2, channel, payload)
  File "/usr/lib/pymodules/python2.6/amqplib/client_0_8/transport.py", line 125, in write_frame
    frame_type, channel, size, payload, 0xce))
  File "<string>", line 1, in sendall
socket.error: [Errno 32] Broken pipe

Version: 1.0.2

Running "python3.2 tests/client_0_8/run_all.py" produce a syntax error because 
u"str" is not supported by python 3.

I attached a small patch that fix this issue.

Which version of amqplib are you using?

    Version 0.6.2-devel

Which broker are you using (RabbitMQ?) which version?

rabbitmq 2.5.0

Which version of Python?

Python 2.6.1

What steps will reproduce the problem?

1. Receive a message from the queue.
2. Compare the message against something without the 'properties' attribute.
3. Watch an exception be raised.

What is the expected output? What do you see instead?

A comparison is a simple operation that should not cause exceptions. Only true 
or false.

Please provide any additional information below.

I've attached a patch to fix the issue.

Which version of amqplib are you using?
0.6.1-1
Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes
Which broker are you using (RabbitMQ?) which version?
2.6.1
Which version of Python?
2.7
What steps will reproduce the problem?
It seems some race condition, sometimes I can see the 2 exception in my log 
file.

What is the expected output? What do you see instead?
The expected behavior is to publish a message but the publishing process dies 
after the exception

Please provide any additional information below.
I can see those 2 exception in the log file of my application using py-amqplib 
through kombu library.
The strange thins is that the first exception is:
<class 'amqplib.client_0_8.exceptions.AMQPChannelException'>, 
AMQPChannelException(404, u"NOT_FOUND - no exchange 
'55f1152e2ca14745a9f85393bc24f5b4' in vhost '/nova'", (60, 40), 
'Channel.basic_publish')

and then there is the other exception:
<class 'amqplib.client_0_8.exceptions.AMQPConnectionException'>, 
AMQPConnectionException(503, u"COMMAND_INVALID - second 'channel.open' seen", 
(20, 10), 'Channel.open'

but if I check in the rabbitmq I can see the exchange.

Unfortunately this problem occurs intermittent and it's difficult to debug, 
maybe the 1.0 version can have a fix to that problem?
Thanks

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.2

Which version of Python?
2.5.2

What steps will reproduce the problem?
Publish a message with an application_headers value that is not 
serializable by the Message class (e.g. None, object()):
chan.basic_publish(Message(application_headers=dict(test=None)), '')

What is the expected output? What do you see instead?
There should be an error raised.
The message is not sent and connection is just silently being dropped. An 
error is also being displayed in the logs of RabbitMQ indicating that the 
message can not be parsed. I think RabbitMQ can also solve this problem by 
throwing an error to the client that the message is malformed, but the 
client (amqplib) should not be sending malformed message in the first 
place. To fix this, if the type of the value is unsupported, an error 
should be raised.

Versions: py-amqplib 0.6.1, RabbitMQ 1.5.4, Python 2.6.4

I would like to perform the following:

  chan.basic_consume(..)
  heartbeat_deadline = new_deadline(...) # Compute a time instant
  while 1:
    next_deadline = max(0, heartbeat_deadline - time.time())
    chan.wait(timeout = next_deadline)
    # We get there if a message was processed,
    # or if nothing was received for the specified amount of time.
    # Check if we reached our deadline:
    if time.time() > heartbeat_deadline:
       chan.basic_publish(...)
       heartbeat_deadline = new_deadline(...)

The purpose is to allow me to run a loop to receive new messages from
a queue synchronously, but that still allow me to send message back to
the server at regular interval (like heartbeat or something like that.)

Could py-ampqlib support that easily? I've not checked sources in detail,
but I think _AbstractTransport._read(self, n) could support an additionnal
parameter 'timeout', which would be used only for receiving the header
of a new frame (other uses would set timeout to None) if .wait() specified
a timeout itself, using select() before the read() operation.

But maybe I misusing the library. Is that another way to obtain the same
behavior? I thought about writing a second process (and thus opening
an additionnal TCP connection) but I'm not really happy with this idea.
I would prefer to perform all the stuff from a single process.

FreeBSD 6.2, Python 2.7.2, connection fails with

   File "/usr/local/lib/python2.7/site-packages/amqplib/client_0_8/connection.py", line 129, in __init__
     self.transport = create_transport(host, connect_timeout, ssl)
   File "/usr/local/lib/python2.7/site-packages/amqplib/client_0_8/transport.py", line 279, in create_transport
     return SSLTransport(host, connect_timeout, ssl)
   File "/usr/local/lib/python2.7/site-packages/amqplib/client_0_8/transport.py", line 179, in __init__
     super(SSLTransport, self).__init__(host, connect_timeout)
   File "/usr/local/lib/python2.7/site-packages/amqplib/client_0_8/transport.py", line 71, in __init__
     for res in socket.getaddrinfo(host, port, 0, 0, socket.SOL_TCP):
 socket.gaierror: [Errno 9] servname not supported for ai_socktype

Seems to be a FreeBSD bug, but can be worked around with this one-line patch

--- transport.py.original       Tue Sep 27 15:45:18 2011
+++ transport.py        Tue Sep 27 15:39:31 2011
@@ -68,7 +68,7 @@
                 port = int(port)

         self.sock = None
-        for res in socket.getaddrinfo(host, port, 0, 0, socket.SOL_TCP):
+        for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM, 
socket.SOL_TCP):
             af, socktype, proto, canonname, sa = res
             try:
                 self.sock = socket.socket(af, socktype, proto)


Probably no reason to be looking for anything other than SOCK_STREAM, so 
perhaps it should just be in there by default.

Which version of amqplib are you using?

amqplib-0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

yes

Which broker are you using (RabbitMQ?) which version?

RabbitMQ 2.4.1

Which version of Python?

python2.6

What steps will reproduce the problem?

1.download tarball 
2.decompress tarball and change directory
3.run 'sudo python2.6 setup.p[y install'

What is the expected output? What do you see instead?

I expected the software to build, but instead I got the following error message:

  running install
  error: invalid Python installation: unable to open /usr/lib64/python2.6/config/Makefile (No such file or directory)


Please provide any additional information below.

I checked and sure enough there is no config directory.  I tried creating the 
config directory, but a Makefile is still needed from somewhere.  Any pointers 
on where I can get the correct Makefile?

Which version of amqplib are you using?
amqplib-0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes

Which broker are you using (RabbitMQ?) which version?
v2.2.0

Which version of Python?
2.6.2

What steps will reproduce the problem?
1. subscribe by putting basic_consume into a thread
2. kill the thread
3. close the channel (doesn't return)
4. hang

What is the expected output? What do you see instead?

 If you control C out, you see the error:
 frame_type, channel, size = unpack('>BHI', self._read(7))
  File "/opt/linux/lib/python2.6/site-packages/amqplib/client_0_8/transport.py", line 200, in _read
    s = self.sock.recv(65536)

The code hangs in the channel close.

Please provide any additional information below.

If you close the connection instead of the channel,
you see the message "socket closed".  So closing the connection
works, but closing the channel doesn't.
If I want to create and destroy channels on-the-fly, this won't work.

Is this a python socket problem or an ampqlib problem?

Which version of amqplib are you using?

amqplib-1.0.1.tgz

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

Yes

Which broker are you using (RabbitMQ?) which version?

N/A - problem with installation

Which version of Python?

2.7.2

What steps will reproduce the problem?

1. pip install amqplib

What is the expected output?

Package installs correctly

What do you see instead?

reading manifest template 'MANIFEST.in'
Traceback (most recent call last):
  File "<string>", line 14, in <module>
  File "C:\Users\sculd\build\amqplib\setup.py", line 47, in <module>
    cmdclass = {'build_py':build_py},
  File "C:\Python27\lib\distutils\core.py", line 152, in setup
    dist.run_commands()
  File "C:\Python27\lib\distutils\dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
    cmd_obj.run()
  File "<string>", line 12, in replacement_run
  File "C:\Python27\lib\site-packages\setuptools\command\egg_info.py", line 252, in find_sources
    mm.run()
  File "C:\Python27\lib\site-packages\setuptools\command\egg_info.py", line 308, in run
    self.read_template()
  File "C:\Python27\lib\site-packages\setuptools\command\sdist.py", line 162, in read_template
    sys.exc_info()[2].tb_next.tb_frame.f_locals['template'].close()
  File "C:\Python27\lib\distutils\text_file.py", line 128, in close
    self.file.close ()
AttributeError: 'NoneType' object has no attribute 'close'

Please provide any additional information below.

I believe that this issue is due to a corrupted MANIFEST.in file.
I have corrected this file to the following and it installed successfully.

MANIFEST.in:

include README TODO CHANGES INSTALL LICENSE
recursive-include docs/*.txt
recursive-include demo/*.py
recursive-include extras/*.py
recursive-include tests/*.py

Current version of pyamqplib does not support Queue.Unbind and Exchange.Unbind 
function.

These are supported by RabbitMQ, but aren't available in the client.  Could you 
add them?

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
yes

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.1

Which version of Python?
2.6.2

What steps will reproduce the problem?
1. Write messages > frame-max from server

What is the expected output? What do you see instead?
Connection is forcibly closed

Please provide any additional information below.
In comparison with pika, it seems amqplib is missing code to handle sending 
fragments in chunks of AMQ frame-max (minus header/footer). 

The same may also apply to the read_frame function.

f

> Which version of amqplib are you using?
0.6

> Which broker are you using (RabbitMQ?) which version?
RabbitMQ

> Which version of Python?
2.5.4

> What steps will reproduce the problem?

1. Make a connection to the broker
2. Request a channel
3. Do something e.g. publish message
4. close channel
5. close connection
6. Repeat #1


> What is the expected output? What do you see instead?

The process memory keeps growing.


> Please provide any additional information below.

The problem has been described by someone at LShift:

  http://www.lshift.net/blog/2008/11/14/tracing-python-memory-leaks

Doing the following fixes the leak
>>> del channel.callbacks
>>> del connection.channels
>>> del connection.connection

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes.

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.2

Which version of Python?

2.6

What steps will reproduce the problem?

$ python setup.py install --single-version-externally-managed
...
error: option --single-version-externally-managed not recognized


What is the expected output? What do you see instead?

No errors.

Please provide any additional information below.

Check for setuptools and use the setuptools version of setup to get the
enhanced functionality therein for packagers.

Super simple patch:

--- setup.py.orig   2010-02-24 12:09:19.000000000 -0800
+++ setup.py    2010-02-24 12:08:50.000000000 -0800
@@ -9,7 +9,10 @@
 """

 import sys
-from distutils.core import setup
+try:
+    from setuptools import setup
+except:
+    from distutils.core import setup

 setup(name = "amqplib",
       description = "AMQP Client Library",
[

--Robert

In abstract channel, lines 94 - 97 are hiding bugs. I'm tracking down a
case where the channel's connection is None, which is an internal bug
(either in the library code or mine) but it's being ignored by that try
catch block.

With the current code, if the programs exits soon after calling the close 
method on the transport, pending transmissions may be discarded. Adding the 
shutdown call before closing the socket asks the TCP stack to send all pending 
packets, and which fixes this problem.

Which version of amqplib are you using? 0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project? Yes

Which broker are you using (RabbitMQ?) which version? RabbitMQ 2.4.1

Which version of Python? 2.6.5

What steps will reproduce the problem?
1. Create a connection.
2. Create a channel.
3. Declare an exchange.
4. Publish a message to the exchange with either immediate or mandatory flags 
set to True.
5. Close the channel.

What is the expected output? What do you see instead?
Expect to have a way to determine if the message could not be published. In my 
testing, I found that the AMQP server sent back a basic_return message, however 
it was never processed by amqplib and left in the Channel.method_queue.

Please provide any additional information below.

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

Which broker are you using (RabbitMQ?) which version?
+---+   +---+
|   |   |   |
|   |   |   |
|   |   |   |
|   +---+   +-------+
|                   |
| RabbitMQ  +---+   |
|           |   |   |
|   v2.2.0  +---+   |
|                   |
+-------------------+


Which version of Python?
2.6.2

What steps will reproduce the problem?
two processes in two windows, one produces, the other consumes 
and may be varied as to how it consumes - whether a basic_get
or a basic_consume inside a thread

1. channel.basic_get() tests ok in the consumer
2. then commanded to threaded subscribe - basic_consume() followed by 
channel.wait()
3. then commanded to Unsubscribe via a thread.kill()
4. then commanded again to channel.basic_get() which now fails, the 
exception is that the queue is no longer recognized. 
5. However, if re-subscribed, the consumer can again consume

What is the expected output? What do you see instead?

The thread kill() on basic_consume() should not prevent basic_get() from 
working.

Please provide any additional information below.

Note: this is an enhancement request, not a bug per se. 


Which version of amqplib are you using?

0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

Yes.

Which broker are you using (RabbitMQ?) which version?

RabbitMQ 1.5.4-0ubuntu1

Which version of Python?

2.6.2

What steps will reproduce the problem?
1. Call basic_consume without a callback.
2. Watch in wild wonder as nothing happens. 

What is the expected output? What do you see instead?

Well, the actual behavior of basic_consume with callback=None matches the 
documented behavior. I just don't really see how that functionality is useful. 
Is there 
some use case in which you would want to call basic_consume and do nothing with 
the messages there?

Maybe my thinking is poisoned by the positions I tend to find myself in with 
amqp. 
This is the second project in as many months where I *really* want to act upon 
the 
messages within the same function or method basic_consume() is called in. 

It seems like it might be reasonable to either make basic_consume a generator 
(so I 
could say 'for msg in basic_consume(blah)', for example), or to just return the 
Message object upon arrival at the consumer's doorstep if callback=None. 

Here's a scenario that shows the type of issue I'm running into: 

I have a main module that launches and spawns a few threads. Each thread talks 
to a 
different queue, and wants to process messages from it. I've decided to 
abstract out 
the low-level queuing details to clean up the core code, so the core code just 
creates 
a QueueConnector object, and call that object's 'consume()' method. My call to 
basic_consume() lives inside of that consume() method. 

Now, it would seem reasonable that I might be able to pass my processing 
function 
as a callback, and I can, but the problem then is that *my core code still 
needs a 
copy of the Message object*, because it's going to be *requeued* to another 
queue 
(but only if the processing was successful!). So, in pseudocode (and if 
basic_consume 
were a generator): 

def consume(blah,blah):
    for msg in chan.basic_consume('q'):
        try:
            process(msg.body)
        except Exception as out:
            print out
            raise
        else:
            msg.ack()
            return/yield msg

Another alternative is to tell basic_deliver to 'return msg' if func is None 
(on line 
1973: http://code.google.com/p/py-
amqplib/source/browse/amqplib/client_0_8/channel.py#1971). It would be returned 
to basic_consume() at that point, and then basic_consume would need to pass it 
back 
to my calling code. I understand the consumer_tag is already being returned, 
but I 
wonder if there isn't some other logical place to get the consumer tag? 

I suppose there's also the issue of chan.wait(), but I'm not sure if there's 
another use 
case outside of basic_consume where that would ever be called. Is there? If 
there 
isn't, why not just make basic_consume a blocking (or optionally blocking) 
procedure? If there *is* another use case for using chan.wait() outside of 
basic_consume, perhaps understanding that better would help me solve my 
problems. As it stands, using chan.wait with basic_consume is pretty confusing 
when 
it comes time to figure out how this program is going to finally end.

Thanks for input or advice (or an enhancement!)
brian

Which version of amqplib are you using? 0.8

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project? No.

Which broker are you using (RabbitMQ?) which version? 1.7.0

Which version of Python? 3.1.1

What steps will reproduce the problem?
1. run setyp.py install
2. flags the files method_framing.py and serialization.py
3.

What is the expected output? What do you see instead?
I expected to see the build complete, which it did after I modified the two 
source files.

Please provide any additional information below.

changed Exception, e to read (Exception, e) in method_framing.py
changed print hex to print (hex) in serialization.py

Which version of amqplib are you using?
0.6.1
Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

Yes, but code inspection leads me to believe this is still an issue

Which broker are you using (RabbitMQ?) which version?
yes (rabbit), not sure of version
Which version of Python?
2.7
What steps will reproduce the problem?
1. create a channel
2. start consuming
3. kill rabbit
4. have a basic_cancel in a try:finally: around your consumer loop

What is the expected output? What do you see instead?

I'd expect the basic_cancel to just do nothing on a closed connection (or 
possibly raise an explicit exception).

What we see is captured here:
https://bugs.launchpad.net/python-oops-amqp/+bug/884539
"Traceback (most recent call last):
  File "bin/amqp2disk", line 49, in <module>
    oopstools.scripts.amqp2disk.main()
  File "/srv/lp-oops.canonical.com/cgi-bin/lpoops/src/oopstools/scripts/amqp2disk.py", line 113, in main
    receiver.run_forever()
  File "/srv/lp-oops.canonical.com/cgi-bin/lpoops/eggs/oops_amqp-0.0.3-py2.6.egg/oops_amqp/receiver.py", line 86, in run_forever
    self._run_forever()
  File "/srv/lp-oops.canonical.com/cgi-bin/lpoops/eggs/oops_amqp-0.0.3-py2.6.egg/oops_amqp/receiver.py", line 112, in _run_forever
    self.channel.basic_cancel(self.consume_tag)
  File "/srv/lp-oops.canonical.com/cgi-bin/lpoops/eggs/amqplib-0.6.1-py2.6.egg/amqplib/client_0_8/channel.py", line 1702, in basic_cancel
    self._send_method((60, 30), args)
  File "/srv/lp-oops.canonical.com/cgi-bin/lpoops/eggs/amqplib-0.6.1-py2.6.egg/amqplib/client_0_8/abstract_channel.py", line 69, in _send_method
    self.connection.method_writer.write_method(self.channel_id,
AttributeError: 'NoneType' object has no attribute 'method_writer'
"

Please provide any additional information below.
We've put a if self.channel.is_open: guard around the call to basic_cancel, but 
it would be nice to have fire-and-forget here, as it seems like an obvious 
idiom to be cancelling consumption in a try:finally:

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
Yes

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.2

Which version of Python?
2.6.4

What steps will reproduce the problem?
1. Publish a message (basic_publish)
2. Get the message (basic_get) with no_ack=False
3. Get the message again (basic_get)

What is the expected output? What do you see instead?
Both basic_get should return the message, but only the first succeeds even 
though the message was not acknowledged.

Please see the attached publisher and consumer which reproduce the issue

I hope I'm not missing something obvious.

Cheers,

Which version of amqplib are you using?

0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

Yes

Which broker are you using (RabbitMQ?) which version?

RabbitMQ 1.6

Which version of Python?

3.1.2

What steps will reproduce the problem?
1. python3 setup.py install
2.
3.

What is the expected output? What do you see instead?

expected output
===========
python3.1 setup.py install
running install
running build
running build_py
copying amqplib/client_0_8/__init__.py -> build/lib/amqplib/client_0_8
copying amqplib/client_0_8/abstract_channel.py -> build/lib/amqplib/client_0_8
copying amqplib/client_0_8/basic_message.py -> build/lib/amqplib/client_0_8
copying amqplib/client_0_8/channel.py -> build/lib/amqplib/client_0_8
copying amqplib/client_0_8/connection.py -> build/lib/amqplib/client_0_8
copying amqplib/client_0_8/method_framing.py -> build/lib/amqplib/client_0_8
copying amqplib/client_0_8/serialization.py -> build/lib/amqplib/client_0_8
running install_lib
copying build/lib/amqplib/client_0_8/__init__.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
copying build/lib/amqplib/client_0_8/abstract_channel.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
copying build/lib/amqplib/client_0_8/basic_message.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
copying build/lib/amqplib/client_0_8/channel.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
copying build/lib/amqplib/client_0_8/connection.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
copying build/lib/amqplib/client_0_8/method_framing.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
copying build/lib/amqplib/client_0_8/serialization.py -> 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/__init__.py to __init__.pyc
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/abstract_channel.py to abstract_channel.pyc
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/basic_message.py to basic_message.pyc
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/channel.py to channel.pyc
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/connection.py to connection.pyc
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/method_framing.py to method_framing.pyc
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/serialization.py to serialization.pyc
running install_egg_info
Removing 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib-0.6.2_devel-py3.1.egg-info
Writing 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib-0.6.2_devel-py3.1.egg-info


Actual Output
=========
python3.1 setup.py install
running install
running build
running build_py
running install_lib
byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/method_framing.py to method_framing.pyc
  File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/amqplib/client_0_8/method_framing.py", line 128
    except Exception, e:
                    ^
SyntaxError: invalid syntax

byte-compiling 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib/client_0_8/serialization.py to serialization.pyc
  File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/amqplib/client_0_8/serialization.py", line 55
    print hex + ''.join(char_dump)
            ^
SyntaxError: invalid syntax

running install_egg_info
Removing 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib-0.6.2_devel-py3.1.egg-info
Writing 
/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/am
qplib-0.6.2_devel-py3.1.egg-info



Please provide any additional information below.

Running the following two commands over the source seems to allow it to install

2to3-3.1 amqplib/ -w
2to3-3.1 setup.py -w


How can I help add python3 support?

Which version of amqplib are you using?
0.6.1  and 0.6.2devel with tonyg's patch for enabling channel.flow control.


Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?

Which broker are you using (RabbitMQ?) which version?

1.7.2
Which version of Python?

2.5 and 2.6


I'm just reporting that tonyg's  http://gist.github.com/399282   patch to add 
channel flow
control works.  I've applied it and tested it in the 0.6.2devel branch.

The only caveat is that after a basic_publish you need to issue something that 
requires
a response from the server.  In my test issuing channel.access_request  worked 
to enable
my client to detect that flow control was enabled/on.

Which version of amqplib are you using?
0.6.1

Have you checked to see if there is a newer version in the "Featured
Downloads" section of the front page of this project?
yes

Which broker are you using (RabbitMQ?) which version?
RabbitMQ 1.7.1

Which version of Python?
2.6.2

What steps will reproduce the problem?
1. Open connection
2. Attempt to write large message (e.g. 22MB)

What is the expected output? What do you see instead?
Instead of sending the message, the connection is forcibly closed.

Please provide any additional information below.
http://hg.barryp.org/py-amqplib-
devel/file/9f9c3e41559c/amqplib/client_0_8/connection.py#l115 (and earlier 
versions) have a hardcoded maximum framesize of 131072 bytes. 

Suggest using kwargs on Connection() method to override frame_max as 
required.

I am using v1.0.2 (most recent as of now) on Python 3.2 with the most recent 
RabbitMQ. When there is a network issue it sometimes crashes on connect 
(amqp.Connection()) with this trace:

\connection.py", line 129, in __init__
    self.transport = create_transport(host, connect_timeout, ssl)
\transport.py", line 297, in create_transport
    return TCPTransport(host, connect_timeout)
\transport.py", line 87, in __init__
    raise socket.error(msg)
UnboundLocalError: local variable 'msg' referenced before assignment

I replaced the offending line
raise socket.error(msg)
with:
raise socket.error('Unable to connect')