cloudamqp / amqproxy Goto Github PK
View Code? Open in Web Editor NEWAn intelligent AMQP proxy, with connection and channel pooling/reusing
Home Page: https://www.cloudamqp.com
License: MIT License
An intelligent AMQP proxy, with connection and channel pooling/reusing
Home Page: https://www.cloudamqp.com
License: MIT License
Hello, I am evaluating this on my staging environment and after a few hours running some of our PHP workers started throwing exceptions when consuming from the Proxy.
I would like to know if there is any restriction over connections (like connections can only be done via sockets), and what could be the cause of the following exception:
Our pods didn't have any restart so I think AMQP Proxy shutdown by itself somehow.
If I want to use it in a Containerized environment like Kubernetes or DC/OS , what is the best practice :
Thanks
docker build -t amqproxy:v0.4.4 https://github.com/cloudamqp/amqproxy.git#v0.4.4
Sending build context to Docker daemon 185.3kB
Step 1/8 : FROM crystallang/crystal
latest: Pulling from crystallang/crystal
5bed26d33875: Pull complete
f11b29a9c730: Pull complete
930bda195c84: Pull complete
78bf9a5ad49e: Pull complete
e27d61588c18: Pull complete
Digest: sha256:b3f86e938a3de7454d466736c013cf25ed37e378f64bb0583555627b6d2ea90f
Status: Downloaded newer image for crystallang/crystal:latest
---> 89062caa0bb5
Step 2/8 : WORKDIR /app
---> Running in b3153b37c2be
Removing intermediate container b3153b37c2be
---> f6afa181fbf2
Step 3/8 : COPY . .
---> 4404ad8c69d8
Step 4/8 : RUN shards build --release --production
---> Running in ba4ee679d3f9
Resolving dependencies
Fetching https://github.com/cloudamqp/amq-protocol.cr.git
Installing amq-protocol (0.3.4)
Building: amqproxy
Error target amqproxy failed to compile:
Showing last frame. Use --error-trace for full trace.
In src/amqproxy/upstream.cr:118:17
118 | rescue ex : Errno | IO::Error | OpenSSL::SSL::Error
^----
Error: Errno is not a subclass of Exception
The command '/bin/sh -c shards build --release --production' returned a non-zero code: 1
I have tried multiple releases, master - same error.
Server is closing channel when confirm fails from exchange not existing when publishing. Think the channel should be cleaned up when receiving confirm errors.
To get around this is there some way to force close the channel in the proxy?
Crystal 0.27.0 [c9d1eef8f] (2018-11-01)
LLVM: 4.0.0
Default target: x86_64-unknown-linux-gnu
Please find the output of below
crystal build --verbose --error-trace --release -o bin/amqproxy src/amqproxy.cr
server.listen(listen_address, listen_port)
�[32;1m^~~~~~�[0m
in src/amqproxy/server.cr:31: instantiating 'TCPServer.class#open(String, Int32)'
TCPServer.open(address, port) do |socket|
�[32;1m^~~~�[0m
in src/amqproxy/server.cr:31: instantiating 'TCPServer.class#open(String, Int32)'
TCPServer.open(address, port) do |socket|
�[32;1m^~~~�[0m
in src/amqproxy/server.cr:41: expanding macro
spawn handle_connection(client, client.remote_address)
�[32;1m^�[0m
in macro 'spawn' /usr/share/crystal/src/concurrent.cr:97, line 11:
1.
2. ->(
3.
4. __arg0 : typeof(client),
5.
6. __arg1 : typeof(client.remote_address),
7.
8.
9. ) {
10. spawn(name: nil) do
�[32;1m>�[0m�[1m 11. handle_connection(�[0m
12.
13. __arg0,
14.
15. __arg1,
16.
17.
18. )
19. end
20.
21. }.call(client, client.remote_address)
22.
23.
instantiating 'handle_connection(TCPSocket+, Socket::IPAddress)'
in src/amqproxy/server.cr:84: instantiating 'AMQProxy::Client.class#new(TCPSocket+)'
c = Client.new(socket)
�[32;1m^~~�[0m
in src/amqproxy/client.cr:11: instantiating 'negotiate_client(IO+)'
@vhost, @user, @password = negotiate_client(@socket)
�[32;1m^~~~~~~~~~~~~~~~�[0m
in src/amqproxy/client.cr:53: instantiating 'AMQProxy::AMQP::Frame.class#decode(IO+)'
start_ok = AMQP::Frame.decode(socket).as(AMQP::Connection::StartOk)
�[32;1m^~~~~~�[0m
in src/amqproxy/amqp/frames.cr:38: undefined method 'at' for Slice(UInt8)
frame_end = payload.at(size)
�[32;1m^~�[0m
================================================================================
Slice(UInt8) trace:
src/amqproxy/amqp/frames.cr:35
payload = Bytes.new(size + 1)
�[32;1m^~~~~~~�[0m
src/amqproxy/amqp/frames.cr:35
payload = Bytes.new(size + 1)
�[32;1m^~~�[0m
/usr/share/crystal/src/slice.cr:83
def self.new(size : Int, *, read_only = false)
/usr/share/crystal/src/slice.cr:83
def self.new(size : Int, *, read_only = false)
�[32;1m^�[0m
/usr/share/crystal/src/slice.cr:84
{% unless T <= Int::Primitive || T <= Float::Primitive %}
�[32;1m^�[0m
/usr/share/crystal/src/slice.cr:89
new(pointer, size, read_only: read_only)
�[32;1m^~~�[0m
/usr/share/crystal/src/slice.cr:67
def initialize(@pointer : Pointer(T), size : Int, *, @read_only = false)
/usr/share/crystal/src/slice.cr:67
def initialize(@pointer : Pointer(T), size : Int, *, @read_only = false)
�[32;1m^�[0m
/usr/share/crystal/src/slice.cr:67
def initialize(@pointer : Pointer(T), size : Int, *, @read_only = false)
vagrant up debian9
Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.Al6wzHKRRU/gpg.1.sh --keyserver keys.gnupg.net --recv-keys 09617FD37CC06B54
gpg: keyserver receive failed: No name
GPG error: http://dist.crystal-lang.org/apt crystal InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 09617FD37CC06B54
The repository 'http://dist.crystal-lang.org/apt crystal InRelease' is not signed.
Is this still supported?
Should the vagrant file follow the installation instruction on crystal-lang.org? https://crystal-lang.org/install/on_debian/#official-crystal-deb-repository
Hello
We got a error with upstream domain name in GCP.
installed on the server:
Consul client with DNS on the localhost: 8600
dnsmask on localhost:53, which redirects all requests of the consul domain to localhost:8600
configuration /etc/resolve.conf:
cat /etc/resolv.conf
domain us-central1-a.c.production-***.internal
search us-central1-a.c.production-***.internal. c.production-***.internal. google.internal. consul. node.consul.
nameserver 127.0.0.1
nameserver 169.254.169.254
in consul we have service name witch resolved each node in rabbit cluster:
host rabbitmq-analytics.service.consul
rabbitmq-analytics.service.consul has address 10.0.15.223
rabbitmq-analytics.service.consul has address 10.0.15.224
rabbitmq-analytics.service.consul has address 10.0.15.249
Installed from relaeses amqproxy
amqproxy -v
0.3.6
then we run amqproxy and got errors:
/usr/bin/amqproxy -d -l 127.0.0.1 -p 5673 amqp://rabbitmq-analytics.service.consul
Proxy upstream: rabbitmq-analytics.service.consul:5672
Proxy listening on 127.0.0.1:5673
Client connection accepted from 127.0.0.1:57532
Upstream connection could not be established
Client connection closed from 127.0.0.1:57532
Unhandled exception in spawn: No address found for rabbitmq-analytics.service.consul:5672 over TCP (Socket::Error)
from ???
from ???
from ???
from ???
from ???
Client connection accepted from 127.0.0.1:57534
Upstream connection could not be established
Client connection closed from 127.0.0.1:57534
Unhandled exception in spawn: No address found for rabbitmq-analytics.service.consul:5672 over TCP (Socket::Error)
from ???
from ???
from ???
from ???
from ???
Client connection accepted from 127.0.0.1:57538
Upstream connection could not be established
Client connection closed from 127.0.0.1:57538
Unhandled exception in spawn: No address found for rabbitmq-analytics.service.consul:5672 over TCP (Socket::Error)
from ???
from ???
from ???
from ???
from ???
Client connection accepted from 127.0.0.1:57540
Client connection accepted from 127.0.0.1:57542
Upstream connection could not be established
Client connection closed from 127.0.0.1:57542
Unhandled exception in spawn: No address found for rabbitmq-analytics.service.consul:5672 over TCP (Socket::Error)
from ???
from ???
from ???
from ???
from ???
We try build from source amqproxy with new version Crystal and it worked fine.
Please make a release built on the new version of the crystal.
Please indicate the version of the crystal on which release is assembled
With 10000 non-persistent connections connecting and disconnecting all the time, there is too much noise in the logs. I understand this approach is to replicate RabbitMQ logs behaviour but this tool is mainly for producers/consumers that can't handle long lived connections.
Hi,
I need help for setting this up in Ubuntu. Followed installation instruction in here https://github.com/cloudamqp/amqproxy
I'm using pika python as client library.
I'm getting this error at journalctl of amqproxy.service
amqproxy[9360]: Upstream error for user '' to vhost '*': #<AMQProxy::Upstream::Error:Cannot establish connection to upstream>
I edited the upstream of amqproxy.service to point to my CloudAMQP URL by running this command
sudo systemctl edit --full amqproxy.service
then I am pointing my client to connect to localhost:5673 and providing the username, password and vhost in the upstream server.
What am I doing wrong? Thanks in advance.
Looks like I can authenticate now but connection is lost after a while, most likely during send or receive.
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 123, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from /usr/share/crystal/src/gc/boehm.cr:71:5 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 10, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from /usr/share/crystal/src/gc/boehm.cr:71:5 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 116, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from /usr/share/crystal/src/gc/boehm.cr:71:5 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 99, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from /usr/share/crystal/src/gc/boehm.cr:71:5 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 115, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from /usr/share/crystal/src/gc/boehm.cr:71:5 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 105, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from /usr/share/crystal/src/gc/boehm.cr:71:5 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Unhandled exception in spawn: Could not write the full body (AMQ::Protocol::Error::FrameEncode)
from lib/amq-protocol/src/amq/protocol/properties.cr:136:23 in '???'
from src/amqproxy/server.cr:112:13 in '???'
from src/amqproxy/pool.cr:17:7 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
In version 0.4.4
there is a still a memory leak similar to #25 It is leaking at ~40kbps.
"...multiple threads connecting... the number of connections increases"
https://github.com/mengz0/amqp
We recently upgraded from version 0.3.6 to 0.4.4 and started receiving the following error in production:
2 clients 2 upstreams
Upstream connection could not be established
Unhandled exception in spawn: getaddrinfo: Temporary failure in name resolution (Socket::Addrinfo::Error)
from usr/share/crystal/src/gc/boehm.cr:110:5 in 'handle_connection'
from vagrant/src/amqproxy/server.cr:38:11 in '->'
from usr/share/crystal/src/fiber.cr:255:3 in 'run'
from ???
We have several proxies in a k8s deployment which directs traffic to CloudAMQP. The pod is a ubuntu:18.04
image if that helps. Let me know if you need any more details.
What I've observed is when our BernardPHP / PhpAmqplib consumer is targeting the Amqproxy, it will hang when no messages remain in the queue. Essentially, it's not able to poll for messages and will hang at a variable point in time when the queue depth is zero.
I put together a set of php files that reproduces the issue.
bernard_test.tar.gz
Hi, Install Crystal link in readme was broken. It had changed from https://crystal-lang.org/reference/installation/ to https://crystal-lang.org/install/
In our QA environment, over the course of two hours the proxy process rose to over 2GB of memory consumed before crashing.
0.4.0 does not seem to exhibit the issue. 0.4.1 does not work with my application, so I'm not able to confirm that the leak was introduced there or not.
Steps to reproduce:
while : ; do php producer.php proxy ; done
I performed the above steps simultaneously to both 0.4.0 and 0.4.2 for 30 minutes or so, resulting in the below screenshot of top, note the VIRT column.
This screenshot is a consumer host that has been running the proxy for about 2 hours.
48core*200G server with zero LA :
via proxy
require_once __DIR__ . '/vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
$t1 = microtime(true);
$connection = new AMQPStreamConnection('localhost', 5673, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('hello', false, true, false, false);
$msg = new AMQPMessage('Hello World!');
$channel->basic_publish($msg, '', 'hello');
echo " [x] Sent 'Hello World!'\n";
$channel->close();
$connection->close();
$t2 = microtime(true);
echo ($t2-$t1);
[x] Sent 'Hello World!'
0.35578203201294
direct to rabbitmq:
[x] Sent 'Hello World!'
0.0085749626159668
355ms vs 8ms, what I'm doing wrong?
We get a connection error in the amqproxy after our php AMQP client does a basic_publish. The messages still goes through but the proxy connection to the upstream AMQP server opens and then closes every time instead of staying open.
Here is some sample code showing how we're publishing from our PHP:
try {
$conn = self::getConnectionForVhost($vhost);
$ch = $conn->channel();
$eventMsg = array(
'properties' => $properties
);
$amqpMessage = new AMQPMessage(
$json_encode($eventMsg),
array(
'content_type' => 'application/json',
'delivery_mode' => 2
)
);
$ch->basic_publish($amqpMessage, $exchange, $routingKey);
$ch->close();
$conn->close();
} catch (Exception $e) {
error_log('ERROR PUBLISHING EVENT ' . $e->getMessage());
return false;
}
amqproxy log
Connected to upstream 18.233.168.244:5671
Error reading from upstream: #<IO::EOFError:End of file reached>
Problem closing upstream: #<Errno:Error reading socket: Connection reset by peer>
RabbitMQ Server log (using cloud AMQP dedicated node version 3.7.10 Erlang 20.1)
[INFO] 2019-08-23 22:25:41.969 [info] <0.18763.5995> connection <0.18763.5995> (54.175.220.154:40096 -> 10.50.144.213:5671): user 'preprod' authenticated and granted access to vhost 'preprod'
[WARNING] 2019-08-23 22:26:33.565 [warning] <0.18763.5995> closing AMQP connection <0.18763.5995> (54.175.220.154:40096 -> 10.50.144.213:5671, vhost: 'preprod', user: 'preprod'): client unexpectedly closed TCP connection
EDIT: added rabbit + erlang version
Hello,
I'm getting this error when trying to connect through the proxy using Celery.
Proxy upstream: __removed__.rmq.cloudamqp.com:5672
Proxy listening on [::]:5673
Client connection error from [::ffff:172.17.0.1]:36944: #<IO::EOFError:Invalid protocol start>
Client connection closed from [::ffff:172.17.0.1]:36944
Client connection error from [::ffff:172.17.0.1]:36946: #<IO::EOFError:Invalid protocol start>
Client connection closed from [::ffff:172.17.0.1]:36946
And from celery:
[2018-07-26 17:03:43,610: DEBUG/MainProcess] | Worker: Starting Hub
[2018-07-26 17:03:43,610: DEBUG/MainProcess] ^-- substep ok
[2018-07-26 17:03:43,610: DEBUG/MainProcess] | Worker: Starting Pool
[2018-07-26 17:03:44,179: DEBUG/MainProcess] ^-- substep ok
[2018-07-26 17:03:44,180: DEBUG/MainProcess] | Worker: Starting Consumer
[2018-07-26 17:03:44,180: DEBUG/MainProcess] | Consumer: Starting Connection
[2018-07-26 17:03:44,192: ERROR/MainProcess] consumer: Cannot connect to amqp://__removed__:**@127.0.0.1:5673/__removed__: Socket closed.
Trying again in 2.00 seconds...
[2018-07-26 17:03:46,208: ERROR/MainProcess] consumer: Cannot connect to amqp://__removed__:**@127.0.0.1:5673/__removed__: Socket closed.
Trying again in 4.00 seconds...
^C
worker: Hitting Ctrl+C again will terminate all running tasks!
worker: Warm shutdown (MainProcess)
[2018-07-26 17:03:46,313: DEBUG/MainProcess] | Worker: Closing Hub...
[2018-07-26 17:03:46,313: DEBUG/MainProcess] | Worker: Closing Pool...
[2018-07-26 17:03:46,314: DEBUG/MainProcess] | Worker: Closing Consumer...
[2018-07-26 17:03:46,314: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2018-07-26 17:03:46,314: DEBUG/MainProcess] | Worker: Stopping Pool...
[2018-07-26 17:03:47,326: DEBUG/MainProcess] | Worker: Stopping Hub...
[2018-07-26 17:03:47,326: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2018-07-26 17:03:47,327: DEBUG/MainProcess] | Consumer: Shutdown Gossip...
[2018-07-26 17:03:47,327: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2018-07-26 17:03:47,327: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2018-07-26 17:03:47,327: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2018-07-26 17:03:47,327: DEBUG/MainProcess] | Consumer: Shutdown Connection...
[2018-07-26 17:03:47,329: DEBUG/MainProcess] removing tasks from inqueue until task handler finished
Connection directly to cloudamqp works fine tho:
[2018-07-26 17:05:22,180: DEBUG/MainProcess] | Worker: Starting Hub
[2018-07-26 17:05:22,180: DEBUG/MainProcess] ^-- substep ok
[2018-07-26 17:05:22,180: DEBUG/MainProcess] | Worker: Starting Pool
[2018-07-26 17:05:22,768: DEBUG/MainProcess] ^-- substep ok
[2018-07-26 17:05:22,769: DEBUG/MainProcess] | Worker: Starting Consumer
[2018-07-26 17:05:22,769: DEBUG/MainProcess] | Consumer: Starting Connection
[2018-07-26 17:05:23,076: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': '__removed__', 'copyright': 'Copyright (C) 2007-2018 Pivotal Software, Inc.', 'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP 20.1', 'product': 'RabbitMQ', 'version': '3.7.7'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
[2018-07-26 17:05:23,267: INFO/MainProcess] Connected to amqp://__removed__:**@__removed__.rmq.cloudamqp.com:5672/__removed__
I'm running the proxy on a docker container and found that if the upstream rabbitmq cluster closes the connection or is not available, the proxy is not able to reconnect. The client connection is constantly being accepted and closed while the proxy errors trying to read from the upstream.
Is there something I'm doing wrong here?
See the log below, thanks
Proxy upstream: rabbitmq:5672
Proxy listening on [::]:5673
Client connection accepted from [::ffff:172.23.0.18]:38426
Connected to upstream 172.23.0.2:5672
Client connection closed from [::ffff:172.23.0.18]:38426
Client connection accepted from [::ffff:172.23.0.18]:38434
Client connection accepted from [::ffff:172.23.0.16]:56310
Connected to upstream 172.23.0.2:5672
Client connection accepted from [::ffff:172.23.0.18]:38468
Connected to upstream 172.23.0.2:5672
Client connection accepted from [::ffff:172.23.0.16]:56338
Connected to upstream 172.23.0.2:5672
Client connection accepted from [::ffff:172.23.0.16]:56368
Connected to upstream 172.23.0.2:5672
Client connection accepted from [::ffff:172.23.0.18]:38510
Connected to upstream 172.23.0.2:5672
Unhandled exception in spawn:
cast from AMQProxy::AMQP::Connection::Close to AMQProxy::AMQP::Channel::CloseOk failed, at /var/amqpproxy/src/amqproxy/upstream.cr:101:22:101 (TypeCastError)
Client connection closed from [::ffff:172.23.0.18]:38434
from var/amqpproxy/src/amqproxy/upstream.cr:100:11 in '->'
from var/amqpproxy/src/amqproxy/pool.cr:255:3 in 'handle_connection'
from var/amqpproxy/src/amqproxy/server.cr:41:13 in '->'
from usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Error reading from upstream: #<Errno:Error reading socket: Connection reset by peer>
Unhandled exception in spawn:
cast from AMQProxy::AMQP::Connection::Close to AMQProxy::AMQP::Channel::CloseOk failed, at /var/amqpproxy/src/amqproxy/upstream.cr:101:22:101 (TypeCastError)
Client connection closed from [::ffff:172.23.0.16]:56310
from var/amqpproxy/src/amqproxy/upstream.cr:100:11 in '->'
from var/amqpproxy/src/amqproxy/pool.cr:255:3 in 'handle_connection'
from var/amqpproxy/src/amqproxy/server.cr:41:13 in '->'
from usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
Error reading from upstream: #<Errno:Error reading socket: Connection reset by peer>
Error reading from upstream: #<Errno:Error reading socket: Connection reset by peer>
Client connection closed from [::ffff:172.23.0.18]:38468
Error reading from upstream: #<Errno:Error reading socket: Connection reset by peer>
Client connection closed from [::ffff:172.23.0.16]:56338
Error reading from upstream: #<Errno:Error reading socket: Connection reset by peer>
Client connection closed from [::ffff:172.23.0.16]:56368
Error reading from upstream: #<Errno:Error reading socket: Connection reset by peer>
Client connection closed from [::ffff:172.23.0.18]:38510
Client connection accepted from [::ffff:172.23.0.18]:38520
Client connection closed from [::ffff:172.23.0.18]:38520
Client connection accepted from [::ffff:172.23.0.18]:38524
Client connection closed from [::ffff:172.23.0.18]:38524
Client connection accepted from [::ffff:172.23.0.18]:38536
Client connection closed from [::ffff:172.23.0.18]:38536
Client connection accepted from [::ffff:172.23.0.16]:56406
Client connection closed from [::ffff:172.23.0.16]:56406
Client connection accepted from [::ffff:172.23.0.18]:38546
Client connection closed from [::ffff:172.23.0.18]:38546
Client connection accepted from [::ffff:172.23.0.16]:56418
Client connection closed from [::ffff:172.23.0.16]:56418
Client connection accepted from [::ffff:172.23.0.18]:38558
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.18]:38558
Client connection accepted from [::ffff:172.23.0.18]:38564
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.18]:38564
Client connection accepted from [::ffff:172.23.0.16]:56432
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.16]:56432
Client connection accepted from [::ffff:172.23.0.18]:38574
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.18]:38574
Client connection accepted from [::ffff:172.23.0.18]:38580
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.18]:38580
Client connection accepted from [::ffff:172.23.0.18]:38584
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.18]:38584
Client connection accepted from [::ffff:172.23.0.18]:38588
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
Client connection closed from [::ffff:172.23.0.18]:38588
Client connection accepted from [::ffff:172.23.0.18]:38592
Error sending to upstream: #<Errno:Error writing to socket: Broken pipe>
I'm experimenting with amqproxy a bit (using a kubernetes cluster). I have:
3.8.13
with no tuning, just a test
user and test
vhost0.5.3
from official docker image started with --debug amqp://rabbit:5672
Then on the nodejs container I run this very simple script:
require('amqplib')
.connect('amqp://test:test@localhost:5673/test') // that's the amqproxy url
.catch(console.error);
And I can see on the rabbitmq logs that the connection was successful (so far so good). After a few seconds I stop the previous script, but as soon as I do this the connection is closed on rabbitmq side.
I expected the amqproxy to keep the connection open at least for a few seconds, but it is immediately closed. Was my assumption wrong ? It looks like #46 but if my assumption is correct I'm willing to get to the bottom of this so I started a fresh one. I will provide a tcpdump soon if that helps.
Relevant logs:
2021-03-31 11:30:17.366 [info] <0.17676.9> accepting AMQP connection <0.17676.9> (10.233.98.89:48262 -> 10.233.125.85:5672)
2021-03-31 11:30:17.373 [info] <0.17676.9> connection <0.17676.9> (10.233.98.89:48262 -> 10.233.125.85:5672): user 'test' authenticated and granted access to vhost 'test'
2021-03-31 11:30:34.875 [info] <0.17676.9> closing AMQP connection <0.17676.9> (10.233.98.89:48262 -> 10.233.125.85:5672, vhost: 'test', user: 'test')
2021-03-31 11:30:34.876 [info] <0.17691.9> Closing all channels from connection '10.233.98.89:48262 -> 10.233.125.85:5672' because it has been closed
2021-03-31 11:29:34 UTC: Proxy upstream: rabbit:5672
2021-03-31 11:29:34 UTC: Proxy listening on 0.0.0.0:5673
2021-03-31 11:30:17 UTC: Client connected: 127.0.0.1:35470
2021-03-31 11:30:34 UTC: Client disconnected: 127.0.0.1:35470: #<AMQProxy::Client::Error:Client disconnected>
2021-03-31 11:30:34 UTC: Client disconnected: 127.0.0.1:35470
I'm trying to get this working on my local development environment. I've installed amqproxy_0.2.0-1.deb and it's running as:
/usr/bin/amqproxy -p 5673 amqp://127.0.0.1:5672
I've updated my app config to connect to port 5673 instead of the usual 5672 for RabbitMQ. Everything else remains the same. Here's the relevant config values for reference:
RABBITMQ_HOST="localhost"
RABBITMQ_PORT="5673"
RABBITMQ_LOGIN="guest"
RABBITMQ_PASSWORD="guest"
RABBITMQ_VHOST="/"
Here's our connection code using php-amqplib:
$streamConnect = new AMQPStreamConnection(
getenv('RABBITMQ_HOST'),
getenv('RABBITMQ_PORT'),
getenv('RABBITMQ_LOGIN'),
getenv('RABBITMQ_PASSWORD'),
getenv('RABBITMQ_VHOST')
);
Here's the error I'm seeing in /var/log/rabbitmq/[email protected]:
=INFO REPORT==== 3-Apr-2018::15:33:11 ===
accepting AMQP connection <0.877.0> (127.0.0.1:45566 -> 127.0.0.1:5672)
=ERROR REPORT==== 3-Apr-2018::15:33:14 ===
closing AMQP connection <0.877.0> (127.0.0.1:45566 -> 127.0.0.1:5672):
{handshake_error,starting,0,
{amqp_error,access_refused,
"PLAIN login refused: user '' - invalid credentials",
'connection.start_ok'}}
It appears that the username isn't being forwarded via the proxy. Is there anything else I should be doing to debug this?
Hello,
I did some testing and it does not seem like the proxy supports TLS?
My RabbitMQ server is configured to verify peer certificates and I get the following error:
read tcp 127.0.0.1:47624->127.0.0.1:5673: read: connection reset by peer
which is expected if the peer is not presenting a certificate.
We are try to use amqproxy in our php stack to avoid performance lack and avoid of ghost connections on rabbitmq server.
But if we have the network issue and rabbitmq is not reachable for proxy then there would be ghost connections from our consumers. That is caused impossibility to send heartbeat by proxy to rabbitmq.
Enabling possibility to send heartbeat to amqp proxy will solve this issue.
Thanks
I have issue #33 reproduced in the latest version (0.4.5)
Unhandled exception in spawn: End of file reached (IO::EOFError)
from ???
from /usr/share/crystal/src/gc/boehm.cr:110:5 in 'read_loop'
from /usr/share/crystal/src/fiber.cr:255:3 in 'run'
from ???
Crystal version:
Crystal 0.34.0 [4401e90f0] (2020-04-06)
LLVM: 8.0.0
Default target: x86_64-unknown-linux-gnu
This problem is reproduced only for consumer connections. The producers work great.
If I have a PHP page:
<?php
echo 'Hello, welcome to my lovely website.';
$connection = new \AMQPConnection(...);
$channel = new \AMQPChannel($connection);
$exchange = new \AMQPExchange($channel);
$exchange->publish(...);
?>
Will the channel get reused on the second execution of this code?
Could I get the proxy to reuse the channel by setting the channel ID when I open the channel? (I don't believe the PECL version allows this but other PHP libraries do).
Since two weeks ago, we experience a problem where the connections opened by our amqproxy instance won't be closed, resulting in more and more open connections, up to the point where our CloudAMQP instance runs out of memory, effectively shutting our application down:
The sharp drops are us deploying a new amqproxy version to Kubernetes, killing all connections during deployment (we're now on 0.4.0).
At this point, I'm not sure what exactly is happening. I suspect it's an issue with one of our services, but I've traced down every single commit in the relevant time frame and I couldn't find anything related to the lower-level queue components.
Our setup looks like the following:
CloudAMQP <----> Google Kubernetes
|
amqproxy
|
+-----+-----+-----+-----+
srv1 srv2 srv3 srv4 srvN
The services all use the PHP AMQP Library with a super lightweight abstraction on top. After we've noticed the problem, we even implemented a shutdown handler to explicitly close the connection under all circumstances, but to no avail.
The connections all look like the following in the RabbitMQ manager:
Virtual host | Name | Node | State | SSL / TLS | Protocol | Channels | Channel max | Client | From Client | To Client | Heartbeat | Connected at |
---|---|---|---|---|---|---|---|---|---|---|---|---|
XXX | 35.xxx.xxx.xxx:2113 | rabbit@XXX-01 | running | ○ | AMQP 0-9-1 | 0 | 0 | AMQProxy0.4.0 | 0iB/s | 0iB/s | 22:06:372019-09-03 | |
XXX | 35.xxx.xxx.xxx:32902 | rabbit@XXX-01 | running | ○ | AMQP 0-9-1 | 0 | 0 | AMQProxy0.4.0 | 0iB/s | 0iB/s | 08:37:022019-09-04 | |
XXX | 35.xxx.xxx.xxx:33072 | rabbit@XXX-01 | running | ○ | AMQP 0-9-1 | 0 | 0 | AMQProxy0.4.0 | 0iB/s | 0iB/s | 02:28:202019-09-04 | |
XXX | 35.xxx.xxx.xxx:33196 | rabbit@XXX-02 | running | ○ | AMQP 0-9-1 | 0 | 0 | AMQProxy0.4.0 | 0iB/s | 0iB/s | 00:38:142019-09-04 | |
XXX | 35.xxx.xxx.xxx:33248 | rabbit@XXX-01 | running | ○ | AMQP 0-9-1 | 0 | 0 | AMQProxy0.4.0 | 0iB/s | 0iB/s | 03:26:462019-09-04 | |
XXX | 35.xxx.xxx.xxx:33528 | rabbit@XXX-01 | running | ○ | AMQP 0-9-1 | 0 | 0 | AMQProxy0.4.0 | 0iB/s | 0iB/s | 17:20:152019-09-03 |
The AMQProxy logs contain lots and lots of entries like the following:
Error reading from upstream: #<Errno:Error reading socket: Bad file descriptor>
Is there anything we can do to prevent this excessive connection spawning?
I faced with problem that AMQProxy not save connections between different clients request. In RabbitMq server`=INFO REPORT==== 3-Mar-2021::18:36:23 ===
accepting AMQP connection <0.1626.0> (192.168.240.2:39874 -> 192.168.240.8:5672)
=INFO REPORT==== 3-Mar-2021::18:36:23 ===
connection <0.1626.0> (192.168.240.2:39874 -> 192.168.240.8:5672): user 'guest' authenticated and granted access to vhost '/'
=INFO REPORT==== 3-Mar-2021::18:36:25 ===
closing AMQP connection <0.1626.0> (192.168.240.2:39874 -> 192.168.240.8:5672, vhost: '/', user: 'guest')` I just see that on every client request create and close new connection instead of reuse previous one. Any thought about that?
Hello,
My application is running on the php-amqplib library. I start my amqproxy instance via the command below in ubuntu terminal.
amqproxy -l 127.0.0.1 -p 5674 amqp://127.0.0.1 -d
my code in php establishes the connection like this,
$this->connection = new AMQPStreamConnection($connection, 5674, $this->user, $this->password);
I then proceed to create 2 instances of the code above on 2 separate terminal sessions.
Upon viewing the management UI connection tab, I see 2 connections. I am expecting 1 only.
Please help.
I am getting from Celery(4.2.1)/Kombu(4.2.1)/PyAMQP(2.3.2):
ConnectionError: Couldn't find appropriate auth mechanism (can offer: AMQPLAIN; available: PLAIN)
but it looks like AMQPLAIN should be supported
amqproxy/src/amqproxy/client.cr
Line 67 in be42d09
Unhandled exception in spawn: AMQ::Protocol::Frame::Body-end was 10, expected 206 (AMQ::Protocol::Error::InvalidFrameEnd)
from src/amqproxy/client.cr:42:15 in '->'
from /usr/share/crystal/src/fiber.cr:255:3 in '???'
from ???
there was already an update by @Kaggggggga #8 to close the upstream when the pipe is broken on reading. but that closing should also be placed in the write method. i worked out a pull request #16
I noticed that AMQProxy logs have \r
as newline. This is very annoying when trying to grep, count lines and whatnot on linux. It means that the newlines have to be converted to \n
first. Actually, I just checked again and I'm also seeing some \n\r
scattered around, which is really exotic. Anyone running AMQProxy on BBC Micro?
I don't know anything about Crystal and I don't wanna, but PHP has a builtin constant for a platform-specific newline: PHP_EOL
. This will be \n
on linux and \r\n
on windows. Although to be fair, does AMQProxy even target windows at all? If not, search/replace \r
with \n
in the codebase is a solution.
Hi, I'm using RabbitMQ v3.8.9 with AMQProxy v0.4.7 and it seems that AMQProxy can't handle transactions.
Here is a portion of AMQProxy's logs showing the error when one of my service tries to use a transaction:
10 clients 16 upstreams
11 clients 16 upstreams
Upstream closed connection: FRAME_ERROR - cannot decode <<>>
2020-10-16T18:27:13.270997652Z
Upstream connection closed when returned
2020-10-16T18:28:19.077851087Z
10 clients 15 upstreams
9 clients 15 upstreams
When the service is directly connected to the broker, transactions are working fine.
Is this a known limitation/bug of AMQProxy? If so, is this something that can be supported in future versions? If I can provide any additional informations, let me know.
Thank you in advance.
Just a bit of context why I use transaction in this service: I know that transactions are generally not recommended since they can significantly decrease the throughput (according to this). However, I specifically need them in this service because I'm sending a large amount of messages (few thousands) on a work queue. If I don't use a transaction, the consumers will start sending processed messages back even though the publishing service is not done dispatching the messages, thus causing weird race conditions.
I took a look at the code (note: I have zero experience of Crystal), but could not find anything related to configuring this.
I'm asking this since RabbitMQ metrics are indicated that the number of connections grows endlessly until amqproxy gets OOM killed by k8s. The metrics show thousands of connections on RabbitMQ's side but I know I don't have that many clients connecting to RabbitMQ (and each client is limited in the number of connections it can opened simultaneously).
Having such a configurable parameter would help control the proxy's pool size according to known usage pattern. For instance, if I know that I'll have at most 200 clients each with 1 active connection, I'll limit the proxy to 250 or so.
Thank you in advance.
hi, I am use
client: amqplib 0.7.1 , https://www.npmjs.com/package/amqplib
rabbitmq-server: 3.7
It's will got the fllowing error on my client
UNEXPECTED_FRAME - expected content header for class 60, got non content header frame instead
any idea, thanks.
This is my docker-compose yaml file of mq deploy.
version: "3.7"
services:
mq-n1:
image: rabbitmq:3.7-management
hostname: rabbit-c1
network_mode: host
restart: always
environment:
- RABBITMQ_ERLANG_COOKIE=rabbit-cookie
- RABBITMQ_DEFAULT_USER=private
- RABBITMQ_DEFAULT_PASS=private
mq-proxy:
image: cloudamqp/amqproxy
network_mode: host
restart: always
command: amqp://127.0.0.1:5672 --port=55672
amqproxy[1367]: 2021-03-22 15:58:49 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584E39433839BF6765A9DE, ErrorHelp[60584E39433839BF6765A9D6]", @classid=60, @MethodID=40) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:00:50 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584EB2433839BF67671EC5, ErrorHelp[60584EB2433839BF67671EBB]", @classid=60, @MethodID=40) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:00:57 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584EB93937334A54B4D7D8, ErrorHelp[60584EB93937334A54B4D7D3]", @classid=60, @MethodID=40) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:01:33 +08:00: Receiving AMQ::Protocol::Frame::Basic::QosOk(@channel=1, @bytesize=4) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:01:33 +08:00: Upstream closed connection: ChannelNotFind
amqproxy[1367]: 2021-03-22 16:01:33 +08:00: Upstream connection closed when returned
amqproxy[1367]: 2021-03-22 16:02:24 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584F10344230DF117686FC, ErrorHelp[60584F10344230DF117686F4]", @classid=60, @MethodID=40) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:03:00 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584F34383038457786EFA4, ErrorHelp[60584F34383038457786EF9B]", @classid=60, @MethodID=40) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:03:26 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584F4E433839BF6768CDB2, ErrorHelp[60584F4E433839BF6768CDAC]", @classid=60, @MethodID=40) but no client to delivery to
amqproxy[1367]: 2021-03-22 16:05:49 +08:00: Receiving AMQ::Protocol::Frame::Channel::Close(@channel=1, @bytesize=96, @reply_code=404, @reply_text="ExchangeNotExist, ReqId:60584FDD433745A76B0C6DAE, ErrorHelp[60584FDD433745A76B0C6D9F]", @classid=60, @MethodID=40) but no client to delivery to
Such As this
I'm facing a problem that amqproxy is falling when verifying the upstream certificate
#<OpenSSL::SSL::Error:SSL_connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed>
is there a way to go around this? (disabling the verification would work well in my case)
My use-case consists of openresty responding with empty_gif
and then pushing the necessary data into a queue in RabbitMQ. Using the AMQP protocol directly, RabbitMQ was able to cope with the number of messages, ingesting at a rate of about 300-400 messages per second. When I installed and configured AMQProxy, that number just skyrocketed, reaching over 600msgs/s.
At one point RabbitMQ just stopped ingesting, and the management interface in CloudAMQP became unavailable and displayed an error message (see below).
I do see the maxConnections setting in the example.ini
file, although I don't see mentions of said upper limit in the codebase, nor documentation on how to set the maximum number of connections to hold open. When I checked the management interface, there were over 6000 connections open, with many of them with a status of blocking
or blocked
. The instance is a Big Bunny.
The load test is being run with Apache AB, ab -k -c200 -n100000 -r 'http://<some address>/1234567890.gif'
. Openresty (nginx) was running with a single worker allowing a maximum of 400 worker connections (which I believed would limit concurrency, and therefore connections). The publisher is using basic_publish
, and consumers for this particular queue are in a different server. Successfully published messages are also successfully processed.
We have other pieces of software using CloudAMQP, and AMQProxy will be very helpful in case we are able to get this prototype right.
Server log sample:
(... alarm setting and clearing repeated many many times above ...)
**********************************************************
*** Publishers will be blocked until this alarm clears ***
**********************************************************
=INFO EVENT==== Fri, 18 Jan 2019 19:27:50 GMT ===
vm_memory_high_watermark clear. Memory used:839187024 allowed:842953236
=WARNING EVENT==== Fri, 18 Jan 2019 19:27:50 GMT ===
memory resource limit alarm cleared on node 'rabbit@<server name>'
=INFO EVENT==== Fri, 18 Jan 2019 19:27:51 GMT ===
vm_memory_high_watermark set. Memory used:955837240 allowed:842953236
=WARNING EVENT==== Fri, 18 Jan 2019 19:27:51 GMT ===
memory resource limit alarm set on node 'rabbit@<server name>'.
**********************************************************
*** Publishers will be blocked until this alarm clears ***
**********************************************************
(... there were many of the message below ...)
=ERROR EVENT==== Fri, 18 Jan 2019 19:28:13 GMT ===
closing AMQP connection <0.16260.3> (x.x.x.x:46390 -> y.y.y.y:5672):
{handshake_timeout,frame_header}
=WARNING EVENT==== Fri, 18 Jan 2019 19:29:07 GMT ===
file descriptor limit alarm set.
********************************************************************
*** New connections will not be accepted until this alarm clears ***
********************************************************************
=WARNING EVENT==== Fri, 18 Jan 2019 19:29:18 GMT ===
file descriptor limit alarm cleared
=INFO EVENT==== Fri, 18 Jan 2019 19:30:06 GMT ===
started TCP Listener on 0.0.0.0:5672
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
started SSL Listener on 0.0.0.0:5671
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
rabbit_stomp: default user 'guest' enabled
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
started STOMP TCP Listener on 0.0.0.0:61613
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
started STOMP SSL Listener on 0.0.0.0:61614
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
rabbit_web_stomp: listening for HTTP connections on 0.0.0.0:15674
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
Management plugin started. Port: 15672
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
Statistics database started.
=INFO EVENT==== Fri, 18 Jan 2019 19:30:07 GMT ===
Server startup complete; 15 plugins started.
* rabbitmq_shovel_management
* rabbitmq_federation_management
* rabbitmq_management
* rabbitmq_web_dispatch
* webmachine
* mochiweb
* rabbitmq_management_agent
* rabbitmq_shovel
* rabbitmq_federation
* rabbitmq_web_stomp
* rabbitmq_stomp
* rabbitmq_consistent_hash_exchange
* amqp_client
* cowboy
* sockjs
One can see the spikes in connections:
and how they seem unrelated to the number of messages in the queues, over that time period:
I installed the service and configured it to hit my rabbitmq server, and my code is configured to hit the amqproxy (and working)
however i dont see any performance increase and see a bunch of this in the system.log:
Client connection closed from [::ffff:127.0.0.1]:36414
Client connection accepted from [::ffff:127.0.0.1]:36422
So can I confirm 1 way or another that its actually reusing a connection?
I'm having the following log error when a php client connects (php-amqplib), and not sure the reason
Connected to upstream x.x.x.x:5672
Unhandled exception in spawn: Invalid time: seconds out of range (ArgumentError)
from ???
from ???
from ???
from ???
from ???
from ???
Connected to upstream x.x.x.x:5672
Unhandled exception in spawn: Invalid time: seconds out of range (ArgumentError)
from ???
from ???
from ???
from ???
from ???
from ???
I couldn't set up amqproxy under debian buster even after looking up debian bash script. It seems to only account for debian stretch.
`docker build -t amqproxy:v0.4.5 https://github.com/cloudamqp/amqproxy.git#v0.4.5
Sending build context to Docker daemon 70.14kB
Step 1/8 : FROM crystallang/crystal
---> 4687b017edf3
Step 2/8 : WORKDIR /app
---> Using cache
---> 6ab8562c6b50
Step 3/8 : COPY . .
---> Using cache
---> 15a7f1d11032
Step 4/8 : RUN shards build --release --production
---> Running in e2c4d603cac8
Resolving dependencies
Fetching https://github.com/cloudamqp/amq-protocol.cr.git
Installing amq-protocol (0.3.6)
Building: amqproxy
Error target amqproxy failed to compile:
Showing last frame. Use --error-trace for full trace.
In lib/amq-protocol/src/amq/protocol/table.cr:227:32
227 | private def skip_field : Int32
^
Error: method must return Int32 but it is returning Int64
`
sometimes it is better or even necessary to use environment variables instead of arguments. the crytsal image in the dokerfile is good to build the binary, but for a running system you have to use a multistage build, so you can shrink your running image to a size of 12.4MB instead of 633MB. but mostly without bash or something else (scratch image) you can't use CMD, where arguments can be passed. So you wish to use ENTRYPOINT with environment variables. i created a feature #18 so the environment variables take place before defaults but after arguments.
$ git clone https://github.com/cloudamqp/amqproxy /tmp/amqproxy
Cloning into '/tmp/amqproxy'...
$ cd /tmp/amqproxy
$ docker build -t amqproxy:latest .
Step 1/8 : FROM crystallang/crystal
latest: Pulling from crystallang/crystal
d7c3167c320d: Pulling fs layer
131f805ec7fd: Pulling fs layer
322ed380e680: Pulling fs layer
6ac240b13098: Pulling fs layer
22da88a5d530: Pulling fs layer
6ac240b13098: Waiting
22da88a5d530: Waiting
322ed380e680: Verifying Checksum
322ed380e680: Download complete
131f805ec7fd: Verifying Checksum
131f805ec7fd: Download complete
d7c3167c320d: Verifying Checksum
d7c3167c320d: Download complete
6ac240b13098: Verifying Checksum
6ac240b13098: Download complete
d7c3167c320d: Pull complete
22da88a5d530: Verifying Checksum
22da88a5d530: Download complete
131f805ec7fd: Pull complete
322ed380e680: Pull complete
6ac240b13098: Pull complete
22da88a5d530: Pull complete
Digest: sha256:afd58df611c04807961539f884bbe8dfd25ea9fd54132cbea2c01f5475689ebd
Status: Downloaded newer image for crystallang/crystal:latest
---> ee810d01edd1
Step 2/8 : WORKDIR /app
---> Running in b5e32584d9af
Removing intermediate container b5e32584d9af
---> 99bff998ec9a
Step 3/8 : COPY . .
---> ba5593b5f51b
Step 4/8 : RUN shards build --release --production
---> Running in 1f9eb895802c
Resolving dependencies
Fetching https://github.com/cloudamqp/amq-protocol.cr.git
Installing amq-protocol (0.3.12)
Building: amqproxy
Error target amqproxy failed to compile:
Showing last frame. Use --error-trace for full trace.
In lib/amq-protocol/src/amq/protocol/frames.cr:416:21
416 | bytes += io.write_bytes(@reply_code, format)
^
Error: no overload matches 'Int64#+' with type Nil
Overloads are:
We are running AmqProxy in a mostly RPC style setup and are experiencing some annoying memory leaks.
It looks like the proces is leaking 8188k sized page tables, which causes the server to run out of memory:
We figured the servers could use a nice upgrade due to recent load issues and steady growth, but it blew throug 4gb of extra ram within a day:
me@api-server-02:~$ ./getfreemem.sh
PID USER %MEM COMMAND
861 censored+ 4.8 /home/censored/amqproxy/amqproxy -l 127.0.0.1 -p 5673 -d amqps://censored.rmq.cloudamqp.com
430 root 0.5 php-fpm: master process (/etc/php/7.4/fpm/php-fpm.conf)
9746 censored+ 0.4 php-fpm: pool censored
9745 censored+ 0.4 php-fpm: pool censored
973 censored+ 0.4 php-fpm: pool censored
972 censored+ 0.4 php-fpm: pool censored
971 censored+ 0.4 php-fpm: pool censored
9615 censored+ 0.4 php-fpm: pool censored
9313 censored+ 0.4 php-fpm: pool censored
9299 censored+ 0.4 php-fpm: pool censored
me@api-server-02:~$ sudo pmap 861
861: /home/censored/amqproxy/amqproxy -l 127.0.0.1 -p 5673 -d amqps://censored.rmq.cloudamqp.com
0000563f03e5a000 896K r-x-- amqproxy
0000563f0413a000 4K r---- amqproxy
0000563f0413b000 4K rw--- amqproxy
0000563f0413c000 2384K rw--- [ anon ]
0000563f054b0000 12132K rw--- [ anon ]
00007fae56907000 4K ----- [ anon ]
00007fae56908000 8188K rw--- [ anon ]
00007fae57107000 4K ----- [ anon ]
00007fae57108000 8188K rw--- [ anon ]
00007fae58907000 4K ----- [ anon ]
00007fae58908000 8188K rw--- [ anon ]
00007fae59107000 4K ----- [ anon ]
00007fae59108000 8188K rw--- [ anon ]
00007fae59907000 4K ----- [ anon ]
00007fae59908000 8188K rw--- [ anon ]
00007fae5a907000 4K ----- [ anon ]
00007fae5a908000 8188K rw--- [ anon ]
00007fae5b107000 4K ----- [ anon ]
00007fae5b108000 8188K rw--- [ anon ]
00007fae5b907000 4K ----- [ anon ]
00007fae5b908000 8188K rw--- [ anon ]
00007fae5c107000 4K ----- [ anon ]
00007fae5c108000 8188K rw--- [ anon ]
00007fae5c907000 4K ----- [ anon ]
00007fae5c908000 8188K rw--- [ anon ]
00007fae5d107000 4K ----- [ anon ]
00007fae5d108000 8188K rw--- [ anon ]
00007fae5d907000 4K ----- [ anon ]
00007fae5d908000 8188K rw--- [ anon ]
00007fae5e107000 4K ----- [ anon ]
00007fae5e108000 8188K rw--- [ anon ]
00007fae5e907000 4K ----- [ anon ]
00007fae5e908000 8188K rw--- [ anon ]
00007fae5f107000 4K ----- [ anon ]
00007fae5f108000 8188K rw--- [ anon ]
00007fae5f907000 4K ----- [ anon ]
00007fae5f908000 8188K rw--- [ anon ]
00007fae60107000 4K ----- [ anon ]
00007fae60108000 8188K rw--- [ anon ]
00007fae60907000 4K ----- [ anon ]
00007fae60908000 8188K rw--- [ anon ]
00007fae61907000 4K ----- [ anon ]
00007fae61908000 8188K rw--- [ anon ]
00007fae62107000 4K ----- [ anon ]
00007fae62108000 8188K rw--- [ anon ]
00007fae62907000 4K ----- [ anon ]
00007fae62908000 8188K rw--- [ anon ]
00007fae63907000 4K ----- [ anon ]
00007fae63908000 8188K rw--- [ anon ]
00007fae64107000 4K ----- [ anon ]
00007fae64108000 8188K rw--- [ anon ]
00007fae64907000 4K ----- [ anon ]
00007fae64908000 8188K rw--- [ anon ]
00007fae65107000 4K ----- [ anon ]
00007fae65108000 8188K rw--- [ anon ]
00007fae65907000 4K ----- [ anon ]
00007fae65908000 8188K rw--- [ anon ]
00007fae66107000 4K ----- [ anon ]
00007fae66108000 8188K rw--- [ anon ]
00007fae66907000 4K ----- [ anon ]
00007fae66908000 8188K rw--- [ anon ]
00007fae67907000 4K ----- [ anon ]
00007fae67908000 8188K rw--- [ anon ]
00007fae68107000 4K ----- [ anon ]
00007fae68108000 8188K rw--- [ anon ]
00007fae68907000 4K ----- [ anon ]
00007fae68908000 8188K rw--- [ anon ]
00007fae69107000 4K ----- [ anon ]
00007fae69108000 8188K rw--- [ anon ]
00007fae69907000 4K ----- [ anon ]
00007fae69908000 8188K rw--- [ anon ]
00007fae6a107000 4K ----- [ anon ]
00007fae6a108000 8188K rw--- [ anon ]
00007fae6a907000 4K ----- [ anon ]
00007fae6a908000 8188K rw--- [ anon ]
00007fae6b107000 4K ----- [ anon ]
00007fae6b108000 8188K rw--- [ anon ]
00007fae6b907000 4K ----- [ anon ]
00007fae6b908000 8188K rw--- [ anon ]
00007fae6c107000 4K ----- [ anon ]
00007fae6c108000 8188K rw--- [ anon ]
00007fae6c907000 4K ----- [ anon ]
00007fae6c908000 8188K rw--- [ anon ]
00007fae6d107000 4K ----- [ anon ]
00007fae6d108000 8188K rw--- [ anon ]
00007fae6d907000 4K ----- [ anon ]
00007fae6d908000 8188K rw--- [ anon ]
00007fae6e107000 4K ----- [ anon ]
00007fae6e108000 8188K rw--- [ anon ]
00007fae6e907000 4K ----- [ anon ]
00007fae6e908000 8188K rw--- [ anon ]
00007fae6f107000 4K ----- [ anon ]
00007fae6f108000 8188K rw--- [ anon ]
00007fae6f907000 4K ----- [ anon ]
00007fae6f908000 8188K rw--- [ anon ]
00007fae70107000 4K ----- [ anon ]
00007fae70108000 8188K rw--- [ anon ]
00007fae70907000 4K ----- [ anon ]
00007fae70908000 8188K rw--- [ anon ]
00007fae71107000 4K ----- [ anon ]
00007fae71108000 8188K rw--- [ anon ]
00007fae71907000 4K ----- [ anon ]
00007fae71908000 8188K rw--- [ anon ]
00007fae72107000 4K ----- [ anon ]
00007fae72108000 8188K rw--- [ anon ]
00007fae72907000 4K ----- [ anon ]
00007fae72908000 8188K rw--- [ anon ]
00007fae73107000 4K ----- [ anon ]
00007fae73108000 8188K rw--- [ anon ]
00007fae73907000 4K ----- [ anon ]
00007fae73908000 8188K rw--- [ anon ]
00007fae74107000 4K ----- [ anon ]
00007fae74108000 8188K rw--- [ anon ]
00007fae74907000 4K ----- [ anon ]
00007fae74908000 8188K rw--- [ anon ]
00007fae75107000 4K ----- [ anon ]
00007fae75108000 8188K rw--- [ anon ]
00007fae75907000 4K ----- [ anon ]
00007fae75908000 8188K rw--- [ anon ]
00007fae76107000 4K ----- [ anon ]
00007fae76108000 8188K rw--- [ anon ]
00007fae76907000 4K ----- [ anon ]
00007fae76908000 8188K rw--- [ anon ]
00007fae77107000 4K ----- [ anon ]
00007fae77108000 8188K rw--- [ anon ]
00007fae77907000 4K ----- [ anon ]
00007fae77908000 8188K rw--- [ anon ]
00007fae78107000 4K ----- [ anon ]
00007fae78108000 8188K rw--- [ anon ]
00007fae78907000 4K ----- [ anon ]
00007fae78908000 8188K rw--- [ anon ]
00007fae79107000 4K ----- [ anon ]
00007fae79108000 8188K rw--- [ anon ]
00007fae79907000 4K ----- [ anon ]
00007fae79908000 8188K rw--- [ anon ]
00007fae7a107000 4K ----- [ anon ]
00007fae7a108000 8188K rw--- [ anon ]
00007fae7a907000 4K ----- [ anon ]
00007fae7a908000 8188K rw--- [ anon ]
00007fae7b107000 4K ----- [ anon ]
00007fae7b108000 8188K rw--- [ anon ]
00007fae7b907000 4K ----- [ anon ]
00007fae7b908000 8188K rw--- [ anon ]
00007fae7c107000 4K ----- [ anon ]
00007fae7c108000 8188K rw--- [ anon ]
00007fae7c907000 4K ----- [ anon ]
00007fae7c908000 8188K rw--- [ anon ]
00007fae7d107000 4K ----- [ anon ]
00007fae7d108000 8188K rw--- [ anon ]
00007fae7d907000 4K ----- [ anon ]
00007fae7d908000 8188K rw--- [ anon ]
00007fae7e107000 4K ----- [ anon ]
00007fae7e108000 8188K rw--- [ anon ]
00007fae7e907000 4K ----- [ anon ]
00007fae7e908000 8188K rw--- [ anon ]
00007fae7f107000 4K ----- [ anon ]
00007fae7f108000 8188K rw--- [ anon ]
00007fae7f907000 4K ----- [ anon ]
00007fae7f908000 8188K rw--- [ anon ]
00007fae80107000 4K ----- [ anon ]
00007fae80108000 8188K rw--- [ anon ]
00007fae80907000 4K ----- [ anon ]
00007fae80908000 8188K rw--- [ anon ]
00007fae81107000 4K ----- [ anon ]
00007fae81108000 8188K rw--- [ anon ]
00007fae81907000 4K ----- [ anon ]
00007fae81908000 8188K rw--- [ anon ]
00007fae82107000 4K ----- [ anon ]
00007fae82108000 8188K rw--- [ anon ]
00007fae82907000 4K ----- [ anon ]
00007fae82908000 8188K rw--- [ anon ]
00007fae83107000 4K ----- [ anon ]
00007fae83108000 8188K rw--- [ anon ]
00007fae83907000 4K ----- [ anon ]
00007fae83908000 8188K rw--- [ anon ]
00007fae84107000 4K ----- [ anon ]
00007fae84108000 8188K rw--- [ anon ]
00007fae84907000 4K ----- [ anon ]
00007fae84908000 8188K rw--- [ anon ]
00007fae85107000 4K ----- [ anon ]
00007fae85108000 8188K rw--- [ anon ]
00007fae85907000 4K ----- [ anon ]
00007fae85908000 8188K rw--- [ anon ]
00007fae86107000 4K ----- [ anon ]
00007fae86108000 8188K rw--- [ anon ]
00007fae86907000 4K ----- [ anon ]
00007fae86908000 8188K rw--- [ anon ]
00007fae87107000 4K ----- [ anon ]
00007fae87108000 8188K rw--- [ anon ]
00007fae87907000 4K ----- [ anon ]
00007fae87908000 8188K rw--- [ anon ]
00007fae88107000 4K ----- [ anon ]
00007fae88108000 8188K rw--- [ anon ]
00007fae88907000 4K ----- [ anon ]
00007fae88908000 8188K rw--- [ anon ]
00007fae89107000 4K ----- [ anon ]
00007fae89108000 8188K rw--- [ anon ]
00007fae89907000 4K ----- [ anon ]
00007fae89908000 8188K rw--- [ anon ]
00007fae8a107000 4K ----- [ anon ]
00007fae8a108000 8188K rw--- [ anon ]
00007fae8a907000 4K ----- [ anon ]
00007fae8a908000 8188K rw--- [ anon ]
00007fae8b107000 4K ----- [ anon ]
00007fae8b108000 8188K rw--- [ anon ]
00007fae8b907000 4K ----- [ anon ]
00007fae8b908000 8188K rw--- [ anon ]
00007fae8c107000 4K ----- [ anon ]
00007fae8c108000 8188K rw--- [ anon ]
00007fae8c907000 4K ----- [ anon ]
00007fae8c908000 8188K rw--- [ anon ]
00007fae8d107000 4K ----- [ anon ]
00007fae8d108000 8188K rw--- [ anon ]
00007fae8d907000 4K ----- [ anon ]
00007fae8d908000 8188K rw--- [ anon ]
00007fae8e107000 4K ----- [ anon ]
00007fae8e108000 8188K rw--- [ anon ]
00007fae8e907000 4K ----- [ anon ]
00007fae8e908000 8188K rw--- [ anon ]
00007fae8f107000 4K ----- [ anon ]
00007fae8f108000 8188K rw--- [ anon ]
00007fae8f907000 4K ----- [ anon ]
00007fae8f908000 8188K rw--- [ anon ]
00007fae90107000 4K ----- [ anon ]
00007fae90108000 8188K rw--- [ anon ]
00007fae90907000 4K ----- [ anon ]
00007fae90908000 8188K rw--- [ anon ]
00007fae91107000 4K ----- [ anon ]
00007fae91108000 8188K rw--- [ anon ]
00007fae91907000 4K ----- [ anon ]
00007fae91908000 8188K rw--- [ anon ]
00007fae92107000 4K ----- [ anon ]
00007fae92108000 8188K rw--- [ anon ]
00007fae92907000 4K ----- [ anon ]
00007fae92908000 8188K rw--- [ anon ]
00007fae93107000 4K ----- [ anon ]
00007fae93108000 8188K rw--- [ anon ]
00007fae93907000 4K ----- [ anon ]
00007fae93908000 8188K rw--- [ anon ]
00007fae94107000 4K ----- [ anon ]
00007fae94108000 8188K rw--- [ anon ]
00007fae94907000 4K ----- [ anon ]
00007fae94908000 8188K rw--- [ anon ]
00007fae95107000 4K ----- [ anon ]
00007fae95108000 8188K rw--- [ anon ]
00007fae95907000 4K ----- [ anon ]
00007fae95908000 8188K rw--- [ anon ]
00007fae96107000 4K ----- [ anon ]
00007fae96108000 8188K rw--- [ anon ]
00007fae96907000 4K ----- [ anon ]
00007fae96908000 8188K rw--- [ anon ]
00007fae97107000 4K ----- [ anon ]
00007fae97108000 8188K rw--- [ anon ]
00007fae97907000 4K ----- [ anon ]
00007fae97908000 8188K rw--- [ anon ]
00007fae98107000 4K ----- [ anon ]
00007fae98108000 8188K rw--- [ anon ]
00007fae98907000 4K ----- [ anon ]
00007fae98908000 8188K rw--- [ anon ]
00007fae99107000 4K ----- [ anon ]
00007fae99108000 8188K rw--- [ anon ]
00007fae99907000 4K ----- [ anon ]
00007fae99908000 8188K rw--- [ anon ]
00007fae9a107000 4K ----- [ anon ]
00007fae9a108000 8188K rw--- [ anon ]
00007fae9a907000 4K ----- [ anon ]
00007fae9a908000 8188K rw--- [ anon ]
00007fae9b107000 4K ----- [ anon ]
00007fae9b108000 8188K rw--- [ anon ]
00007fae9b907000 4K ----- [ anon ]
00007fae9b908000 8188K rw--- [ anon ]
00007fae9c107000 4K ----- [ anon ]
00007fae9c108000 8188K rw--- [ anon ]
00007fae9c907000 4K ----- [ anon ]
00007fae9c908000 8188K rw--- [ anon ]
00007fae9d107000 4K ----- [ anon ]
00007fae9d108000 8188K rw--- [ anon ]
00007fae9d907000 4K ----- [ anon ]
00007fae9d908000 8188K rw--- [ anon ]
00007fae9e107000 4K ----- [ anon ]
00007fae9e108000 8188K rw--- [ anon ]
00007fae9e907000 4K ----- [ anon ]
00007fae9e908000 8188K rw--- [ anon ]
00007fae9f107000 4K ----- [ anon ]
00007fae9f108000 8188K rw--- [ anon ]
00007fae9f907000 4K ----- [ anon ]
00007fae9f908000 8188K rw--- [ anon ]
00007faea0107000 4K ----- [ anon ]
00007faea0108000 8188K rw--- [ anon ]
00007faea0907000 4K ----- [ anon ]
00007faea0908000 8188K rw--- [ anon ]
00007faea1107000 4K ----- [ anon ]
00007faea1108000 8188K rw--- [ anon ]
00007faea1907000 4K ----- [ anon ]
00007faea1908000 8188K rw--- [ anon ]
00007faea2107000 4K ----- [ anon ]
00007faea2108000 8188K rw--- [ anon ]
00007faea2907000 4K ----- [ anon ]
00007faea2908000 8188K rw--- [ anon ]
00007faea3107000 4K ----- [ anon ]
00007faea3108000 8188K rw--- [ anon ]
00007faea3907000 4K ----- [ anon ]
00007faea3908000 8188K rw--- [ anon ]
00007faea4107000 4K ----- [ anon ]
00007faea4108000 8188K rw--- [ anon ]
00007faea4907000 4K ----- [ anon ]
00007faea4908000 8188K rw--- [ anon ]
00007faea5107000 4K ----- [ anon ]
00007faea5108000 8188K rw--- [ anon ]
00007faea5907000 4K ----- [ anon ]
00007faea5908000 8188K rw--- [ anon ]
00007faea6107000 4K ----- [ anon ]
00007faea6108000 8188K rw--- [ anon ]
00007faea6907000 4K ----- [ anon ]
00007faea6908000 8188K rw--- [ anon ]
00007faea7107000 4K ----- [ anon ]
00007faea7108000 8188K rw--- [ anon ]
00007faea7907000 4K ----- [ anon ]
00007faea7908000 8188K rw--- [ anon ]
00007faea8107000 4K ----- [ anon ]
00007faea8108000 8188K rw--- [ anon ]
00007faea8907000 4K ----- [ anon ]
00007faea8908000 8188K rw--- [ anon ]
00007faea9107000 4K ----- [ anon ]
00007faea9108000 8188K rw--- [ anon ]
00007faea9907000 4K ----- [ anon ]
00007faea9908000 8188K rw--- [ anon ]
00007faeaa107000 4K ----- [ anon ]
00007faeaa108000 8188K rw--- [ anon ]
00007faeaa907000 4K ----- [ anon ]
00007faeaa908000 8188K rw--- [ anon ]
00007faeab107000 4K ----- [ anon ]
00007faeab108000 8188K rw--- [ anon ]
00007faeab907000 4K ----- [ anon ]
00007faeab908000 8188K rw--- [ anon ]
00007faeac107000 4K ----- [ anon ]
00007faeac108000 8188K rw--- [ anon ]
00007faeac907000 4K ----- [ anon ]
00007faeac908000 8188K rw--- [ anon ]
00007faead107000 4K ----- [ anon ]
00007faead108000 8188K rw--- [ anon ]
00007faead907000 4K ----- [ anon ]
00007faead908000 8188K rw--- [ anon ]
00007faeae107000 4K ----- [ anon ]
00007faeae108000 8188K rw--- [ anon ]
00007faeae907000 4K ----- [ anon ]
00007faeae908000 8188K rw--- [ anon ]
00007faeaf107000 4K ----- [ anon ]
00007faeaf108000 8188K rw--- [ anon ]
00007faeaf907000 4K ----- [ anon ]
00007faeaf908000 8188K rw--- [ anon ]
00007faeb0107000 4K ----- [ anon ]
00007faeb0108000 8188K rw--- [ anon ]
00007faeb0907000 4K ----- [ anon ]
00007faeb0908000 8188K rw--- [ anon ]
00007faeb1107000 4K ----- [ anon ]
00007faeb1108000 8188K rw--- [ anon ]
00007faeb1907000 4K ----- [ anon ]
00007faeb1908000 8188K rw--- [ anon ]
00007faeb2107000 4K ----- [ anon ]
00007faeb2108000 8188K rw--- [ anon ]
00007faeb2907000 4K ----- [ anon ]
00007faeb2908000 8188K rw--- [ anon ]
00007faeb3107000 4K ----- [ anon ]
00007faeb3108000 8188K rw--- [ anon ]
00007faeb3907000 4K ----- [ anon ]
00007faeb3908000 8188K rw--- [ anon ]
00007faeb4107000 4K ----- [ anon ]
00007faeb4108000 8188K rw--- [ anon ]
00007faeb4907000 4K ----- [ anon ]
00007faeb4908000 8188K rw--- [ anon ]
00007faeb5107000 4K ----- [ anon ]
00007faeb5108000 8188K rw--- [ anon ]
00007faeb5907000 4K ----- [ anon ]
00007faeb5908000 8188K rw--- [ anon ]
00007faeb6107000 4K ----- [ anon ]
00007faeb6108000 8188K rw--- [ anon ]
00007faeb6907000 4K ----- [ anon ]
00007faeb6908000 8188K rw--- [ anon ]
00007faeb7107000 4K ----- [ anon ]
00007faeb7108000 8188K rw--- [ anon ]
00007faeb7907000 4K ----- [ anon ]
00007faeb7908000 8188K rw--- [ anon ]
00007faeb8107000 4K ----- [ anon ]
00007faeb8108000 8188K rw--- [ anon ]
00007faeb8907000 4K ----- [ anon ]
00007faeb8908000 8188K rw--- [ anon ]
00007faeb9107000 4K ----- [ anon ]
00007faeb9108000 8188K rw--- [ anon ]
00007faeb9907000 4K ----- [ anon ]
00007faeb9908000 8188K rw--- [ anon ]
00007faeba107000 4K ----- [ anon ]
00007faeba108000 8188K rw--- [ anon ]
00007faeba907000 4K ----- [ anon ]
00007faeba908000 8188K rw--- [ anon ]
00007faebb107000 4K ----- [ anon ]
00007faebb108000 8188K rw--- [ anon ]
00007faebb907000 4K ----- [ anon ]
00007faebb908000 8188K rw--- [ anon ]
00007faebc107000 4K ----- [ anon ]
00007faebc108000 8188K rw--- [ anon ]
00007faebc907000 4K ----- [ anon ]
00007faebc908000 8188K rw--- [ anon ]
00007faebd107000 4K ----- [ anon ]
00007faebd108000 8188K rw--- [ anon ]
00007faebd907000 4K ----- [ anon ]
00007faebd908000 8188K rw--- [ anon ]
00007faebe107000 4K ----- [ anon ]
00007faebe108000 8188K rw--- [ anon ]
00007faebe907000 4K ----- [ anon ]
00007faebe908000 8188K rw--- [ anon ]
00007faebf107000 4K ----- [ anon ]
00007faebf108000 8188K rw--- [ anon ]
00007faebf907000 4K ----- [ anon ]
00007faebf908000 8188K rw--- [ anon ]
00007faec0107000 4K ----- [ anon ]
00007faec0108000 8188K rw--- [ anon ]
00007faec0907000 4K ----- [ anon ]
00007faec0908000 8188K rw--- [ anon ]
00007faec1107000 4K ----- [ anon ]
00007faec1108000 8188K rw--- [ anon ]
00007faec1907000 4K ----- [ anon ]
00007faec1908000 8188K rw--- [ anon ]
00007faec2107000 4K ----- [ anon ]
00007faec2108000 8188K rw--- [ anon ]
00007faec2907000 4K ----- [ anon ]
00007faec2908000 8188K rw--- [ anon ]
00007faec3107000 4K ----- [ anon ]
00007faec3108000 8188K rw--- [ anon ]
00007faec3907000 4K ----- [ anon ]
00007faec3908000 8188K rw--- [ anon ]
00007faec4107000 4K ----- [ anon ]
00007faec4108000 8188K rw--- [ anon ]
00007faec4907000 4K ----- [ anon ]
00007faec4908000 8188K rw--- [ anon ]
00007faec5107000 4K ----- [ anon ]
00007faec5108000 8188K rw--- [ anon ]
00007faec5907000 4K ----- [ anon ]
00007faec5908000 8188K rw--- [ anon ]
00007faec6107000 4K ----- [ anon ]
00007faec6108000 8188K rw--- [ anon ]
00007faec6907000 4K ----- [ anon ]
00007faec6908000 8188K rw--- [ anon ]
00007faec7107000 4K ----- [ anon ]
00007faec7108000 8188K rw--- [ anon ]
00007faec7907000 4K ----- [ anon ]
00007faec7908000 8188K rw--- [ anon ]
00007faec8107000 4K ----- [ anon ]
00007faec8108000 8188K rw--- [ anon ]
00007faec8907000 4K ----- [ anon ]
00007faec8908000 8188K rw--- [ anon ]
00007faec9107000 4K ----- [ anon ]
00007faec9108000 8188K rw--- [ anon ]
00007faec9907000 4K ----- [ anon ]
00007faec9908000 8188K rw--- [ anon ]
00007faeca107000 4K ----- [ anon ]
00007faeca108000 8188K rw--- [ anon ]
00007faeca907000 4K ----- [ anon ]
00007faeca908000 8188K rw--- [ anon ]
00007faecb107000 4K ----- [ anon ]
00007faecb108000 8188K rw--- [ anon ]
00007faecb907000 4K ----- [ anon ]
00007faecb908000 8188K rw--- [ anon ]
00007faecc107000 4K ----- [ anon ]
00007faecc108000 8188K rw--- [ anon ]
00007faecc907000 4K ----- [ anon ]
00007faecc908000 8188K rw--- [ anon ]
00007faecd107000 4K ----- [ anon ]
00007faecd108000 8188K rw--- [ anon ]
00007faecd907000 4K ----- [ anon ]
00007faecd908000 8188K rw--- [ anon ]
00007faece107000 4K ----- [ anon ]
00007faece108000 8188K rw--- [ anon ]
00007faece907000 4K ----- [ anon ]
00007faece908000 8188K rw--- [ anon ]
00007faecf107000 4K ----- [ anon ]
00007faecf108000 8188K rw--- [ anon ]
00007faecf907000 4K ----- [ anon ]
00007faecf908000 8188K rw--- [ anon ]
00007faed0107000 4K ----- [ anon ]
00007faed0108000 8188K rw--- [ anon ]
00007faed0907000 4K ----- [ anon ]
00007faed0908000 8188K rw--- [ anon ]
00007faed1107000 4K ----- [ anon ]
00007faed1108000 8188K rw--- [ anon ]
00007faed1907000 4K ----- [ anon ]
00007faed1908000 8188K rw--- [ anon ]
00007faed2107000 32776K rw--- [ anon ]
00007faed4109000 4K ----- [ anon ]
00007faed410a000 8188K rw--- [ anon ]
00007faed4909000 4K ----- [ anon ]
00007faed490a000 8188K rw--- [ anon ]
00007faed5109000 4K ----- [ anon ]
00007faed510a000 8188K rw--- [ anon ]
00007faed5909000 4K ----- [ anon ]
00007faed590a000 8188K rw--- [ anon ]
00007faed6109000 4K ----- [ anon ]
00007faed610a000 8188K rw--- [ anon ]
00007faed6909000 4K ----- [ anon ]
00007faed690a000 8188K rw--- [ anon ]
00007faed7109000 4K ----- [ anon ]
00007faed710a000 8188K rw--- [ anon ]
00007faed7909000 4K ----- [ anon ]
00007faed790a000 8188K rw--- [ anon ]
00007faed8109000 4K ----- [ anon ]
00007faed810a000 8188K rw--- [ anon ]
00007faed8909000 4K ----- [ anon ]
00007faed890a000 8188K rw--- [ anon ]
00007faed9109000 4K ----- [ anon ]
00007faed910a000 8188K rw--- [ anon ]
00007faed9909000 4K ----- [ anon ]
00007faed990a000 8188K rw--- [ anon ]
00007faeda109000 4K ----- [ anon ]
00007faeda10a000 8188K rw--- [ anon ]
00007faeda909000 4K ----- [ anon ]
00007faeda90a000 8188K rw--- [ anon ]
00007faedb109000 4K ----- [ anon ]
00007faedb10a000 8188K rw--- [ anon ]
00007faedb909000 4K ----- [ anon ]
00007faedb90a000 8188K rw--- [ anon ]
00007faedc109000 4K ----- [ anon ]
00007faedc10a000 8188K rw--- [ anon ]
00007faedc909000 4K ----- [ anon ]
00007faedc90a000 8188K rw--- [ anon ]
00007faedd109000 4K ----- [ anon ]
00007faedd10a000 8188K rw--- [ anon ]
00007faede108000 4K ----- [ anon ]
00007faede109000 8188K rw--- [ anon ]
00007faede908000 65540K rw--- [ anon ]
00007faee2909000 4K ----- [ anon ]
00007faee290a000 8188K rw--- [ anon ]
00007faee3109000 4K ----- [ anon ]
00007faee310a000 8188K rw--- [ anon ]
00007faee3909000 4K ----- [ anon ]
00007faee390a000 8188K rw--- [ anon ]
00007faee4109000 4K ----- [ anon ]
00007faee410a000 8188K rw--- [ anon ]
00007faee4909000 4K ----- [ anon ]
00007faee490a000 8188K rw--- [ anon ]
00007faee5109000 4K ----- [ anon ]
00007faee510a000 8188K rw--- [ anon ]
00007faee5909000 4K ----- [ anon ]
00007faee590a000 8188K rw--- [ anon ]
00007faee6109000 4K ----- [ anon ]
00007faee610a000 8188K rw--- [ anon ]
00007faee6909000 16384K rw--- [ anon ]
00007faee7909000 4K ----- [ anon ]
00007faee790a000 8188K rw--- [ anon ]
00007faee8109000 4K ----- [ anon ]
00007faee810a000 8188K rw--- [ anon ]
00007faee8909000 4K ----- [ anon ]
00007faee890a000 8188K rw--- [ anon ]
00007faee9109000 4K ----- [ anon ]
00007faee910a000 8188K rw--- [ anon ]
00007faee9909000 16384K rw--- [ anon ]
00007faeea909000 4K ----- [ anon ]
00007faeea90a000 8188K rw--- [ anon ]
00007faeeb109000 4K ----- [ anon ]
00007faeeb10a000 8188K rw--- [ anon ]
00007faeeb909000 4K ----- [ anon ]
00007faeeb90a000 8188K rw--- [ anon ]
00007faeec109000 4K ----- [ anon ]
00007faeec10a000 8188K rw--- [ anon ]
00007faeec909000 4K ----- [ anon ]
00007faeec90a000 8188K rw--- [ anon ]
00007faeed109000 4K ----- [ anon ]
00007faeed10a000 8188K rw--- [ anon ]
00007faeed909000 4K ----- [ anon ]
00007faeed90a000 8188K rw--- [ anon ]
00007faeee109000 16392K rw--- [ anon ]
00007faeef10b000 4K ----- [ anon ]
00007faeef10c000 8188K rw--- [ anon ]
00007faef010a000 4K ----- [ anon ]
00007faef010b000 8188K rw--- [ anon ]
00007faef090a000 4K ----- [ anon ]
00007faef090b000 8188K rw--- [ anon ]
00007faef110a000 4K ----- [ anon ]
00007faef110b000 8188K rw--- [ anon ]
00007faef190a000 4K ----- [ anon ]
00007faef190b000 8188K rw--- [ anon ]
00007faef210a000 4K ----- [ anon ]
00007faef210b000 8188K rw--- [ anon ]
00007faef290a000 65540K rw--- [ anon ]
00007faef690b000 4K ----- [ anon ]
00007faef690c000 8188K rw--- [ anon ]
00007faef710b000 4K ----- [ anon ]
00007faef710c000 8188K rw--- [ anon ]
00007faef790b000 4K ----- [ anon ]
00007faef790c000 8188K rw--- [ anon ]
00007faef810b000 4K ----- [ anon ]
00007faef810c000 8188K rw--- [ anon ]
00007faef890b000 4K ----- [ anon ]
00007faef890c000 8188K rw--- [ anon ]
00007faef910b000 4K ----- [ anon ]
00007faef910c000 8188K rw--- [ anon ]
00007faef990b000 4K ----- [ anon ]
00007faef990c000 8188K rw--- [ anon ]
00007faefa10b000 4K ----- [ anon ]
00007faefa10c000 8188K rw--- [ anon ]
00007faefa90b000 4K ----- [ anon ]
00007faefa90c000 8188K rw--- [ anon ]
00007faefb10b000 4K ----- [ anon ]
00007faefb10c000 8188K rw--- [ anon ]
00007faefb90b000 4K ----- [ anon ]
00007faefb90c000 8188K rw--- [ anon ]
00007faefc10b000 4K ----- [ anon ]
00007faefc10c000 8188K rw--- [ anon ]
00007faefc90b000 4K ----- [ anon ]
00007faefc90c000 8188K rw--- [ anon ]
00007faefd10b000 4K ----- [ anon ]
00007faefd10c000 8188K rw--- [ anon ]
00007faefd90b000 4K ----- [ anon ]
00007faefd90c000 8188K rw--- [ anon ]
00007faefe10b000 4K ----- [ anon ]
00007faefe10c000 8188K rw--- [ anon ]
00007faefe90b000 4K ----- [ anon ]
00007faefe90c000 8188K rw--- [ anon ]
00007faeff10b000 4K ----- [ anon ]
00007faeff10c000 8188K rw--- [ anon ]
00007faeff90b000 4K ----- [ anon ]
00007faeff90c000 8188K rw--- [ anon ]
00007faf0010b000 4K ----- [ anon ]
00007faf0010c000 8188K rw--- [ anon ]
00007faf0090b000 4K ----- [ anon ]
00007faf0090c000 8188K rw--- [ anon ]
00007faf0110b000 4K ----- [ anon ]
00007faf0110c000 8188K rw--- [ anon ]
00007faf0190b000 4K ----- [ anon ]
00007faf0190c000 8188K rw--- [ anon ]
00007faf0210b000 4K ----- [ anon ]
00007faf0210c000 8188K rw--- [ anon ]
00007faf0290b000 4K ----- [ anon ]
00007faf0290c000 8188K rw--- [ anon ]
00007faf0310b000 4K ----- [ anon ]
00007faf0310c000 8188K rw--- [ anon ]
00007faf0390b000 4K ----- [ anon ]
00007faf0390c000 8188K rw--- [ anon ]
00007faf0410b000 4K ----- [ anon ]
00007faf0410c000 8188K rw--- [ anon ]
00007faf0490b000 4K ----- [ anon ]
00007faf0490c000 8188K rw--- [ anon ]
00007faf0510b000 4K ----- [ anon ]
00007faf0510c000 8188K rw--- [ anon ]
00007faf0590b000 4K ----- [ anon ]
00007faf0590c000 8188K rw--- [ anon ]
00007faf0610b000 4K ----- [ anon ]
00007faf0610c000 8188K rw--- [ anon ]
00007faf0690b000 4K ----- [ anon ]
00007faf0690c000 8188K rw--- [ anon ]
00007faf0710b000 4K ----- [ anon ]
00007faf0710c000 8188K rw--- [ anon ]
00007faf0790b000 4K ----- [ anon ]
00007faf0790c000 8188K rw--- [ anon ]
00007faf0810b000 4K ----- [ anon ]
00007faf0810c000 8188K rw--- [ anon ]
00007faf0890b000 4K ----- [ anon ]
00007faf0890c000 8188K rw--- [ anon ]
00007faf0910b000 4K ----- [ anon ]
00007faf0910c000 8188K rw--- [ anon ]
00007faf0990b000 4K ----- [ anon ]
00007faf0990c000 8188K rw--- [ anon ]
00007faf0a10b000 4K ----- [ anon ]
00007faf0a10c000 8188K rw--- [ anon ]
00007faf0a90b000 4K ----- [ anon ]
00007faf0a90c000 8188K rw--- [ anon ]
00007faf0b10b000 4K ----- [ anon ]
00007faf0b10c000 8188K rw--- [ anon ]
00007faf0b90b000 16384K rw--- [ anon ]
00007faf0ce01000 4K ----- [ anon ]
00007faf0ce02000 8188K rw--- [ anon ]
00007faf0d601000 4K ----- [ anon ]
00007faf0d602000 8188K rw--- [ anon ]
00007faf0de01000 4K ----- [ anon ]
00007faf0de02000 8188K rw--- [ anon ]
00007faf0e601000 11304K rw--- [ anon ]
00007faf0f10b000 4K ----- [ anon ]
00007faf0f10c000 8188K rw--- [ anon ]
00007faf0f90b000 4K ----- [ anon ]
00007faf0f90c000 8188K rw--- [ anon ]
00007faf1010b000 4K ----- [ anon ]
00007faf1010c000 8188K rw--- [ anon ]
00007faf1090b000 4K ----- [ anon ]
00007faf1090c000 8188K rw--- [ anon ]
00007faf1110b000 4K ----- [ anon ]
00007faf1110c000 8188K rw--- [ anon ]
00007faf1190b000 4K ----- [ anon ]
00007faf1190c000 8188K rw--- [ anon ]
00007faf1210b000 16384K rw--- [ anon ]
00007faf13451000 4K ----- [ anon ]
00007faf13452000 8188K rw--- [ anon ]
00007faf13c51000 4K ----- [ anon ]
00007faf13c52000 8188K rw--- [ anon ]
00007faf14451000 4K ----- [ anon ]
00007faf14452000 8188K rw--- [ anon ]
00007faf14c51000 4K ----- [ anon ]
00007faf14c52000 8188K rw--- [ anon ]
00007faf15451000 13032K rw--- [ anon ]
00007faf1610b000 4K ----- [ anon ]
00007faf1610c000 8188K rw--- [ anon ]
00007faf1690b000 4K ----- [ anon ]
00007faf1690c000 8188K rw--- [ anon ]
00007faf1710b000 4K ----- [ anon ]
00007faf1710c000 8188K rw--- [ anon ]
00007faf1790b000 4K ----- [ anon ]
00007faf1790c000 8188K rw--- [ anon ]
00007faf1810b000 4K ----- [ anon ]
00007faf1810c000 8188K rw--- [ anon ]
00007faf1890b000 4K ----- [ anon ]
00007faf1890c000 8188K rw--- [ anon ]
00007faf1910b000 4K ----- [ anon ]
00007faf1910c000 8188K rw--- [ anon ]
00007faf1990b000 4K ----- [ anon ]
00007faf1990c000 8188K rw--- [ anon ]
00007faf1a10b000 4K ----- [ anon ]
00007faf1a10c000 8188K rw--- [ anon ]
00007faf1a90b000 4K ----- [ anon ]
00007faf1a90c000 8188K rw--- [ anon ]
00007faf1b10b000 4K ----- [ anon ]
00007faf1b10c000 8188K rw--- [ anon ]
00007faf1b90b000 4K ----- [ anon ]
00007faf1b90c000 8188K rw--- [ anon ]
00007faf1c10b000 4K ----- [ anon ]
00007faf1c10c000 8188K rw--- [ anon ]
00007faf1c90b000 4K ----- [ anon ]
00007faf1c90c000 8188K rw--- [ anon ]
00007faf1d10b000 4K ----- [ anon ]
00007faf1d10c000 8188K rw--- [ anon ]
00007faf1d90b000 4K ----- [ anon ]
00007faf1d90c000 8188K rw--- [ anon ]
00007faf1e10b000 4K ----- [ anon ]
00007faf1e10c000 8188K rw--- [ anon ]
00007faf1e90b000 4K ----- [ anon ]
00007faf1e90c000 8188K rw--- [ anon ]
00007faf1f10b000 4K ----- [ anon ]
00007faf1f10c000 8188K rw--- [ anon ]
00007faf1f90b000 4K ----- [ anon ]
00007faf1f90c000 8188K rw--- [ anon ]
00007faf2010b000 4K ----- [ anon ]
00007faf2010c000 8188K rw--- [ anon ]
00007faf2090b000 4K ----- [ anon ]
00007faf2090c000 8188K rw--- [ anon ]
00007faf2110b000 4K ----- [ anon ]
00007faf2110c000 8188K rw--- [ anon ]
00007faf21984000 40476K rw--- [ anon ]
00007faf2410b000 2880K rw--- [ anon ]
00007faf243db000 4K ----- [ anon ]
00007faf243dc000 8188K rw--- [ anon ]
00007faf24bdb000 4K ----- [ anon ]
00007faf24bdc000 8188K rw--- [ anon ]
00007faf253db000 4K ----- [ anon ]
00007faf253dc000 8188K rw--- [ anon ]
00007faf25bdb000 4K ----- [ anon ]
00007faf25bdc000 8188K rw--- [ anon ]
00007faf263db000 4K ----- [ anon ]
00007faf263dc000 8188K rw--- [ anon ]
00007faf26bdb000 4K ----- [ anon ]
00007faf26bdc000 8188K rw--- [ anon ]
00007faf2764f000 1536K rw--- [ anon ]
00007faf277cf000 4K ----- [ anon ]
00007faf277d0000 8188K rw--- [ anon ]
00007faf27fcf000 6096K rw--- [ anon ]
00007faf285c3000 4K ----- [ anon ]
00007faf285c4000 8188K rw--- [ anon ]
00007faf28dc3000 16384K rw--- [ anon ]
00007faf29dc3000 4K ----- [ anon ]
00007faf29dc4000 8188K rw--- [ anon ]
00007faf2a5c3000 624K rw--- [ anon ]
00007faf2a65f000 4K ----- [ anon ]
00007faf2a660000 8188K rw--- [ anon ]
00007faf2ae5f000 4K ----- [ anon ]
00007faf2ae60000 8188K rw--- [ anon ]
00007faf2b65f000 256K rw--- [ anon ]
00007faf2bd9f000 512K rw--- [ anon ]
00007faf2c45f000 1536K rw--- [ anon ]
00007faf2c5df000 4K ----- [ anon ]
00007faf2c5e0000 8188K rw--- [ anon ]
00007faf2cddf000 256K rw--- [ anon ]
00007faf2ce1f000 4K ----- [ anon ]
00007faf2ce20000 8188K rw--- [ anon ]
00007faf2d61f000 4K ----- [ anon ]
00007faf2d620000 8188K rw--- [ anon ]
00007faf2de1f000 768K rw--- [ anon ]
00007faf2dedf000 12K r---- libnss_files-2.28.so
00007faf2dee2000 28K r-x-- libnss_files-2.28.so
00007faf2dee9000 8K r---- libnss_files-2.28.so
00007faf2deeb000 4K ----- libnss_files-2.28.so
00007faf2deec000 4K r---- libnss_files-2.28.so
00007faf2deed000 4K rw--- libnss_files-2.28.so
00007faf2deee000 24K rw--- [ anon ]
00007faf2df27000 5988K rw--- [ anon ]
00007faf2e519000 10348K rw--- [ anon ]
00007faf2ef34000 7936K rw--- [ anon ]
00007faf2f6f4000 4K ----- [ anon ]
00007faf2f6f5000 8188K rw--- [ anon ]
00007faf2fef4000 4K ----- [ anon ]
00007faf2fef5000 8188K rw--- [ anon ]
00007faf306f4000 4K ----- [ anon ]
00007faf306f5000 8188K rw--- [ anon ]
00007faf30ef4000 64K rw--- [ anon ]
00007faf30f04000 4K ----- [ anon ]
00007faf30f05000 8192K rw--- [ anon ]
00007faf31705000 4K ----- [ anon ]
00007faf31706000 8192K rw--- [ anon ]
00007faf31f06000 4K ----- [ anon ]
00007faf31f07000 9232K rw--- [ anon ]
00007faf3280b000 136K r---- libc-2.28.so
00007faf3282d000 1312K r-x-- libc-2.28.so
00007faf32975000 304K r---- libc-2.28.so
00007faf329c1000 4K ----- libc-2.28.so
00007faf329c2000 16K r---- libc-2.28.so
00007faf329c6000 8K rw--- libc-2.28.so
00007faf329c8000 16K rw--- [ anon ]
00007faf329cc000 12K r---- libgcc_s.so.1
00007faf329cf000 68K r-x-- libgcc_s.so.1
00007faf329e0000 12K r---- libgcc_s.so.1
00007faf329e3000 4K ----- libgcc_s.so.1
00007faf329e4000 4K r---- libgcc_s.so.1
00007faf329e5000 4K rw--- libgcc_s.so.1
00007faf329e6000 4K r---- libdl-2.28.so
00007faf329e7000 4K r-x-- libdl-2.28.so
00007faf329e8000 4K r---- libdl-2.28.so
00007faf329e9000 4K r---- libdl-2.28.so
00007faf329ea000 4K rw--- libdl-2.28.so
00007faf329eb000 8K r---- librt-2.28.so
00007faf329ed000 16K r-x-- librt-2.28.so
00007faf329f1000 8K r---- librt-2.28.so
00007faf329f3000 4K r---- librt-2.28.so
00007faf329f4000 4K rw--- librt-2.28.so
00007faf329f5000 8K rw--- [ anon ]
00007faf329f7000 336K r-x-- libevent-2.1.so.6.0.2
00007faf32a4b000 2044K ----- libevent-2.1.so.6.0.2
00007faf32c4a000 8K r---- libevent-2.1.so.6.0.2
00007faf32c4c000 4K rw--- libevent-2.1.so.6.0.2
00007faf32c4d000 24K r---- libpthread-2.28.so
00007faf32c53000 60K r-x-- libpthread-2.28.so
00007faf32c62000 24K r---- libpthread-2.28.so
00007faf32c68000 4K r---- libpthread-2.28.so
00007faf32c69000 4K rw--- libpthread-2.28.so
00007faf32c6a000 16K rw--- [ anon ]
00007faf32c6e000 52K r---- libm-2.28.so
00007faf32c7b000 636K r-x-- libm-2.28.so
00007faf32d1a000 852K r---- libm-2.28.so
00007faf32def000 4K r---- libm-2.28.so
00007faf32df0000 4K rw--- libm-2.28.so
00007faf32df1000 8K r---- libpcre.so.3.13.3
00007faf32df3000 328K r-x-- libpcre.so.3.13.3
00007faf32e45000 120K r---- libpcre.so.3.13.3
00007faf32e63000 4K r---- libpcre.so.3.13.3
00007faf32e64000 4K rw--- libpcre.so.3.13.3
00007faf32e65000 532K r---- libcrypto.so.1.1
00007faf32eea000 1656K r-x-- libcrypto.so.1.1
00007faf33088000 572K r---- libcrypto.so.1.1
00007faf33117000 4K ----- libcrypto.so.1.1
00007faf33118000 192K r---- libcrypto.so.1.1
00007faf33148000 8K rw--- libcrypto.so.1.1
00007faf3314a000 16K rw--- [ anon ]
00007faf3314e000 116K r---- libssl.so.1.1
00007faf3316b000 308K r-x-- libssl.so.1.1
00007faf331b8000 104K r---- libssl.so.1.1
00007faf331d2000 4K ----- libssl.so.1.1
00007faf331d3000 36K r---- libssl.so.1.1
00007faf331dc000 16K rw--- libssl.so.1.1
00007faf331e0000 8K rw--- [ anon ]
00007faf331e9000 4K r---- ld-2.28.so
00007faf331ea000 120K r-x-- ld-2.28.so
00007faf33208000 32K r---- ld-2.28.so
00007faf33210000 4K r---- ld-2.28.so
00007faf33211000 4K rw--- ld-2.28.so
00007faf33212000 4K rw--- [ anon ]
00007ffd5fe7a000 132K rw--- [ stack ]
00007ffd5ff3d000 12K r---- [ anon ]
00007ffd5ff40000 8K r-x-- [ anon ]
total 3538460K
Procinfo shows use of some 3.6gb of RAM
me@api-server-02:~$ cat /proc/861/status
Name: amqproxy
Umask: 0022
State: S (sleeping)
Tgid: 861
Ngid: 0
Pid: 861
PPid: 496
TracerPid: 0
Uid: 1013 1013 1013 1013
Gid: 1018 1018 1018 1018
FDSize: 512
Groups: 33 1017 1018
NStgid: 861
NSpid: 861
NSpgid: 861
NSsid: 496
VmPeak: 3620496 kB
VmSize: 3595920 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 394108 kB
VmRSS: 394060 kB
RssAnon: 387452 kB
RssFile: 6608 kB
RssShmem: 0 kB
VmData: 3583136 kB
VmStk: 132 kB
VmExe: 896 kB
VmLib: 4880 kB
VmPTE: 2412 kB
VmSwap: 0 kB
HugetlbPages: 0 kB
CoreDumping: 0
Threads: 4
SigQ: 0/31797
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001001000
SigCgt: 00000001a0814442
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
NoNewPrivs: 0
Seccomp: 0
Speculation_Store_Bypass: vulnerable
Cpus_allowed: f
Cpus_allowed_list: 0-3
Mems_allowed: 00000000,00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 24003617
nonvoluntary_ctxt_switches: 139748
We were consuming messages from RabbitMQ queues through the amqpproxy
.
We noticed that we had one specific queue which caused our consuming application to loose the connection to RabbitMQ.
After some analysing we found that the messages in that particular queue contained a timestamp in ms as shown in the screenshot.
We played around with the timestamp and the connection did not die if the timestamp was in seconds instead of milliseconds.
Guess it would be nice to not close the connection if the timestamp is in milliseconds.
This is the error message the amqpproxy
gave us:
amqp-proxy | Unhandled exception in spawn: Invalid time: seconds out of range (ArgumentError)
amqp-proxy | from ???
amqp-proxy | from ???
amqp-proxy | from ???
amqp-proxy | from ???
amqp-proxy | from ???
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.