xogroup / bunnybus Goto Github PK

Context

Add an option to new BunnyBus() to allow the subscribe() method to ignore header checks when pumping messages off of the queue.

Context

• node version: 8
• bunnybus version: latest
• environment (node, browser): node

What are you trying to achieve or the steps to reproduce ?

Calling bunnyBus.subscriptions.clearAll(); should close all connections. Currently the only way to stop the dangling connection is to call the internal methods. This dangling connection will cause test runners to hang and other weird/undesirable behavior.

Context

Rename the following header properties and fix the test to adhere as well.

• createAt -> createdAt
• errorAt -> erroredAt

Context

Create a BunnyBus identifier for messages originating from publish(), requeue(), send() and reject. This will enable the APIs for subscribe() and get() to be able to identify if messages should be processed as part of the BunnyBus ecosystem. Any messages without such identifier should be automatically ack()ed off or reject()ed to the error queue with some sort of header value signifying the reason.

The motivation behind this is to allow for multitenancy of a single Rabbit cluster between two different driver set. In the even vhost configuration was not designed correctly, messages may be sent and received between incompatible coupling.

In addition, this identifier should perhaps also contain a optional semver version identifier. There might be two possible take on this where we use the semver of the BunnyBus version based off of what exist in package.json or we also allow the end client to supply one through publish([options]) or send([options])

Test should provide proof against:

• identifier matching behavior
• identifier mistmatch behavior
• in cases of versioning, applying semver matching appropriately.

Currently it seems I can't add custom options when publishing/sending a message

One of the use case I'm thinking is expiration

Is that intended or a missing feature ?

Context

Make a version of the load test for the new Promise paths.

What are you trying to achieve or the steps to reproduce ?

Create equal coverage for both node callback and Promise code paths.

Context

The current connection object does not track amqp close events. This is a hole in error handling and needs to be mitigated.

Context

When the configuration does not contain value to instantiate a good connection, the runtime hangs.

What are you trying to achieve or the steps to reproduce ?

it.only('should not freak out', (done) => {

        // initialize with bad configs to trigger the catch
        let bunnyBus = new BunnyBus({
            server: 'fake'
        });

        return bunnyBus.publish({event: 'thing.event_happened', content: false})
            .catch(() => {

                // re-up with default configs
                bunnyBus = new BunnyBus();

                return bunnyBus.publish({event: 'thing.event_happened', content: true})
                    .then(() => {
                        expect(true).to.be.true();
                        done();
                    })
                    .catch(done);
            })
            .catch(done);
    })

What result did you expect ?

• Should pass

What result did you observe ?

• Times out the test

Context

Add documentation around error types within API.md.

Context

This is an issue tracker to help as a high level checklist for items needing to be accomplished.

TODO

• Setup build for Travis CI or CircleCI
  • Need docker build support to handle rabbitmq image
• Load Test
  • using callback
  • using Promise
• grep all TODO in code base and fix.
• adding Promise returns object when callback is not provided
• API documentation
  • TOC listing
  • Core API Documentation with Callback
  • Core API Documentation with Promise
• CONTRIB documentation
• TEST documentation
• API support for rabbit-killswitch support
  • exposing the subscription state management
• Logging support
  • inherit EventEmitter interface on to BunnyBus
    • create events for log.info, log.error, log.fatal, log.warn, log.debug
    • create log interface adapter for above events when a log contract is provided through log configuration.
    • define log contract + log configuration
• Optimization
  • Optimize for caching in Connection
  • Optimize for caching in Channel
  • Optimize for caching in Exchange
  • Optimize for caching in Queue

Context

• bunnybus version: 4.x.x and earlier

On calling reject, bunnybus actually calls an ack on the amqplib driver - https://github.com/xogroup/bunnybus/blob/master/lib/index.js#L654

This is bad both because it's inefficient and because it means deadlettering policies won't be correctly applied (and rabbit won't display rejects/deadletters correctly in stats).

Therefore we should instead create the error queue on first connection (like we do with regular queue) and correctly swap over to rejects.

Context

• node version: 8.9.4
• bunnybus version: 2.3.0
• environment (node, browser): Node
• any relevant modules used with (eslint, mocha, etc.): Sinon, Bluebird
• any other relevant information:

What are you trying to achieve or the steps to reproduce ?

1. In your tests, set a sinon spy on bunnyBus.publish like so: sinon.spy(bunnyBus, 'publish');
2. CallbunnyBus.publish and chain a Promise (the default native Promise or any particular library) onto it instead of a callback.
3. console.log(bunnyBus.publish.callCount) after the call and see that it is 2 instead of the expected 1. If you look at the args passed to the two spied calls, they are identical. The event is actually only published once but the spy leads you to believe that it was published twice.

What result did you expect ?

When you spy on bunnyBus.publish, call bunnyBus.publish once and return a Promise, then the spy should only see one call of bunnyBus.publish.

What result did you observe ?

When you spy on bunnyBus.publish, call bunnyBus.publish once and return a Promise, then the spy tracks two identical calls of bunnyBus.publish.

The problem seems to be happening here in the code. publish is called once with the original invocation, and then it is called again if no callback was provided. This results in the sinon spy accurately registering two calls when the rabbit event was only published once. If you are spying on bunnyBus.publish in your tests and expecting each call to represent an event that was published then this behavior is misleading.

Just rephrasing from fail to set to something along the lines of fail to override the EventEmitter to clarify that default logging behavior will not be broken on failed contract validation.

Context

For the subscribe() contract, the handlers currently contain a hash pair for routeKey/event assigned with a handler of signature (message, ack, reject, requeue) => {}. The routeKey is used to bind queue to the exchange for message routing. This routeKey can accept wild cards of the form * and #. When we create a handler and assign it a full wild card or partially wild card value, the binding takes place on RabbitMQ, but BunnyBus doesn't respect it correctly.

Example,

routeKey of value * is used. A messages comes through and lands in the queue with a concrete routeKey value of systemA.message-created. This message will not be routed to the handler because the value does not match what was registered. We need to put in place a better matching system than just checking hasProperty().

This is the offending LOC.

Context

We would like to be able to control BunnyBus like a remote for turn on and off subscriptions. There are times when it is useful to be able to shut down processing of a queue even when there are messages in the queue so that debugging can occur. On the same note, turning the process back on when necessary.

Context

• back fill a test for this patch.

Related to

• #53

Context

CLI Tool to help with the following

• Shovel messages from one queue to another
  • From error queue back to originating queue
  • Filter messages while shoveling with custom plugins
• Show # of messages in queue.
• Pipe messages in and out of RabbitMQ using tty stdio.
• Help with upgrade process for major BunnyBus versions

Examples of Possible Usage

• Make a selection from Postgres and pipe to bunnybus to publish
  • pgql 'select * from table' | bunnybus -c config.json publish --options.routeKey 'someKey'
• Subscribe using bunnybus and tee it to a file and also pipe back to bunnybus to publish
  • bunnybus -c config.json subscribe -q 'queue1' --confirmation ack | tee history.log | bunnybus -c config.json publish --options.routeKey 'someKey'
• Subscribe using bunnybus and tee it to peco for filtering redirected to a file.
  • bunnybus -c config.json subscribe -q 'queue1' --confirmation ack | peco > selected.log

Goal is to fork this off into a bunnybus-cli project.

Context

The existing handlers contract consist of (message, ack, reject, requeue) => {}. The message is the deserialized data from payload.content. But there maybe times when it is desirable to also ascertain the other meta information associated to the payload such has transactionId and retryCount.

I would like to allow for this feature when subscribe(queue, [options]) is called with an options.subscribeMeta set to true. When the condition is met, then the handlers contract can change to (message, meta, ack, reject, requeue) => {}.

meta will be a complex object where the scope meta.headers will contain all of the information contained within payload.properties.headers.

Context

The subscribe() interface does not properly identify the routeKey when only the message within the payload.content contains it.

What are you trying to achieve or the steps to reproduce ?

• Use send() with a payload like { event : 'a', prop1 : 'value1' }.
• Then subscribe() with a handler listening to a.

What result did you expect ?

The handler listening to a should get the message routed to it.

What result did you observe ?

BunnyBus swallows the message because the event value was not identified.

Context

Rename all private methods specified in the documentation to start with _. So it is clear to end users the separation of public and private methods through naming conventions.

Context

We want to allow for any type of A+ Promise based framework to be monkey patched into to our runtime. Most important, we want the ability to use bluebird. We also don't want to manage the this dependency internally and allow BunnyBus to be Promise framework agnostic while leverage the default ES6 Promise as a safe default.

Context

• node version: 10.15.3
• bunnybus version: 3.1.1
• environment (node, browser): node

I took note of issue 84 and the relevant pull request.

What are you trying to achieve or the steps to reproduce ?

I'm trying to send a message directly to the queue and make it persistent. See the following code fragment:

const sendOptions = { persistent: true };
this.bunnyBus.send(message, queueName, sendOptions)

What result did you expect ?

I expected the created message to be persistent.

What result did you observe ?

The message was created, but not persistent according to RabbitMQ. However, the following fragment of code did work as expected:

const publishOptions = { persistent: true };
this.bunnyBus.publish(message, publishOptions)

I've also tried to use deliveryMode: 2; but unfortunately this had the same result. Both messages (the sent and published ones) targeted the same, durable queue.

Context

Create a new method that takes in a JS Object that describes the rabbit topology and creates it.

Context

With the introduction of Node8 as LTS in October. It is time to shore up testing around await/async testing.

What are you trying to achieve or the steps to reproduce ?

We want to make sure async/awaitcalls have similar expected behavior as the raw Promise invoked ones.

Context

• node version: 8
• bunnybus version: latest
• environment (node, browser): node

What are you trying to achieve or the steps to reproduce ?

return bunnyBus.subscribe('hapi-reviews-reminder', {
        'reviews.webhook': function(message, ack, reject, requeue) {
            return myOtherHandler(message, ack, reject, requeue);
        }
});

In this case myOtherHandler will lock up and not ack(), reject(), requeue() or throw any errors.

What result did you expect ?

Either the docs should specify I need to rebind my scope, or bunnybus shouldn't depend on an explicit scope in message handlers.

Context

• node version:
• bunnybus version:
• environment (node, browser):
• any relevant modules used with (eslint, mocha, etc.):
• any other relevant information:

What are you trying to achieve or the steps to reproduce ?

• Update underlying dependencies to work with Node 16.

What result did you expect ?

Node 14 and 16 to be supported engines

What result did you observe ?

Only Node 10 and 12 are supported and trying to install bunnybus on projects using newer versions of Node yields warnings.

Context

• node version: 10.15.3
• bunnybus version: 3.1.1
• environment (node, browser): node
• We're using the RabbitMQ delayed message plugin

What are you trying to achieve or the steps to reproduce ?

I'm trying to publish a message to a RabbitMQ delayed message exchange (type 'x-delayed-message', as defined by the RabbitMQ delayed message plug-in), using BunnyBus. When I declare this exchange as 'globalExchange', I run into an error, because BunnyBus 'assumes' that this exchange is of type 'topic'. This is hardcoded in the _publish function.

It would be nice to make this 'globalExchangeType' configurable, or perhaps make the _publish function agnostic of the exchange type for existing exchanges.

What result did you observe ?

I got the following error message:

Operation failed: ExchangeDeclare; 406 (PRECONDITION-FAILED) with message "PRECONDITION_FAILED - inequivalent arg 'type' for exchange 'Bot' in vhost '/': received 'topic' but current is ''x-delayed-message''"

Remove npm-shrinkwrap.json.
Move commands from Make to npm scripts.
Any other unneeded artifacts.
Prepare for Travis CI / Circle CI build system.

Rearranging the hierarchy of the documentation for readability.
Add headers to separate BunnyBus getters/setters from methods.

Context

Create some documentation around examples. Should add examples into a new file named Example.md and linked from README.md

Context

Add logging calls where apprioriate.

Context

• Add component diagram
• Add usage diagram

Context

• bunnybus version: 4.2.1

Steps

• When the retryCount of a message is equal maxRetryCount, bunnybus will auto reject the message and put it into an error queue;
• The message in error queue, its headers.RetryCount will be equal to maxRetryCount https://github.com/xogroup/bunnybus/blob/master/lib/index.js#L644
• retryCount will affects message processing when consume the error queue.
  https://github.com/xogroup/bunnybus/blob/master/lib/index.js#L519

Expect

• Remove or rename retryCount when reject a message;

Context

When booting up new BunnyBus fresh, when publish() is called in parallel vigorously, connection handling does not mitigate parallel instantiation which is a race condition.

What result did you expect ?

Expected only a single connection to start up

What result did you observe ?

{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at exports._errnoException (util.js:1022:11)
    at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }

All bunnybus class methods in the public API should support ES6 native promises when no callback is provided.

The current integration tests written with the callback contract should be copied into a second file integration-promise.js and re-written to utilize the promise API.

Context

Right now trying to subscribe to an _error queue with bunnybus binds new events (causing regular events to flow directly into the _error queue), and creates _error_error queues which is undesirable behavior.

It would be nice to be able to disable this behavior with subscribe, or have the 'get' method work like subscribe.

This should also be captured in documentation.

What are you trying to achieve or the steps to reproduce ?

What result did you expect ?

What result did you observe ?