GithubHelp home page GithubHelp logo

mkfyi / nestjs-rmq Goto Github PK

View Code? Open in Web Editor NEW
1.0 1.0 0.0 406 KB

A decent NestJS module for a more advanced communication between microservices using RabbitMQ.

License: MIT License

JavaScript 1.15% Shell 0.15% TypeScript 98.69%
amqp javascript nest nest-framework nodejs rabbitmq typescript

nestjs-rmq's Introduction

Nest Logo

A decent module for a more advanced communication between microservices.

Package License publish

Description

This package provides a more advanced communication between microservices using the full power of RabbitMQ.

Core library features

  • Separate server/client components
  • Allowing multiple connections to one or more RabbitMQ server
  • Just implement the QueueHandler interface and mark the class with one of the following decorators
    1. @Listener() - Basic consumer, the simplest thing that does something
    2. @Worker() - Work Queues, distributing tasks among workers
    3. @PubSub() - Publish/Subscribe, sending messages to many consumers at once
    4. @Routing() - Routing, receiving messages selectively
    5. @Topics() - Topics, receiving messages based on a pattern
    6. @Rpc() - RPC, Request/reply pattern
  • Optional validation of message content using class-validator
  • Lightweight wrapper of the amqplib for the Nest ecosystem

Installation

nest-rmq must be integrated into the ecosystem of Nest, so your application must require @nestjs/common and @nestjs/core. You can replace all npm commands with the package manager of your choice. So if you would like to use yarn, replace npm with yarn.

# from official npm registry
$ npm i --save @mkfyi/nestjs-rmq

# using yarn
$ yarn add @mkfyi/nestjs-rmq

# from GitHub package registry
$ npm i --save --registry=https://npm.pkg.github.com @mkfyi/nestjs-rmq

# from GitHub package registry using yarn
$ yarn add --registry=https://npm.pkg.github.com @mkfyi/nestjs-rmq

Since nest-rmq is built on top of amqplib you also need to install the types for it.

$ npm install -D @types/amqplib

Usage

Import the RabbitMQModule from @mkfyi/nestjs-rmq and call the forRoot() method inside the imports of your application module. You can also set a custom name for the connection, otherwise default will be used.

Initialization

import { RabbitMQModule } from '@mkfyi/nestjs-rmq';

@Module({
  imports: [
    // ...
    RabbitMQModule.forRoot({
      connection: {
        hostname: '',
        username: '',
        password: '',
      },
    }),
  ],
})
export class AppModule {}

If you prefer to use environment variables, consider adding the @nestjs/config and use forRootAsync() method instead.

@Module({
  imports: [
    // ...
    RabbitMQModule.forRootAsync({
      connection: {
        imports: [ConfigModule],
        useFactory: (config: ConfigService) => ({
          hostname: config.get('AMQP_HOSTNAME'),
          username: config.get('AMQP_USERNAME'),
          password: config.get('AMQP_PASSWORD'),
        }),
        inject: [ConfigService],
      },
    }),
  ],
})
export class AppModule {}
Multiple connections

You can also create multiple connections, just pass the object as above into an array and add the name property. This name is being used for the QueueHandler and QueueAdapter.

@Module({
  imports: [
    // ...
    RabbitMQModule.forRootAsync({
      connection: [
        {
          name: 'default',
          imports: [ConfigModule],
          useFactory: (config: ConfigService) => ({
            hostname: config.get('AMQP_HOSTNAME'),
            username: config.get('AMQP_USERNAME'),
            password: config.get('AMQP_PASSWORD'),
          }),
          inject: [ConfigService],
        },
        {
          name: 'stage',
          imports: [ConfigModule],
          useFactory: (config: ConfigService) => ({
            hostname: config.get('AMQP_STAGE_HOSTNAME'),
            username: config.get('AMQP_STAGE_USERNAME'),
            password: config.get('AMQP_STAGE_PASSWORD'),
          }),
          inject: [ConfigService],
        }
      ],
    }),
  ],
})
export class AppModule {}
Adapters (client)

You have to configure the adapters properts in order send messages to the respective queue.

@Module({
  imports: [
    // ...
    RabbitMQModule.forRootAsync({
      connection: {
        name: 'default',
        imports: [ConfigModule],
        useFactory: (config: ConfigService) => ({
          hostname: config.get('AMQP_HOSTNAME'),
          username: config.get('AMQP_USERNAME'),
          password: config.get('AMQP_PASSWORD'),
        }),
        inject: [ConfigService],
      },
      adapters: [
        {
          name: 'BACKEND_SERVICE',
          queue: 'example.worker,
          type: QueueAdapterType.Worker,
          connection: 'default',
        },
      ],
    }),
  ],
})
export class AppModule {}

The example shown above creates an adapter named BACKEND_SERVICE for the default connection. The value of the name property can be injected as QueueAdapter using @Inject(). You may want to change to RpcQueueAdapter for this type.

@Injectable()
export class MyService {
  public constructor(
    @Inject('BACKEND_SERVICE')
    private readonly worker: QueueAdapter,
  ) {}

  public notifyUsernameUpdate(id: string, name: string) {
    return this.worker.send({ id, name });
  }
}
Handlers (server)

Every custom queue handler has to implement the QueueHandler interface. As for the adapters, there is a separate interface for RPC based handlers called RpcQueueHandler.

@Worker({ queue: 'example.worker' })
export class ExampleWorkerQueueHandler implements QueueHandler {
  public async execute(msg: Message): Promise<void> {
    console.log(msg.object());
  }
}

Don't forget to add your queue handlers to the application module providers.

@Module({
  imports: [
    // ...
    RabbitMQModule.forRootAsync({
      // ...
    }),
  ],
  providers: [
    // ...
    ExampleWorkerQueueHandler,
  ],
})
export class AppModule {}

License

nest-rmq is MIT licensed.

nestjs-rmq's People

Contributors

mkfyi avatar

Stargazers

 avatar

Watchers

 avatar

nestjs-rmq's Issues

TypeError when throwing an exception during RPC requests

TypeScript throws TypeError: Cannot destructure property 'error' of 'this.object(...)' as it is undefined. when you throw an exception inside the execute() method of a RpcQueueHandler.

/home/dev/Desktop/test-app/node_modules/@mkfyi/nestjs-rmq/dist/core/wrappers/answer.wrapper.js:9
            const { error, message } = this.object();
                    ^
TypeError: Cannot destructure property 'error' of 'this.object(...)' as it is undefined.
    at new AnswerWrapper (/home/dev/Desktop/test-app/node_modules/@mkfyi/nestjs-rmq/dist/core/wrappers/answer.wrapper.js:9:21)
    at channel.consume.noAck (/home/dev/Desktop/test-app/node_modules/@mkfyi/nestjs-rmq/dist/core/adapter/rpc.queue-adapter.js:19:36)
    at Channel.BaseChannel.dispatchMessage (/home/dev/Desktop/test-app/node_modules/amqplib/lib/channel.js:483:12)
    at Channel.BaseChannel.handleDelivery (/home/dev/Desktop/test-app/node_modules/amqplib/lib/channel.js:492:15)
    at Channel.emit (node:events:511:28)
    at /home/dev/Desktop/test-app/node_modules/amqplib/lib/channel.js:278:10
    at Channel.content [as handleMessage] (/home/dev/Desktop/test-app/node_modules/amqplib/lib/channel.js:331:9)
    at Channel.C.acceptMessageFrame (/home/dev/Desktop/test-app/node_modules/amqplib/lib/channel.js:246:31)
    at Channel.C.accept (/home/dev/Desktop/test-app/node_modules/amqplib/lib/channel.js:399:17)
    at Connection.mainAccept [as accept] (/home/dev/Desktop/test-app/node_modules/amqplib/lib/connection.js:63:33)

On the server side you'll receive an undefined:

[Nest] 42391  - 04/06/2023, 17:54:36   ERROR [RabbitMQ] received undefined in example.rpc queue: Example Message

Allow different return types for RPC based queue handlers

The current implementation of the RpcQueueHandler<T> returns Reply<T> from execute() which means, that currently any implementation of RpcQueueHandler must satisfy Reply.

It would be better if the implementation would look something like:

export type RpcReturnTypes =
  | boolean
  | number
  | string
  | Uint8Array
  | Record<string, unknown>;

export type RpcQueueHandler<T extends RpcReturnTypes = RpcReturnTypes> =
  QueueHandler<T>;

This change must also be applied to the RpcQueueManager, which is just a small change.

- export class RpcQueueManager extends BaseQueueManager<Reply> {
+ export class RpcQueueManager extends BaseQueueManager<RpcReturnTypes> {

  protected async bind(
    channel: Channel,
-    handler: QueueHandler<Reply>,
+    handler: QueueHandler<RpcReturnTypes>,
    metadata: QueueHandlerMetadata,
  ): Promise<void> {
    await channel.assertQueue(metadata.queue, { durable: false });
    await channel.prefetch(1);
    await channel.consume(metadata.queue, (msg) => {
      if (msg !== null) {
        const message = new MessageWrapper<ConsumeMessageFields>(msg);

        handler
          .execute(message)
-         .then((reply) => this.reply(channel, message, reply))
+         .then((reply) => this.reply(channel, message, { payload: reply }))
          .catch((e) =>
            this.reply(
              channel,
              message,
              this.exceptionHandler.handle(e, metadata.queue),
            ),
          )
          .finally(() => channel.ack(msg));
      }
    });

Unable to send Date type

When sending data which doesn't match instanceof Uint8Array, JSON.stringify is used to ensure that anything that isn't a string will be serialized. Since node.js internally calls toISOString() for the Date type during JSON.stringify, each consumer will always receive the property as string.

A custom service that wraps JSON.stringify and JSON.parse with a configurable parser (?) property inside BaseModuleOptions could resolve this. The configurable property must be injected into the new service.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.