golevelup / nestjs Goto Github PK

Hello,
thanks for this library! I was wondering how it is possible write (i.e. publish) on the message bus.
Using the pubSubHandler I can receive the messages, but it is not clear to me how to write messages on an exchange.

Thanks, best wishes
Thomas

Hey, thanks for the awesome package. I updated to the current version, and now when I deploy and sometimes running locally I get:

Error: Failed to connect to a RabbitMQ broker within a timeout of 5000ms
   at AmqpConnection.init (/app/node_modules/@golevelup/nestjs-rabbitmq/lib/amqp/connection.js:59:99)

It'll happen a few times when deploying, those containers will fail, then a new one will spin up and eventually it will work. Any idea what's happening? I've confirmed that Rabbit is running.

Currently the DiscoveryService provides a convenient API for being able to scan through a NestJS app's dependencies to find matching Controllers/Providers and their method handlers. There are situations where it would be useful to also surface information about the module that these items were discovered inside of.

As part of this change we should also reduce the amount of information being returned from Discovery such that it doesn't expose all the innards of the InstanceWrapper. The API should return:

• handlers
• controllers/providers (both the constructed instance from the DI container and the Constructor Function currently called metatype)
• reflect metadata that was searched for
• the name of the controller/provider

Hey - thanks for working on this! I've installed per the readme and when I run the service, I get this error. Any ideas what might be causing it?

Thank you!

I have the latest version of these fantastic RabbitMQ-Module. But i can not use the forRoot method like documented in the README. The function parameters look unnormal, why are there two parameters? What is the moduleCtor?

(method) IConfigurableDynamicRootModule<RabbitMQModule, RabbitMQConfig>.forRoot(moduleCtor: Type<RabbitMQModule>, moduleConfig: RabbitMQConfig): DynamicModule Expected 2 arguments, but got 1.

My example implementation like in README:

@Module({ imports: [ RabbitMQModule.forRoot({ exchanges: [ { name: 'exchange1', type: 'topic' } ], uri: 'amqp://rabbitmq:rabbitmq@localhost:5672', connectionInitOptions: { wait: false } }) ], providers: [EventService] }) export class EventModule { }

Hi,
We use @nestjs-plus/rabbitmq to communicate with rabbitmq.
If we use it directly in a given service (with the implementation associated with this level), it is functional.
But we want to have a RabbitModule module that would contain the services to publish or receive the messages and use this service in other modules. But we face either errors of transpiration or errors where the service is not recognized.

So, we wonder if this feature is feasible with @nestjs-plus/rabbitmq and if so, what are we doing wrong?

rabbitmq.service.ts

import { Injectable } from '@nestjs/common';
import { AmqpConnection } from '@nestjs-plus/rabbitmq';

@Injectable()
export class RabbitmqClientService {
  constructor(private readonly amqpConnection: AmqpConnection) {}

  public async publishMessage(exchange: string, subscribeRoute: string, message: {}) {
    await this.amqpConnection.publish(exchange, subscribeRoute, {
      message,
    });
    return {
      result: 'Published message',
    };
  }
}

rabbit.module.ts :

import { Module } from '@nestjs/common';
import { RabbitMQModule } from '@nestjs-plus/rabbitmq';
import { AmqpConnection } from '@nestjs-plus/rabbitmq';

import { RabbitmqClientService } from './services/rabbitmq.service';
import { RabbitmqController } from './controllers/rabbitmq.controller';

@Module({
  imports: [
    RabbitMQModule.forRootAsync({
      useFactory: () => ({
        exchanges: [
          {
            name: 'exchange1',
            type: 'topic',
          },
          {
            name: 'exchange2',
            type: 'fanout',
          },
        ],
        uri: 'amqp://rabbitmq:rabbitmq@localhost:5672',
      }),
    }),
    AmqpConnection
  ],
  controllers: [RabbitmqController],
  providers: [AmqpConnection, RabbitmqClientService],
  exports: [RabbitmqClientService]
})
export class RabbitModule {}

PodcastController.ts :

(...)
 public constructor(
    private readonly podcastService: PodcastService,
    private readonly rabbitmqClientService: RabbitmqClientService,
  ) {}
(...)
})
  public async getListPodcast(): Promise<Redaction[]> {
    const queue: QueueDto = {
      name: 'queue_test',
    };
    const exchange =  'exchange_test';
    const podcasts = await this.podcastService.findAll();
    const message: RabbitmqEvent = {
      eventName: EventName.PodcastListWasRequested,
      objectId: uuid(),
      objectVersion: 1,
      commandPayload: {
        podcasts,
      },
      eventCreatedBy: 'fdf9859f-6f51-42b0-b7a5-b2c9f697cad6',
      eventCreatedAt: Date.now(),
    };
    await this.rabbitmqClientService.publishMessage(
      queue.name,
      exchange,
      message
    );

    return podcasts;
  }

we have this error

 Nest can't resolve dependencies of the RabbitmqClientService (?). Please make sure that the argument AmqpConnection at index [0] is available in the PodcastModule context.

Potential solutions:
- If AmqpConnection is a provider, is it part of the current PodcastModule?
- If AmqpConnection is exported from a separate @Module, is that module imported within PodcastModule?
  @Module({
    imports: [ /* the Module containing AmqpConnection */ ]
  })
 +1ms

if we modify the module rabbit.module.ts as well :

controllers: [RabbitmqController],
providers: [AmqpConnection, RabbitmqClientService],
exports: [RabbitmqClientService, AmqpConnection],

and podcast.module.ts :

controllers: [PodcastController],
providers: [PodcastService, RabbitmqClientService, AmqpConnection, ...queryHandler],
exports: [RabbitmqClientService],

We do not have a compilation problem, however, when we call the getListPodcast method (in REST with a GET), we have this error:

TypeError: Cannot read property 'publish' of undefined

It's no longer working with nest version 6 due to the REQUEST

Attempting to create a multi-queue listener with the following code

@Module({
    imports: [RabbitMQModule.forRoot(RabbitMQModule, { "uri": "amqp://guest:guest@localhost:5672" })],
    providers: [WorkerService],
    exports: [
        RabbitMQModule,
        WorkerService
    ]
})
export class WorkerModule {}

@Injectable()
export class WorkerService {
    constructor(private queue: AmqpConnection) {}

    @RabbitSubscribe({
        exchange: '',
        routingKey: '',
        queue: 'test:first'
    })
    public async handleTestFirst(data: {}): Promise<void> {
        console.log('Received test:first message', data);
        this.queue.publish('', 'test:second', data);
    }

    @RabbitSubscribe({
        exchange: '',
        routingKey: '',
        queue: 'test:second'
    })
    public async handleTestSecond(data: {}): Promise<void> {
        console.log('Received test:second message', data);
    }
}

Every time I run the app I get the error Error: Operation failed: QueueBind; 403 (ACCESS-REFUSED) with message "ACCESS_REFUSED - operation not permitted on the default exchange", what causes the error to occur?

I'd prefer to remain on the default exchange so I don't have to alter an existing RabbitMQ instance too much, it will be easier if I can use the default exchange and port some code to NestJS

How to reconnect rabbitmq when heartbeat timeout?

• Integrate with Semantic Commits
• Include Changelogs
• Investigate automatically publishing from CI/CD (Travis) on commits to master

Reference:
https://medium.com/@kevinkreuzer/the-way-to-fully-automated-releases-in-open-source-projects-44c015f38fd6

Testing in the monorepo works fine but is complaining about processing js files.

ts-jest[jest-transformer] (WARN) Got a `.js` file to compile while `allowJs` option is not set to `true` (file: /home/jesse/code/nestjs-plus/packages/common/lib/index.js). To fix this:
  - if you want TypeScript to process JS files, set `allowJs` to `true` in your TypeScript config (usually tsconfig.json)
  - if you do not want TypeScript to process your `.js` files, in your Jest config change the `transform` key which value is `ts-jest` so that it does not match `.js` files anymore

allowJs is turned on already in tsconfig.spec.json so something strange is going on

In a publish/subscribe would it be possible to nack the message if handling fails? or do we have requeue it manually?

If (inadvertently) a provider has a null instance, the module will crash.

This can be fixed by filtering, before passing for processing:

const providers = (await this.providers(providerFilter)).filter(it => it.instance != undefined);

Example scenario:

I had a config module (@Inject('config.SENDGRID_API_KEY')) that returned null for a given value. Not intended as the config is required, however if the config was optional, this would be a show stopper.

The RabbitMQ module should be built such that it plays nicely with the existing Nest processing pipeline as far as the ability to integrate with other Nest components such as Pipes, Interceptors and Guards. The @nestjs/microservices package accomplishes this but it doesn't appear that it exposes the necessary functionality to glue these things together in an easy to use way. The current approach of using ExternalContextCreator to build up the handler function doesn't appear to be working currently.

RabbitMQ handlers (for both PubSub and RPC) should:

• Work with Pipes
• Work with Interceptors
• Work with Guards

Hi. I run your demo. the controller publish message to rabbitqm by amqpconetion. the service can't receive the message. how could i do?

hi, thanks for this library.
I want to use AmqpConnection for call rpc. I can use publish function for broadcast messages,but for RPC I don't know how to do that. can you explain more about send messages

Good morning.

Is the google pubsub project active?

Tks.

Sorry for the multiple issues! Got everything working and I see the exchanges are created. Is there a process to create queue bindings or should that be implemented manually in the app?

Thank you!

It would be great to have actual documentation for packages

The solution is to just install it, but figured either it should be a dep in this package's package.json file or specify in the docs to also install this module.

(originally posted this as a secondary issue in #58)

The following items should be added to the QueryParams:

interface AssertQueue { exclusive?: boolean; durable?: boolean; autoDelete?: boolean; arguments?: any; messageTtl?: number; expires?: number; deadLetterExchange?: string; deadLetterRoutingKey?: string; maxLength?: number; maxPriority?: number; }

Currently the DiscoveryService contains a lot of code paths that will cause it to Reflect metadata from components multiple times. Adding an internal caching layer that extracts all meta once at initialization opens up some cool performance optimizations as well as potential improvements for the public API to allow extracting multiple meta values in one call

Hi @WonderPanda,

According to docs, a handler is available within DiscoveredMethod:

export interface DiscoveredMethod {
  handler: (...args: any[]) => any;
  methodName: string;
  parentClass: DiscoveredClass;
}

But how we can intercept the actual method's call to access args values? The same way plain decorators allow us to do.

Hey,

It could be nice to pass a filter function for the metaKey in order to be able to get all the target which respect the result of that function.

For example:

AND

await discoveryService.providersWithMetaAtKey(metaKeys => !difference(['key1', 'key2'], metaKeys).length);

OR

await discoveryService.providersWithMetaAtKey(metaKeys => metaKeys.some(k => k === 'key1' || k === 'key2'));

what do you think about that ?

Hi!

Have found such a strange issue: instance properties of discovered controllers are undefined in discovery results.

I think that it can happen because controllers discovery is in the discovery service constructor, so it should be delayed at least to the finish of the app bootstraping process. Moreover, discovery process could be implemented in lazy way.

I'll prepare PR for that shortly.

Thanks.

Hey, sorry if I'm misunderstanding how Rabbit is supposed to work. Right now we're creating an exchange for each event and that should route to one or more service queues. So CreateUserEvent routes to user-service-queue and logger-service-queue. In our Nest module, we're listening to that event. If I create a listener and specify the exchange and the queue, the underlying function ends up being triggered on every message that comes into the queue, even if it didn't come from that exchange. If I omit the queue in the listener decorator, that specific function will create its own default queue.

Is there a way to only use the 4 main service queues while listening for messages which originated from CreateUserEvent and were routed to user-service-queue? Am I missing something / doing something dumb here?

AmqpConnection is a general purpose Typescript client for interacting with AMQP. It should live in it's own repository and be published to NPM outside of the @nestjs-plus organization so that it can be easily consumed by other projects

Hi,

I am evaluating solutions for using an amqp broker with Nest.js and your project looks very promising.

I tried the pub/sub feature by adapting your kitchen-sink example and it works well, but I noticed that the timeout Exception is thrown on every message:

UnhandledPromiseRejectionWarning: Error: Failed to receive response within timeout of 10000ms
    at MapSubscriber.rxjs_1.interval.pipe.operators_1.map.x [as project] (/Users/mkrone/Development/nestjs-plus/examples/kitchen-sink/node_modules/@nestjs-plus/rabbitmq/src/amqp/AmqpConnection.ts:79:15)

Maybe the use of rxjs timeout or takeUntil operators can fix this in packages/rabbitmq/src/amqp/AmqpConnection.ts?

Let me know if I can help out with this.

Is it possible to see what scope the parent provider/controller of the discovered method is?

Hi.
It would be great and more more reliable if there was a way to use amqplib.ConfirmChannel in RabbitMQ.

the global interceptor handle result of http reponse .but message from rabbitmq don't get response object ,it will occur TypeError. meanwhile，rabbitmqm will doesn't get ack. the consumer wiil alway get the message

Hi. I'm trying to implement custom message parsing and serializing feature for RabbitMQ module because I would like to exchange messages in XML format.

So far I have made necessary changes and trying to launch integration/rabbitmq example, but it doesn't have @golevelup/nestjs-rabbitmq dependency in package.json and it's also using forRootAsync method for module setup which doesn't exists.

I assume there is minor inconsistency between packages and I would like to create PR to fix it. Could you provide additional information about this?

When starting an application that is using the DiscoveryService (e.g. by importing the RabbitMQModule or using the discovery package directly), and using a Provider that is not provided in a way so that a classType is available, the application fails at startup with the following exception:

TypeError
    at Object.getMetadata (/dev/reflect-metadata/node_modules/@nestjs/core/node_modules/reflect-metadata/Reflect.js:354:23)
    at component (/dev/reflect-metadata/node_modules/@nestjs-plus/discovery/lib/discovery.service.js:30:53)
    at filter.x (/dev/reflect-metadata/node_modules/@nestjs-plus/discovery/lib/discovery.service.js:52:65)
    at Array.filter (<anonymous>)
    at DiscoveryService.<anonymous> (/dev/reflect-metadata/node_modules/@nestjs-plus/discovery/lib/discovery.service.js:52:53)

This is caused by the withMetaAtKey function, which calls the ReflectMetadata.getMetadata method which will throw a TypeError if the passed target is not of type object.
In my case, importing the TypeOrmModule causes the method to throw, because its TypeOrmModuleOptions has a classType of null.

{ name: 'TypeOrmModuleOptions',
  instance:
   { ... },
  classType: null,
  parentModule:
   { name: 'TypeOrmCoreModule',
     instance:
      ...

As a workaround I wrap the calls to getMetadata in try/catch which works in my case but may not be an ideal solution: https://github.com/michaelkrone/nestjs-plus/blob/master/packages/discovery/src/discovery.service.ts#L24

As title. i want to config param by file.

As we continue to build out packages that integrate with third party systems and frameworks (eg RabbitMQ, Caching, etc) it's important to ensure that a proper integration test pipeline exists that actually utilizes nestjs-plus packages in a real environment. This will help to ensure that functionality works as intended and that there are no regressions as the codebase evolves.

This pipeline should be integrated with Docker Compose and be capable of running on Travis (or an equivalent CI) platform as part of normal builds and before new packages are released.

Hi,
It is a really nice lib. Thank you!

Is it possible to use multiple connections to send/receive messages? If so I'd appreeciate it isf someone give an example of configuration and using.

Need to expose PublishOptions to Request so that you can modify them if necessary. It's an optional parameter so it should not be a breaking change.

It would be awesome if there was a way to provide multiple injectable values from a single module. I'm thinking something like

createConfigurableDynamicRootModule<T, U>(moduleConfigToken: [InjectionTokens] | InjectionToken, moduleProperties: Partial<
    Pick<ModuleMetadata, 'imports' | 'exports' | 'providers'>
  > = {
    imports: [],
    exports: [],
    providers: []
  }
)

Where you could provide multiple tokens. Maybe create a third parameter for a token map that tells the system how to create the relationship between the tokens and their configs. Say if multiple tokens were provided like SERVICE_OPTIONS and INTERCEPTOR_OPTIONS then there could be an overall options options passed to forRoot or returned from forRootAsync that could look like

{
  service: myServiceOptions,
  interceptor: myInterceptorOptions,
}

and a map (object) that looks like

{
  SERVICE_OPTIONS: service,
  INTERCEPTOR_OPTIONS: interceptor
}

(or maybe reversed) so that when in a service we have @Inject(SERVICE_OPTIONS) we get the value myServiceOptions. What do you think? Let me know if you need more insight.

For caching, I created: https://www.npmjs.com/package/typescript-cacheable
^-- Take a look, perhaps it can help?

• Similar to RabbitMQ Module, supply decorators to glue service methods to subscription handlers
• Investigate providing an awaitable client that allows for simple responses over PubSub based on correlationId to simulate RPC functionality for requests. Can reuse code from the AmqpConnection If this ends up being too opinionated of an implementation at least provide building blocks that it could be set up on a per app basis
• Investigate spinning up and tearing down GCP PubSub infrastructure as part of integration testing

Currently the createSubscriber method in AmqpConnectionwhich is used by theRpcSubscribedecorator attempts to assert the exchange before subscribing. As part of this it's passing through a default exchangeType value fortopic`.

This results in errors at startups exchange types other than topic:

RabbitMQModule.build({
      exchanges: [
        {
          name: 'exchange1',
          type: 'topic',
        },
        {
          name: 'exchange2',
          type: 'fanout',
        },
      ],
      uri: 'amqp://rabbitmq:rabbitmq@localhost:5672',
    })

produces the error:

Error: Channel closed by server: 406 (PRECONDITION-FAILED) with message "PRECONDITION_FAILED - inequivalent arg 'type' for exchange 'exchange2' in vhost '/': received 'topic' but current is 'fanout'"

This should be handled by the module init based on the exchanges provided to the RabbitMQModule so exchanged can be asserted once (with proper values) instead of allowing it to fall through to individual handlers.

Libraries that choose to consume AmqpConnection for use cases outside of the Decorator patterns exposed by the module will be responsible for doing their own exchange assertion.

Hi, i have few questions regarding the install / usage of the package :
Can you add to your documentation an example of the main.ts file please ?
Do we need @nestjs/microservice package ?

In fact i don't how to start, a full example would be really appreciate (or at least the minimum dependecies & main.ts example).

Now that Nest 6 is available, the @nestjs-plus packages need to be updated to support it. This will entail a new major version release for all current packages which means that version 5 support will no longer receive new functionality (to avoid semver issues).

The discovery package is the biggest blocker now as it depends on NestJS DI internals which have been updated significantly.

So I tried

    RabbitMQModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        exchanges: [
          {
            name: config.rmqExchange,
            type: 'topic'
          }
        ],
        uri: config.rmqUrl
      })
    })

and at first I got the following error, which was odd because I'm pretty sure this is a bug?

So then i installed @nestjs-plus/common and then i got what looks like an implementation error on my part:

I'm basically trying to pass in a config from another module (my ConfigModule which exposes a ConfigService) Any thoughts?

Hi,
It is a really nice module but it is missing connection retry on init or on error.
Are you interested in implementing this feature?

Hi. Is't possible to bind queue to multiple routing keys from exchange.

example:
Have exchange someExchange with type topic
Have queue someQueue, and i would like someExchange will send messages to someQueue with routing keys ['some.#', '*.another'].

Code below bind only latest routing key at the array

export const RabbitSubscribeAndBindQueue = ({name: queue, binds}: IRabbitQueueConfig) => (
  target: any,
  propertyKey: string,
  descriptor: PropertyDescriptor,
) => {
  binds.forEach(({exchange, routingKey}) => {
    RabbitSubscribe({
      exchange,
      queue,
      routingKey,
    })(target, propertyKey, descriptor);
  });
};

Update the RabbitMQModule builder methods to use the forRoot and forRootAsync to stay with the conventions of the existing NestJS ecosystem. Investigate a better way to allow for injection of a config service to allow people to pull the necessary AMQP config information from their existing tools for config management.

Hi, I'm seeing some test failures when running npm test in the kitchen-sink example directory.

I've added these dependencies to make the project run:

  "dependencies": {
    "@nestjs-plus/common": "^1.1.0",
    "@nestjs-plus/rabbitmq": "^1.4.1",

Test results:

 PASS  src/app.controller.spec.ts
 PASS  src/rabbit-example/messaging/messaging.service.spec.ts (5.057s)
 FAIL  src/rabbit-example/messaging/messaging.controller.spec.ts (5.084s)
  ● Messaging Controller › should be defined

    Nest can't resolve dependencies of the MessagingController (?). Please make sure that the argument AmqpConnection at index [0] is available in the _RootTestModule context.

    Potential solutions:
    - If AmqpConnection is a provider, is it part of the current _RootTestModule?
    - If AmqpConnection is exported from a separate @Module, is that module imported within _RootTestModule?
      @Module({
        imports: [ /* the Module containing AmqpConnection */ ]
      })

      at Injector.lookupComponentInExports (../node_modules/@nestjs/core/injector/injector.js:183:19)

  ● Messaging Controller › should be defined

    expect(received).toBeDefined()

    Received: undefined

      14 | 
      15 |   it('should be defined', () => {
    > 16 |     expect(controller).toBeDefined();
         |                        ^
      17 |   });
      18 | });
      19 | 

      at Object.it (rabbit-example/messaging/messaging.controller.spec.ts:16:24)

Test Suites: 1 failed, 2 passed, 3 total
Tests:       1 failed, 2 passed, 3 total
Snapshots:   0 total
Time:        5.505s
Ran all test suites.

Let me know how if there's a way to fix the tests?

Thanks,
Patrick

Hi, I'm trying to use the DiscoveryService in my GraphqlService inside createGqlOptions.

async createGqlOptions(): Promise<GqlModuleOptions> {
        const costMap = await this.discoveryService
            .providersWithMetaAtKey(COST_MAP_METADATA);
        console.log(costMap); // never executed
        // ...
}

but the console.log is never executed as the promise is never resolved.

If I change it to this:

async createGqlOptions(): Promise<GqlModuleOptions> {
        const costMap = this.discoveryService
            .providersWithMetaAtKey(COST_MAP_METADATA).then(console.log);
        // ...
}

The promise resolves, but if I wrap everything inside the then it again does not resolve. Any idea why?

I have many instances of my application running, some of which I'd like to consume from RabbitMQ queues and others that should be dedicated to serving API requests.

I'm using the RabbitSubscribe decorator on a service method to connect a consumer.

@RabbitSubscribe({
    exchange: 'exampleExchange.created',
    routingKey: 'exampleExchange.*',
    queue: 'exampleExchange.created',
  })
  public async process(event): Promise<void> {
    ...
  }

Is there a way to stop this from consuming from the queue based on my application config, while allowing the application to publish to the same exchanges/queues?