Global concurrency limit about bull

[mgpai22]

Thank you, this is working well for me! Will still leave this issue up because I think the ability to do this globally will be useful.

[mgpai22]

Here is the concurrency documentation on the bullMQ side: https://docs.bullmq.io/guide/workers/concurrency

[mckrava]

I have similar issue but docs above provided me not "optimistic" info. So could you check correctness of my understanding:
So I have 1 queue and named jobs in this queue (create_post, update_post, follow_post, etc.). These events must be processed in the same order as they were added to the queue (it's required) and parallel processing is not allowed. So here is my @processor class:

@Processor('SOCIAL_ENTITY_JOBS')
export class Processor {
  constructor() {}

  @Process({ name: 'create_post', concurrency: 1 })
  async create_post(job: Job<unknown>) {
    // handling logic is here
  }

  @Process({ name: 'update_post', concurrency: 1 })
  async update_post(job: Job<unknown>) {
    // handling logic is here
  }
}

as result if I define only one @process for jobs create_post so concurrency: 1 works correct and jobs are processing one by one. But when I add @process for update_post jobs, concurrency doesn't work anymore and I have 2 parallel processing of even the same job create_post. This behaviour is in nest/bull documentation

WARNING
When defining multiple consumers for the same queue, the concurrency option in @Process({ concurrency: 1 }) won't take effect. The minimum concurrency will match the number of consumers defined. This also applies even if @Process() handlers use a different name to handle named jobs.

But, what if I need exactly global concurrency for this queue?

[mckrava]

This thread was like a yellow duck for me so after my post I realised a solution. If it's still interesting for smb (@mgpai22 ):
Check this section in Bull docs and especially this part Specifying * as the process name will make it the default processor for all named jobs..
As result my solution looks like this:

@Processor('SOCIAL_ENTITY_JOBS')
export class Processor {
  constructor() {}
  
  @Process({ name: '*', concurrency: 1 })
  async processJob(job: Job<unknown>) {
    switch (job.name) {
      case "create_post":
        return this.create_post(job);
      case "update_post":
        return this.update_post(job);
    }
  }

  async create_post(job: Job<unknown>) {
    // handling logic is here
  }

  async update_post(job: Job<unknown>) {
    // handling logic is here
  }
}

[kamilmysliwiec]

@nestjs/bull is just a simple wrapper of bull package so if you need any extra features you should report this fr/issue in their repository.