s-r-x / bull-monitor

🐂 Standard UI for Bull and BullMQ.

bull-monitor

Standard UI for Bull and BullMQ.

Demo

Usage

• Express
• Koa
• Hapi
• Fastify
• Nest
• CLI

bull-monitor's Issues

Express 5 Support?

I've been using the bull monitor with express 4.x and am upgrading my project to express 5.x. However, bull monitor stopped working after upgrading the express version. The dashboard loads fine but the graphql requests are stuck at pending status.

Any idea?

,

Data search term should be exact equal to data search key

Currently, data search term input performs a "like" query inside the object values, when it should be exact equal IMHO

example:

If i type: Product onto the term, it matches items with term ProductOption when it should only match Product (exact equal ===).

My sugestion is to add "like wildcards" support to this field, so it will become:

Product === Product
Product% === ProductOption
%ProductOption% === AProductOption

Jhon === Jhon
Jhon% === Jhon Doe
%Jhon% === Marcus Jhon Doe

Stacktrace

Render job's stacktrace somewhere if one is not empty(maybe inside the timeline popover or above the data)

Error when gettings failed jobs

Sometimes we have a few failed jobs but when trying to list them in bull-monitor we get this message.

Javascript console shows this.

Readonly Support

It would be nice to have read-only support which hides/disables any write operations like pausing queue, adding job, clearing stats etc. Maybe this is already there but don't see anywhere the initialization specifications.

Clean command doesn't update the drawer inline indicator when there are only one type of jobs in selected queue

FR: allow to export multiple job as a json file

@s-r-x it would be nice to be able to export multiple jobs as a JSON file. Thanks!

How to pass bull queue instance to BullMonitorExpress once the BullMonitorExpress instantiate

I'm trying to integrate @bull-monitor/express
in in loopback4. I somehow integrate the Bull-monitor-express in loopback4. below is an example that I have followed.

import { BullMonitorExpress } from '@bull-monitor/express';
import Express from 'express';

const port = 3000;
const baseUrl = '/some/nested/url';
(async () => {
const app = Express();
const monitor = new BullMonitorExpress({ queues: [] });
await monitor.init();
app.use(baseUrl, monitor.router);
app.listen(port, () => console.log(http://localhost:${port}${baseUrl}));
})();

if I pass queues while new BullMonitorExpress({ queues}) is instantiating its perfectly working fine. but I'm not able to pass queues post instantiation of new BullMonitorExpress({ queues: []}) .
I'm not able to find any thing regarding this in the document. can somebody help me with this?

User authentication needed to access UI page

Curently it seems that any one can access the bull-monitor page, how to protect the page with password or jwt token?

• Nestjs
  import { BullModule } from '@nestjs/bull';
  import { MiddlewareConsumer, Module, NestModule } from '@nestjs/common';
  import { BullMonitorService } from './bull-monitor.service';
  import { Q_AUDIO_NAME } from '../audio-queue/audio.interface';

@module({
imports: [BullModule.registerQueue({ name: Q_AUDIO_NAME })],
providers: [BullMonitorService],
})
export class BullMonitorModule implements NestModule {
constructor(private monitor: BullMonitorService) {}
async configure(consumer: MiddlewareConsumer) {
await this.monitor.init();
consumer.apply(this.monitor.router).forRoutes('/bull-monitor');
}
}

Repeatable jobs do not work properly

Repeatable jobs could not be added correctly via "Add Job".

The job is placed in the "delayed" state and is also executed at the specified time, but never again afterwards. If I create the job directly via the Bull library with the exact same parameters, it remains in the "delayed" state until (if) an end condition occurs.

My concrete example (without end condition):

Job
{ "payload": ... }
Options
{ "repeat": { "every": 60000 } }

The job is even given an ID that indicates that it is a "repeatable job", but it is not executed again.

@nestjs/platform-fastify bug

I've tried to set this up with my nestjs queue consumer but i keep getting the following error: when trying to use BullMonitorFastify

[Nest] 30444  - 10/02/2022, 17:09:20   ERROR [ExceptionsHandler] this.willStart is not a function
TypeError: this.willStart is not a function
    at ApolloServer.createHandler (C:\x\node_modules\apollo-server-fastify\src\ApolloServer.ts:44:35)
    at plugin (C:\x\node_modules\@bull-monitor\fastify\src\index.ts:12:21)
    at Object.use (C:\x\node_modules\@nestjs\core\middleware\utils.js:51:24)
    at C:\x\node_modules\@nestjs\core\router\router-proxy.js:9:23
    at Holder.done (C:\x\node_modules\middie\engine.js:107:13)
    at Object.run (C:\x\node_modules\middie\engine.js:59:12)
    at Object.runMiddie (C:\x\node_modules\middie\index.js:41:21)
    at hookIterator (C:\x\node_modules\fastify\lib\hooks.js:237:10)
    at next (C:\x\node_modules\fastify\lib\hooks.js:164:16)
    at hookRunner (C:\x\node_modules\fastify\lib\hooks.js:187:3)

bull-monitor.service.ts

import { BullMonitorFastify } from '@bull-monitor/fastify';
import { InjectQueue } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';
import { BullAdapter } from '@bull-monitor/root/dist/bull-adapter';
import { QueueName } from '@composer/queue/enums/index.enum';

@Injectable()
export class BullMonitorService extends BullMonitorFastify {
  constructor(
    @InjectQueue(QueueName.FEED) feedQueue: Queue,
    @InjectQueue(QueueName.STOCK) stockQueue: Queue
  ) {
    super({
      queues: [new BullAdapter(feedQueue), new BullAdapter(stockQueue)],
    });
  }
}

My QueueServiceModule (aka AppModule)

import { MiddlewareConsumer, Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bull';
import { QueueName } from '@composer/queue/enums/queues.enum';
import { StockConsumer } from './consumers/stock.consumer';
import { FeedConsumer } from './consumers/feed.consumer';
import { HttpModule } from '@nestjs/axios';
import { StockModule } from '@composer/stock';
import { ClientModule } from '@composer/client';
import { TypeOrmModule } from '@nestjs/typeorm';
import { ConfigService, ConfigModule } from '@nestjs/config';
import { CoreModule } from '@composer/core';
import { IntegrationsModule } from '@composer/integrations';
import configuration from '@composer/core/config/configuration';
import { BullMonitorService } from './bull-monitor.service';

@Module({
  imports: [
    CoreModule,
    ConfigModule.forRoot({
      load: [configuration],
    }),
    BullModule.forRootAsync({
      imports: [ConfigModule],
      useFactory: async (configService: ConfigService) => ({
        redis: {
          port: parseInt(configService.get('bullMq.redis.port')),
          host: configService.get('bullMq.redis.host'),
          password: configService.get('bullMq.redis.password'),
          tls: {
            servername: configService.get('bullMq.redis.host'),
          },
        },
        defaultJobOptions: configService.get('bullMq.defaultJobOptions'),
      }),
      inject: [ConfigService],
    }),
    BullModule.registerQueue(
      {
        name: QueueName.STOCK,
      },
      {
        name: QueueName.FEED,
      }
    ),
    HttpModule,
    TypeOrmModule.forRootAsync({
      imports: [ConfigModule],
      useFactory: async (configService: ConfigService) => ({
        type: 'mssql',
        host: configService.get('database.host'),
        port: parseInt(configService.get('database.port')),
        username: configService.get('database.username'),
        password: configService.get('database.password'),
        options: {
          enableArithAbort: true,
          trustServerCertificate: true,
        },
        pool: {
          min: 0,
          max: 100,
        },
        maxQueryExecutionTime: 500,
        logger: 'advanced-console',
        autoLoadEntities: true,
      }),
      inject: [ConfigService],
    }),
    IntegrationsModule,
    StockModule,
    ClientModule,
  ],
  providers: [StockConsumer, FeedConsumer, BullMonitorService],
})
export class QueueServiceModule {
  constructor(private monitor: BullMonitorService) {}
  async configure(consumer: MiddlewareConsumer) {
    await this.monitor.init();
    consumer.apply(this.monitor.plugin).forRoutes('/bull-monitor');
  }
}

Queue list name truncated

The queue name is truncated because of the sidebar width. Maybe add the queue name on the right side as the title? or overflow on the MuiListItemText-root instead, so still be able to scroll on the text? and also add <title> Tag to each queue?

Thanks.

Error getting Redis info from Redis cloud database

Im getting Cannot return null for non-nullable field RedisInfo.total_system_memory_human when i try to see redis info, when connected to redis cloud.

Any ideas?

(Love this UI, great job)

Resizable drawer issue after scrolling

Hi, @saade @s-r-x please check the below recording, I can't resize the drawer after scrolling.

• Chrome: 91.0.4472.77 (Official Build) (x86_64)
• macOS Big Sur
• "@bull-monitor/express": "^0.26.0"

---

Aslo, do you think it is possible to allow resizable width until to 50vw?

Export entire queue as JSON

It would be nice to export the entire queue as a JSON file. Currently is only possible export selected items.

In this real life example, i need to export all 1k items to a file for further investigation

[ui] migrate from snowpack to vite

Export active status jobs not working

Hi! Thanks for #26, but it seems not working.

Link back to Github

Would be nice to have link to this Github repository from the monitoring dashboard.

Support object progress

I keep getting a rendering error when the job progress is an object { "percents:" 100, "currentStep": "finished" }:

Error: Int cannot represent non-integer value: { percents: 100, currentStep: "finished" }: {"response":{"errors":[{"message":"Int cannot represent non-integer value: { percents: 100, currentStep: \"finished\" }","locations":[{"line":5,"column":5}],"path":["jobs",0,"progress"],"extensions":{"code":"INTERNAL_SERVER_ERROR","exception":{"message":"Int cannot represent non-integer value: { percents: 100, currentStep: \"finished\" }"}...

Maybe it should be a way to with discard the progress, or be able to have an option to map the int value with the right key

handle metrics collector errors

Support for bullmq ?

Is there a plan to support bullmq?

Search jobs by name

Any plans on being able to search for jobs by name and not just id?

migrate to mui 5

format returnValue when importing jobs as json

Enum "JobStatus" cannot represent value: "stuck"

undocumented "stuck" job status

Better navigation url for queue and job

Looking for a better navigation URL for queue and job, so after refreshing browser that we won't lose the view and we can also easy to share queues or jobs.

^ Example above that could produce URLs like:

• queue URL: https://s-r-x.github.io/bull-monitor/queue-2
• queue type URL: https://s-r-x.github.io/bull-monitor/queue-2?active=true
• job id URL(SEARCH): https://s-r-x.github.io/bull-monitor/queue-2?active=true&search=true&jobId=228750
• job data key URL(search): https://s-r-x.github.io/bull-monitor/queue-2?active=true&search=true&key='key'&term='value-24'

Also, would be great to add a share button for each job.

Thanks!!!

Documentation?

Is there any documentation regarding how to use:

• the graphql playground (with some examples?)
• the data search key/value?

search with all status

Hello,

im looking for search in all status. i look this page : https://github.com/s-r-x/bull-monitor/blob/main/search-examples.md

jsonata is very good, i like it. But i cant find my job same time. its may complated, failed or waiting i dont know..

timestamp in metrics chart tooltip is not visible in dark mode, and should be formatted to human readable date

Metrics shows only recent data points.

The metrics section shows only most recent (about 150) data points. Does the backend erase the older ones ? Can we have a way to select the duration and or may be a scroller to view the old metrics also.

FR: a better way to search job instead of job id

Hi, is possible to have an advanced job search?

Currently, job id search is not that useful for us because of the number of the same jobs that run on the same day or time.

To give you an example how our list looks like:

How about being able to deep search inside of each job? like we can search job data's key value? Thanks!

Nestjs

How to add in nestjs?

Summary for processing time metrics

Would be nice to have summary row for the processing time metrics which shows average, minimum and maximum values like one from Grafana given below:

JSON lint tooltip wrong placed.

The tooltip for JSON lint is placed behind 'Add Job' modal.

gql: validate jobs pagination args

where offset >= 0 and limit > 0

Retry multiple failed jobs feature

Hi,

Can this have a retry multiple failed jobs feature?
I think the table should be easy to migrate with checkbox: https://material-ui.com/components/tables/#data-table

Thanks!

3.0.0 ts 4.2 error

'get' and 'set' accessor must have the same type.

queues getter/setter

Command line

Would be nice to have command line tool for quick start without having to write any code.

feat: Add a bigger (perhaps customized) value to 'per page'

I have a lot of jobs inside a queue, and sometimes i need to investigate an error or retry more than 75 itens.

My suggestion is to add a bigger value (like mysql uses 500), or a input field to enter a custom amount of itens per page

add expand to full screen button to create job and update data modals

Core package depends on older bull version

https://github.com/s-r-x/bull-monitor/blob/main/packages/root/package.json#L42

The latest version of bull js is 4.7.0 so this library is quite behind.
https://github.com/OptimalBits/bull/blob/develop/CHANGELOG.md

[UI] Queue prefix

I think it should be really nice and useful to show the queue prefix on the sidebar.

I have a multi-tenant app, and each tenant is using a different prefix to identify it. I don't have any visual separation between them.

Suggestion A

Suggestion B (i think its better)

Cannot return null for non-nullable field Query.jobs

filter jobs before send it to client

add job processing time metric

NestJS + Fastify

I am using Bull for the very first time.
I've written a small NestJS app based on Fastify.

Could someone please guide me through how to setup bull-monitor with NestJS Fastify?

Status chips are transparent inside tables

#21 caused status chips inside tables to be transparent too.

I think #21 should only affect the sidebar chips, not all chips

Redis ACL

The version of ioredis used, doesn't work with Redis ACLs, as it doesn't pass a user to AUTH.
I tested the old version you use, and get the following error, but with e.g. 5.0.5 and the same URL it works completely.

[ioredis] Unhandled error event: ReplyError: WRONGPASS invalid username-password pair or user is disabled.
at parseError (/root/node_modules/redis-parser/lib/parser.js:179:12)
at parseType (/root/node_modules/redis-parser/lib/parser.js:302:14)

Graph shows no data and "Invalid Date" error

Is there some way to fix the Invalid Date error shown at bottom right?

add "beautify" button to the TextEditor component

[NEW FEATURE] Login Page

Hey guys, how are you doing? Are you planning to implement a login page? I'd love to protect my project with a login page.
If you don't, I'm thinking to create a pull request with this feature.