GithubHelp home page GithubHelp logo

ejhayes / bull-monitor Goto Github PK

View Code? Open in Web Editor NEW
52.0 2.0 25.0 3.31 MB

All in one bull monitoring and reporting utility

Home Page: https://hub.docker.com/r/ejhayes/nodejs-bull-monitor

JavaScript 1.36% Shell 2.02% TypeScript 95.37% Dockerfile 1.25%
arena bull bullmq dashboard job-queue redis-queue bull-board bull-master

bull-monitor's People

Contributors

dependabot[bot] avatar ejhayes avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

roderik jhbae200 nadishan guigotv gox-ai zppeter otanriverdi tbvgl iwpnd pandelisz naldomadeira yharaskrik snappycommerce afillatre pandocorp rsnider-chg cardtonic-technologies rzhosan wiseng90 dahakagh nwebrde kyo06 giankaz joaocrulhas

bull-monitor's Issues

Bull Dashboard not working

image

Bull Dashboard is not working. Here's the log file


[Nest] 1   - 12/06/2021, 1:11:18 PM   [NestFactory] Starting Nest application...

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] AppModule dependencies initialized +45ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] ConfigModule dependencies initialized +0ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] RedisModule dependencies initialized +0ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] HealthModule dependencies initialized +1ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] DiscoveryModule dependencies initialized +0ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] LoggerModule dependencies initialized +3ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] VersionModule dependencies initialized +1ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] EventEmitterModule dependencies initialized +1ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] RedisCoreModule dependencies initialized +1ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] MetricsModule dependencies initialized +0ms

[Nest] 1   - 12/06/2021, 1:11:18 PM   [InstanceLoader] BullModule dependencies initialized +1ms

{"context":"bootstrap","timestampDiff":"","message":"Listening on HTTP port 3005","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullDashboardMiddleware","timestampDiff":"+5ms","message":"Creating middleware","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"RoutesResolver","timestampDiff":"+1ms","message":"HealthController {/health}:","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"RouterExplorer","timestampDiff":"+2ms","message":"Mapped {/health, GET} route","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"RoutesResolver","timestampDiff":"+0ms","message":"MetricsController {/metrics}:","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"RouterExplorer","timestampDiff":"+1ms","message":"Mapped {/metrics, GET} route","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"RoutesResolver","timestampDiff":"+0ms","message":"VersionController {/version}:","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"RouterExplorer","timestampDiff":"+0ms","message":"Mapped {/version, GET} route","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+1ms","message":"Bootstrapping","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+1ms","message":"Waiting for redis to be ready","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"NestApplication","timestampDiff":"+4ms","message":"Nest application successfully started","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+27ms","message":"Redis connection READY! Configuring watchers for bull queues.","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+4ms","message":"Redis config for notify-keyspace-events: 'AK'","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+0ms","message":"Updating notify-keyspace-events to '$AK'","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+3ms","message":"Queues currently monitored: <none>","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+0ms","message":"Loading queues from queuePrefix: 'bull'","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+338ms","message":"Pruning unused queues: <none>","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

{"context":"BullQueuesService","timestampDiff":"+2ms","message":"Subscribed to keyspace events for bull","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}

Getting error while running through docker-compose

this is my configuration:

bull-monitor:
    image: ejhayes/nodejs-bull-monitor:latest
    ports:
      - 8300:8300
    environment:
      REDIS_HOST: xxxxxxxxxxxx
      REDIS_PORT: 6379
      PORT: 8300
      BULL_WATCH_QUEUE_PREFIXES: multi
      UI: bull-master

Error:

bull-monitor_1                      | ReplyError
bull-monitor_1                      | ReplyError: ERR unknown command `config`, with args beginning with: `GET`, `notify-keyspace-events`,
bull-monitor_1                      |     at parseError (/app/node_modules/redis-parser/lib/parser.js:179:12)
bull-monitor_1                      |     at parseType (/app/node_modules/redis-parser/lib/parser.js:302:14)

While running docker-compose up i am getting below error.
image

Queues not detected

Hey !
First, thx for creating this tool !

I'm using since few months bull-dashboard but I was searching an alternative with more features. So on the paper your tool is perfect for my case :) So I started by adding it to my stack in my docker-compose file.

bullmonitor:
    image: ejhayes/nodejs-bull-monitor
    restart: always
    ports:
      - 3003:3000
    environment:
      REDIS_HOST: redis-jobs
      REDIS_PORT: 6379

Here was my config for bullboard :

bullboard:
    image: deadly0/bull-board
    restart: always
    ports:
      - 3004:3000
    environment:
      REDIS_HOST: redis-jobs

When I start my stack, everything looks fine, the dashboard is starting etc.. but it never detect my queues. ( It tested with bull-master UI, same issue ).

image

If I check on my old bullboard at the same time, the queues are here and I can see the jobs etc...

I only found a tricky method ( maybe more witchcraft ) to make it work. I restart my old bullboard while bull-monitor is running and then my queues are detected by bull-monitor. I don't have enough knowledge in bull to explain why but it works ๐Ÿคท

So my question is, how can I fix that ? Did I do something wrong in my configuration ?
Here are the logs of bull-monitor.

Thx in advance :)
Have a good day !

โžœ  api git:(dev) โœ— docker logs c0b
OAuth2 Proxy Disabled
{"@timestamp":"2022-11-07T13:27:50","context":"Daemon","label":"bull-monitor","log":{"level":"info"},"message":"Starting bull-monitor process","timestampDiff":""}
{"@timestamp":"2022-11-07T13:27:52","context":"NestFactory","label":"bull-monitor","log":{"level":"info"},"message":"Starting Nest application...","timestampDiff":""}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+81ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+16ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+0ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"AppModule dependencies initialized","timestampDiff":"+16ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"ConfigModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"RedisModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"HealthModule dependencies initialized","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"DiscoveryModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"LoggerModule dependencies initialized","timestampDiff":"+12ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"VersionModule dependencies initialized","timestampDiff":"+12ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"EventEmitterModule dependencies initialized","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"MetricsModule dependencies initialized","timestampDiff":"+5ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"RedisCoreModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"BullModule dependencies initialized","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"bootstrap","label":"bull-monitor","log":{"level":"info"},"message":"Listening on HTTP port 3000","timestampDiff":"+1949ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RoutesResolver","label":"bull-monitor","log":{"level":"info"},"message":"HealthController {/health}:","timestampDiff":"+6ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RouterExplorer","label":"bull-monitor","log":{"level":"info"},"message":"Mapped {/health, GET} route","timestampDiff":"+7ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RoutesResolver","label":"bull-monitor","log":{"level":"info"},"message":"MetricsController {/metrics}:","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RouterExplorer","label":"bull-monitor","log":{"level":"info"},"message":"Mapped {/metrics, GET} route","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RoutesResolver","label":"bull-monitor","log":{"level":"info"},"message":"VersionController {/version}:","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RouterExplorer","label":"bull-monitor","log":{"level":"info"},"message":"Mapped {/version, GET} route","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Bootstrapping","timestampDiff":"+5ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"NestApplication","label":"bull-monitor","log":{"level":"info"},"message":"Nest application successfully started","timestampDiff":"+17ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"[subscriber] ready","timestampDiff":"+71ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"[publish] ready","timestampDiff":"+7ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Subscribed to 1 keyspace event(s) for 'bull'","timestampDiff":"+4ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Registering pmessage listener","timestampDiff":"+0ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Redis connection READY! Configuring watchers for bull queues.","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Redis config for notify-keyspace-events: ''","timestampDiff":"+3ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Updating notify-keyspace-events to '$AK'","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Loading queues from queuePrefix: 'bull'","timestampDiff":"+0ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Pruning unused queues: <none>","timestampDiff":"+5ms"}

Bull Pro Support

Add support for bull pro queues

hgetall bull:SOME_BULLMQ_PRO_QUEUE:meta
1) "opts.maxLenEvents"
2) "10000"
3) "version"
4) "bullmq-pro"

The only discernible difference between this and bullmq queues is that the version key does not exist (returns nil)

Allow Redis with a password

Our redis instance uses a password, the nestjs-redis module allows to use this. It would be great if you would enable this in the docker image

export default {
    host: process.env.REDIS_HOST,
    port: parseInt(process.env.REDIS_PORT),
    db: parseInt(process.env.REDIS_DB),
    password: process.env.REDIS_PASSWORD,
    keyPrefix: process.env.REDIS_PRIFIX,
}

Missing metrics for completed jobs that are automatically removed

Hey @ejhayes

do you happen to know if I have to keep completed and failed jobs saved in order for bull-monitor to work?

I have one worker with a config like so:

    new Worker('my-queue', myJobProcessor, {
        autorun: false,
        connection: new Redis(`${process.env.REDIS_URI}`, {
            maxRetriesPerRequest: null,
        }),
        removeOnComplete: { count: 50 },
        removeOnFail: { count: 250 },
    });

And bull-monitor, just as expected, is able to create metrics on jobs in that queue.

Now I added another worker, and because the amount of data transmitted inside the job is too big, I'm deleting completed and failed job right away like so:

    new Worker('my-other-queue', myOtherJobProcessor, {
        autorun: false,
        connection: new Redis(`${process.env.REDIS_URI}`, {
            maxRetriesPerRequest: null,
        }),
        removeOnComplete: { count: 0 },
        removeOnFail: { count: 0 },
    });

For the latter bull-monitor is crashing on me with the following:

TypeError: Cannot read properties of undefined (reading 'name')
at QueueEvents.<anonymous> (/src/bull/bullmq-metrics.factory.ts:267:37)
...

which looks like to come down on the queueEvents in the bull-metrics.factory

example bullmq-metrics.factory.ts#L204

    queueEvents.on('stalled', async (event) => {
      const job = await queue.getJob(event.jobId);
      const jobLabels = {
        [LABEL_NAMES.JOB_NAME]: job.name,
        ...labels,
      };
      this.jobs_stalled.inc(jobLabels, 1);
    });

Am I correct to assume that .getJob(event.jobId) returns undefined at this point so the the labels cannot be build for the job when I'm not retaining some of the completed and failed jobs?

feature request: allow connection url

Hey @ejhayes ๐Ÿ‘‹

any chance you could offer the option to pass a redis url instead of the host/port? that would allow to connect optionally via tls using rediss:// or redis://.

Looks like @nestjs/redis is supporting it here. As for implementation one could prioritise an env such as REDIS_URL over REDIS_HOST/REDIS_PORT/REDIS_PASSWORD if set.

wdyt?

when update the docker configs with k8s manifest missing metrics

I have update mentioned docker configs with kubernetes i have received some metrics but i dont see some other metrics below,

  • jobs_waiting
  • jobs_stalled
  • jobs_failed
  • jobs_completed
  • jobs_delayed
  • job_duration
  • job_wait_duration
  • job_attempts

these are missing. below my k8s deloyment config

below my k8s deployment config

apiVersion: apps/v1
kind: Deployment
metadata:
name: bull-monitor
namespace: staging-monitoring
labels:
app: bull-monitor
spec:
replicas: 1
selector:
matchLabels:
app: bull-monitor
template:
metadata:
labels:
app: bull-monitor
spec:
containers:
- name: bull-monitor
image: ejhayes/nodejs-bull-monitor:latest
ports:
- containerPort: 3000
env:
- name: NODE_ENV
value: "production"
- name: REDIS_HOST
value: "service-staging-redis.hh-staging.svc.cluster.local"
- name: REDIS_PORT
value: "6379"
- name: PORT
value: "3000"
- name: BULL_WATCH_QUEUE_PREFIXES
value: "bull"
- name: UI
value: "arena"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: < name of secret?
key: REDIS_PASSWORD
- name: COLLECT_NODEJS_METRICS
value: "true"

below i turned off probes because pods getting restarted always

  #   livenessProbe:
  #     httpGet:
  #       path: /
  #       port: 3000
  #     initialDelaySeconds: 30
  #     periodSeconds: 10
  #   readinessProbe:
  #     httpGet:
  #       path: /
  #       port: 3000
  #     initialDelaySeconds: 30
  #     periodSeconds: 10
  # restartPolicy: Always

apiVersion: v1
kind: Service
metadata:
name: bull-monitor
namespace: staging-monitoring
labels:
app: bull-monitor
annotations:
prometheus.io/scrape: 'true'
prometheus.io/port: '3000'
spec:
type: NodePort
ports:

  • port: 3000
    targetPort: 3000
    nodePort: 30000
    selector:
    app: bull-monitor

after this i add scrape_config ass below to prometheus server ,

  • job_name: 'bull-exporter'
    scrape_interval: 5s
    static_configs:
    - targets: ['bull-monitor.staging-monitoring.svc.cluster.local:3000', 'service-staging-redis.staging.svc.cluster.local:6379']

but when i see grafana after import dashbards i see only below,
image

image

metrics are
image

kindly advise me !! @ejhayes

ARM build

Hi, I'd like to use bull-monitor Docker on my M1 Mac development machine, but it's only available as an amd64 build. Could you build it for ARM as well?

Issue:

โ ‡ bull-monitor The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s

BullMQ v4

I like this package, in that it has a Grafana Dashboard and Prometheus Metrics.
However it does not work for me with BullMQ v4.8.0 (the latest version)

Are there any plans to supporting BullMQ v4?

oauth2-proxy example

Hello,
Can we have an example of integration with OAUTH2 proxy?
I managed to add new variables related to the proxy.
When accessing app, I now have the oauth-proxy page and it redirects me to the IdP (keycloak for me).
I can see that the authentification is sucessfull as they are messages like:

{<USER>@<DOMAIN> [2023/07/13 15:10:47] [AuthSuccess] Authenticated via OAuth2

But then it redirect me to the URL like https://<URL>/oauth2/callback?state=XXXX&session_state=XXXX and I have a 502 badGateway

Bull Support

Add support for Bull queues

keys bull:QUEUE_NAME:wait

In bull there will not be an equivalent bull:QUEUE_NAME:meta key so this is our only indicator that this is a regular bull queue

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.