ejhayes / bull-monitor Goto Github PK
View Code? Open in Web Editor NEWAll in one bull monitoring and reporting utility
Home Page: https://hub.docker.com/r/ejhayes/nodejs-bull-monitor
All in one bull monitoring and reporting utility
Home Page: https://hub.docker.com/r/ejhayes/nodejs-bull-monitor
Bull Dashboard is not working. Here's the log file
[Nest] 1 - 12/06/2021, 1:11:18 PM [NestFactory] Starting Nest application...
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] AppModule dependencies initialized +45ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] ConfigModule dependencies initialized +0ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] RedisModule dependencies initialized +0ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] HealthModule dependencies initialized +1ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] DiscoveryModule dependencies initialized +0ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] LoggerModule dependencies initialized +3ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] VersionModule dependencies initialized +1ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] EventEmitterModule dependencies initialized +1ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] RedisCoreModule dependencies initialized +1ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] MetricsModule dependencies initialized +0ms
[Nest] 1 - 12/06/2021, 1:11:18 PM [InstanceLoader] BullModule dependencies initialized +1ms
{"context":"bootstrap","timestampDiff":"","message":"Listening on HTTP port 3005","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullDashboardMiddleware","timestampDiff":"+5ms","message":"Creating middleware","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"RoutesResolver","timestampDiff":"+1ms","message":"HealthController {/health}:","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"RouterExplorer","timestampDiff":"+2ms","message":"Mapped {/health, GET} route","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"RoutesResolver","timestampDiff":"+0ms","message":"MetricsController {/metrics}:","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"RouterExplorer","timestampDiff":"+1ms","message":"Mapped {/metrics, GET} route","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"RoutesResolver","timestampDiff":"+0ms","message":"VersionController {/version}:","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"RouterExplorer","timestampDiff":"+0ms","message":"Mapped {/version, GET} route","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+1ms","message":"Bootstrapping","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+1ms","message":"Waiting for redis to be ready","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"NestApplication","timestampDiff":"+4ms","message":"Nest application successfully started","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+27ms","message":"Redis connection READY! Configuring watchers for bull queues.","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+4ms","message":"Redis config for notify-keyspace-events: 'AK'","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+0ms","message":"Updating notify-keyspace-events to '$AK'","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+3ms","message":"Queues currently monitored: <none>","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+0ms","message":"Loading queues from queuePrefix: 'bull'","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+338ms","message":"Pruning unused queues: <none>","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
{"context":"BullQueuesService","timestampDiff":"+2ms","message":"Subscribed to keyspace events for bull","label":"bull-monitor","@timestamp":"2021-12-06T13:11:19","log":{"level":"debug"}}
this is my configuration:
bull-monitor:
image: ejhayes/nodejs-bull-monitor:latest
ports:
- 8300:8300
environment:
REDIS_HOST: xxxxxxxxxxxx
REDIS_PORT: 6379
PORT: 8300
BULL_WATCH_QUEUE_PREFIXES: multi
UI: bull-master
Error:
bull-monitor_1 | ReplyError
bull-monitor_1 | ReplyError: ERR unknown command `config`, with args beginning with: `GET`, `notify-keyspace-events`,
bull-monitor_1 | at parseError (/app/node_modules/redis-parser/lib/parser.js:179:12)
bull-monitor_1 | at parseType (/app/node_modules/redis-parser/lib/parser.js:302:14)
Hey !
First, thx for creating this tool !
I'm using since few months bull-dashboard but I was searching an alternative with more features. So on the paper your tool is perfect for my case :) So I started by adding it to my stack in my docker-compose file.
bullmonitor:
image: ejhayes/nodejs-bull-monitor
restart: always
ports:
- 3003:3000
environment:
REDIS_HOST: redis-jobs
REDIS_PORT: 6379
Here was my config for bullboard :
bullboard:
image: deadly0/bull-board
restart: always
ports:
- 3004:3000
environment:
REDIS_HOST: redis-jobs
When I start my stack, everything looks fine, the dashboard is starting etc.. but it never detect my queues. ( It tested with bull-master UI, same issue ).
If I check on my old bullboard at the same time, the queues are here and I can see the jobs etc...
I only found a tricky method ( maybe more witchcraft ) to make it work. I restart my old bullboard while bull-monitor is running and then my queues are detected by bull-monitor. I don't have enough knowledge in bull to explain why but it works ๐คท
So my question is, how can I fix that ? Did I do something wrong in my configuration ?
Here are the logs of bull-monitor.
Thx in advance :)
Have a good day !
โ api git:(dev) โ docker logs c0b
OAuth2 Proxy Disabled
{"@timestamp":"2022-11-07T13:27:50","context":"Daemon","label":"bull-monitor","log":{"level":"info"},"message":"Starting bull-monitor process","timestampDiff":""}
{"@timestamp":"2022-11-07T13:27:52","context":"NestFactory","label":"bull-monitor","log":{"level":"info"},"message":"Starting Nest application...","timestampDiff":""}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+81ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+16ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+0ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InjectorLogger","label":"bull-monitor","log":{"level":"info"},"message":"Nest encountered an undefined dependency. This may be due to a circular import or a missing dependency declaration.","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"AppModule dependencies initialized","timestampDiff":"+16ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"ConfigModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"RedisModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"HealthModule dependencies initialized","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"DiscoveryModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"LoggerModule dependencies initialized","timestampDiff":"+12ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"VersionModule dependencies initialized","timestampDiff":"+12ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"EventEmitterModule dependencies initialized","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"MetricsModule dependencies initialized","timestampDiff":"+5ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"RedisCoreModule dependencies initialized","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:52","context":"InstanceLoader","label":"bull-monitor","log":{"level":"info"},"message":"BullModule dependencies initialized","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"bootstrap","label":"bull-monitor","log":{"level":"info"},"message":"Listening on HTTP port 3000","timestampDiff":"+1949ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RoutesResolver","label":"bull-monitor","log":{"level":"info"},"message":"HealthController {/health}:","timestampDiff":"+6ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RouterExplorer","label":"bull-monitor","log":{"level":"info"},"message":"Mapped {/health, GET} route","timestampDiff":"+7ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RoutesResolver","label":"bull-monitor","log":{"level":"info"},"message":"MetricsController {/metrics}:","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RouterExplorer","label":"bull-monitor","log":{"level":"info"},"message":"Mapped {/metrics, GET} route","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RoutesResolver","label":"bull-monitor","log":{"level":"info"},"message":"VersionController {/version}:","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"RouterExplorer","label":"bull-monitor","log":{"level":"info"},"message":"Mapped {/version, GET} route","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Bootstrapping","timestampDiff":"+5ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"NestApplication","label":"bull-monitor","log":{"level":"info"},"message":"Nest application successfully started","timestampDiff":"+17ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"[subscriber] ready","timestampDiff":"+71ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"[publish] ready","timestampDiff":"+7ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Subscribed to 1 keyspace event(s) for 'bull'","timestampDiff":"+4ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Registering pmessage listener","timestampDiff":"+0ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Redis connection READY! Configuring watchers for bull queues.","timestampDiff":"+2ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Redis config for notify-keyspace-events: ''","timestampDiff":"+3ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Updating notify-keyspace-events to '$AK'","timestampDiff":"+1ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Loading queues from queuePrefix: 'bull'","timestampDiff":"+0ms"}
{"@timestamp":"2022-11-07T13:27:54","context":"BullQueuesService","label":"bull-monitor","log":{"level":"info"},"message":"Pruning unused queues: <none>","timestampDiff":"+5ms"}
Hi, I want to integrate this with Grafana dashboard( https://grafana.com/grafana/dashboards/14538 (import into grafana using id: 14538)). How can I do it?
Add support for bull pro queues
hgetall bull:SOME_BULLMQ_PRO_QUEUE:meta
1) "opts.maxLenEvents"
2) "10000"
3) "version"
4) "bullmq-pro"
The only discernible difference between this and bullmq queues is that the version
key does not exist (returns nil
)
Our redis instance uses a password, the nestjs-redis module allows to use this. It would be great if you would enable this in the docker image
export default {
host: process.env.REDIS_HOST,
port: parseInt(process.env.REDIS_PORT),
db: parseInt(process.env.REDIS_DB),
password: process.env.REDIS_PASSWORD,
keyPrefix: process.env.REDIS_PRIFIX,
}
Hey @ejhayes
do you happen to know if I have to keep completed and failed jobs saved in order for bull-monitor to work?
I have one worker with a config like so:
new Worker('my-queue', myJobProcessor, {
autorun: false,
connection: new Redis(`${process.env.REDIS_URI}`, {
maxRetriesPerRequest: null,
}),
removeOnComplete: { count: 50 },
removeOnFail: { count: 250 },
});
And bull-monitor, just as expected, is able to create metrics on jobs in that queue.
Now I added another worker, and because the amount of data transmitted inside the job is too big, I'm deleting completed and failed job right away like so:
new Worker('my-other-queue', myOtherJobProcessor, {
autorun: false,
connection: new Redis(`${process.env.REDIS_URI}`, {
maxRetriesPerRequest: null,
}),
removeOnComplete: { count: 0 },
removeOnFail: { count: 0 },
});
For the latter bull-monitor is crashing on me with the following:
TypeError: Cannot read properties of undefined (reading 'name')
at QueueEvents.<anonymous> (/src/bull/bullmq-metrics.factory.ts:267:37)
...
which looks like to come down on the queueEvents in the bull-metrics.factory
example bullmq-metrics.factory.ts#L204
queueEvents.on('stalled', async (event) => {
const job = await queue.getJob(event.jobId);
const jobLabels = {
[LABEL_NAMES.JOB_NAME]: job.name,
...labels,
};
this.jobs_stalled.inc(jobLabels, 1);
});
Am I correct to assume that .getJob(event.jobId)
returns undefined
at this point so the the labels cannot be build for the job when I'm not retaining some of the completed and failed jobs?
Hey @ejhayes ๐
any chance you could offer the option to pass a redis url instead of the host/port? that would allow to connect optionally via tls using rediss://
or redis://
.
Looks like @nestjs/redis
is supporting it here. As for implementation one could prioritise an env such as REDIS_URL
over REDIS_HOST/REDIS_PORT/REDIS_PASSWORD
if set.
wdyt?
Any plans to add support for a redis db number option?
I have update mentioned docker configs with kubernetes i have received some metrics but i dont see some other metrics below,
these are missing. below my k8s deloyment config
below my k8s deployment config
apiVersion: apps/v1
kind: Deployment
metadata:
name: bull-monitor
namespace: staging-monitoring
labels:
app: bull-monitor
spec:
replicas: 1
selector:
matchLabels:
app: bull-monitor
template:
metadata:
labels:
app: bull-monitor
spec:
containers:
- name: bull-monitor
image: ejhayes/nodejs-bull-monitor:latest
ports:
- containerPort: 3000
env:
- name: NODE_ENV
value: "production"
- name: REDIS_HOST
value: "service-staging-redis.hh-staging.svc.cluster.local"
- name: REDIS_PORT
value: "6379"
- name: PORT
value: "3000"
- name: BULL_WATCH_QUEUE_PREFIXES
value: "bull"
- name: UI
value: "arena"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: < name of secret?
key: REDIS_PASSWORD
- name: COLLECT_NODEJS_METRICS
value: "true"
# livenessProbe:
# httpGet:
# path: /
# port: 3000
# initialDelaySeconds: 30
# periodSeconds: 10
# readinessProbe:
# httpGet:
# path: /
# port: 3000
# initialDelaySeconds: 30
# periodSeconds: 10
# restartPolicy: Always
apiVersion: v1
kind: Service
metadata:
name: bull-monitor
namespace: staging-monitoring
labels:
app: bull-monitor
annotations:
prometheus.io/scrape: 'true'
prometheus.io/port: '3000'
spec:
type: NodePort
ports:
after this i add scrape_config ass below to prometheus server ,
but when i see grafana after import dashbards i see only below,
kindly advise me !! @ejhayes
Hi, I'd like to use bull-monitor Docker on my M1 Mac development machine, but it's only available as an amd64 build. Could you build it for ARM as well?
Issue:
โ bull-monitor The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
I like this package, in that it has a Grafana Dashboard and Prometheus Metrics.
However it does not work for me with BullMQ v4.8.0 (the latest version)
Are there any plans to supporting BullMQ v4?
Hello,
Can we have an example of integration with OAUTH2 proxy?
I managed to add new variables related to the proxy.
When accessing app, I now have the oauth-proxy page and it redirects me to the IdP (keycloak for me).
I can see that the authentification is sucessfull as they are messages like:
{<USER>@<DOMAIN> [2023/07/13 15:10:47] [AuthSuccess] Authenticated via OAuth2
But then it redirect me to the URL like https://<URL>/oauth2/callback?state=XXXX&session_state=XXXX
and I have a 502 badGateway
Add support for Bull queues
keys bull:QUEUE_NAME:wait
In bull there will not be an equivalent bull:QUEUE_NAME:meta
key so this is our only indicator that this is a regular bull
queue
Hello,
Is is possible to add the READONLY mode configurable for bull-board?
felixmosh/bull-board#220
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.