darky / bull-repl Goto Github PK

Hi there,

Thanks for this very useful tool! We've recently had an issue with the cleanup operation, which quickly gets very expensive as the queue size grows. Because it has the potential to block the Redis server for hours, it makes it dangerous to use bull-repl on a production server. I've suggested a fix to the Bull library and it has been integrated in version 4.8.1.

Would it be possible to bump the bull dependency to the latest version in bull-repl?

Thanks!

Eric

Running npm i -g bull-repl@next on Node v14.9.0 and running bull-repl results in:

line 1: use strict: command not found
line 2: syntax error near unexpected token `newline'
line 2: `/// <reference types="./typing" />'

The connect command just prints the help output and says missing required argument

BULL-REPL> connect --prefix repl --redis redis://[email protected]:6379

  Missing required argument. Showing Help:

  Usage: connect [options] <queue>

  connect to bull queue

  Options:

    --help                 output usage information
    -p, --prefix <prefix>  prefix to use for all queue jobs
    -r, --redis <redis>    redis url in format: redis://[:password@]host[:port][/db-number][?option=value]; default redis://localhost:6379

versions:

/home/kev/.nvm/versions/node/v13.7.0/lib
└── [email protected]

Is there any way to restart all failed jobs in queue?

I get the following error when running bull-repl:

λ ~/ bull-repl
/Users/timon/.asdf/lib/commands/shim-exec.sh: /Users/timon/.asdf/installs/nodejs/10.16.0/.npm/bin/bull-repl: /usr/local/bin/node: bad interpreter: No such file or directory
/Users/timon/.asdf/lib/commands/shim-exec.sh: line 26: /Users/timon/.asdf/installs/nodejs/10.16.0/.npm/bin/bull-repl: Undefined error: 0

I suspect this happens because of hashbangify in the build script. Hashbangify looks at your local setup which may not be the same as everyone else's, I think it should just be #!/usr/bin/env node :)

I have a BullMQ + redis instance in k8s that I am trying to connect to

First I port forward:

kubectl port-forward --namespace default svc/redis-master 54321:6379 &

Then with bull-repl@next:

BULL-REPL> connect -r redis://localhost:54321 my-queue
Handling connection for 54321
connect ECONNREFUSED 127.0.0.1:6379
BULL-REPL> (node:812) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1117:14)
(node:812) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:812) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[ioredis] Unhandled error event: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1117:14)

I can connect ok with rdcli

~$ rdcli -p 54321
127.0.0.1:54321> Handling connection for 54321

127.0.0.1:54321>

I need to add a job but I don't know exactly what is the correct syntax.

This is my command:
add "{\"orderId\":185652}"

But it returns:

Error occured, seems "data" incorrect json

I was tried many variations and still get Incorrect period error. Can you provide a simple example of this command? Thank you!

I have 3631530 failed jobs right now. Using just failed just cause process to hang forever. I would like to see only few failed jobs, by specyfing eg. --limit 100 (or defaulting to printing only subset).

In v0.21.0 connection works pretty well, with very handy -r redis url connection.

    --help                 output usage information
    -p, --prefix <prefix>  prefix to use for all queue jobs
    -r, --redis <redis>    redis url in format: redis://[:password@]host[:port][/db-number][?option=value]; default redis://localhost:6379

In 0.22.0 connection doesn't work. And -r option is missing.

    --help                 output usage information
    --prefix <prefix>      prefix to use for all queue jobs
    -h, --host <host>      redis host for connection
    -p, --port <port>      redis port for connection
    -d, --db <db>          redis db for connection
    --password <password>  redis password for connection
    -c, --cert <cert>      absolute path to pem certificate if TLS used

Please can you return -r option

Wen try to add a new job using

add '{"cnpj": "12345678901234"}'

Getting error Object.fromEntries is not a function

Because of this #50?

I think it might be a good idea what are the differences (I'm guessing bull major version incompatibilities?) between 0.x and 1.x :)

How can I use the "clean" command to clean everything in the "failed" status of a queue? I've tried all manner of different things...

clean -s failed

clean --status="failed"

...and it never works, always displays the help topic.

Thanks for building this.

I'd like to be able to clean all jobs in a queue from a script. I see that I can at least connect in a single line, ie, bull-repl connect, but there doesn't seem to be a way to run another command after that? I have to go in as a human and manually type the command to clean a queue myself.

@joebowbeer, feel free to make PR if you have some ideas

I have got errors like this when I install bull-repl，what is going wrong with this？

[root@VM_10_186_centos ~]# npm install bull-repl@next -g
npm WARN deprecated @hapi/[email protected]: joi is leaving the @Hapi organization and moving back to 'joi' (hapijs/joi#2411)
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
/usr/local/bin/bull-repl -> /usr/local/lib/node_modules/bull-repl/index.js

[email protected] postinstall /usr/local/lib/node_modules/bull-repl/node_modules/core-js
node -e "try{require('./postinstall')}catch(e){}"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon:

https://opencollective.com/core-js
https://www.patreon.com/zloirock

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)

[email protected] postinstall /usr/local/lib/node_modules/bull-repl/node_modules/node-jq
npm run install-binary

[email protected] install-binary /usr/local/lib/node_modules/bull-repl/node_modules/node-jq
node scripts/install-binary.js

Building jq from https://github.com/stedolan/jq/releases/download//jq-1.6/jq-1.6.tar.gz
[Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/bull-repl/node_modules/node-jq/d51835be-fc7e-4faf-b374-64aef47a544d'] {
errno: -13,
code: 'EACCES',
syscall: 'mkdir',
path: '/usr/local/lib/node_modules/bull-repl/node_modules/node-jq/d51835be-fc7e-4faf-b374-64aef47a544d'
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install-binary: node scripts/install-binary.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install-binary script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] postinstall: npm run install-binary
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-01-27T16_29_25_471Z-debug.log

Im trying to filter completed jobs list by timestamp with completed --filter='{"timestamp": 2020-07-09T13:57:32.701Z}'
Surely Im doing something wrong, which syntax I need to use?

Hi!

One feature I think would be helpful is the ability to add the name of the queue you want to connect to when you invoke the command, like bull-repl my-bull-queue.

This would allow us to have a generalized command in package.json that is the same across multiple projects, f ex yarn queue or yarn bull that resolves to bull-repl project-specific-queue. That would cut down on the time it would take to get up and running (browsing through code to find the name of the queue + the connect step).

On several issues (#13, #18, #8) people mention the -r option, but I can't get it to work on 0.24:

BULL-REPL> connect --help

  Missing required argument. Showing Help:

  Usage: connect [options] <queue>

  Connect to bull queue

  Options:

    --help                 output usage information
    --prefix <prefix>      Prefix to use for all queue jobs
    -h, --host <host>      Redis host for connection
    -p, --port <port>      Redis port for connection
    -d, --db <db>          Redis db for connection
    --password <password>  Redis password for connection
    -c, --cert <cert>      Absolute path to pem certificate if TLS used

BULL-REPL> connect --redis redis://u:p@h:6379 queue
  Invalid option: 'redis'. Showing Help:
...

BULL-REPL> connect -r redis://u:p@h:6379 queue
  Invalid option: 'r'. Showing Help:
...

What am I doing wrong?

command: completed -t 1h

Error: spawn /home/alchemist/.nvm/versions/node/v16.13.1/lib/node_modules/bull-repl/node_modules/node-jq/bin/jq ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
    at onErrorNT (node:internal/child_process:477:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on ChildProcess instance at:
    at Process.ChildProcess._handle.onexit (node:internal/child_process:288:12)
    at onErrorNT (node:internal/child_process:477:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /home/alchemist/.nvm/versions/node/v16.13.1/lib/node_modules/bull-repl/node_modules/node-jq/bin/jq',
  path: '/home/alchemist/.nvm/versions/node/v16.13.1/lib/node_modules/bull-repl/node_modules/node-jq/bin/jq',
  spawnargs: [
    '{root: [.root[] | select((.timestamp | strptime("%Y-%m-%dT%H:%M:%S.%3Z") | mktime | . * 1000) >= 1660672044796)]}'
  ]
}

Building jq from https://github.com/stedolan/jq/releases/download//jq-1.6/jq-1.6.tar.gz
Error: Command failed: /bin/sh -c autoreconf -fi
configure.ac:36: warning: macro 'AM_PROG_LIBTOOL' not found in library
configure.ac:36: error: possibly undefined macro: AM_PROG_LIBTOOL
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1


    at /usr/lib/node_modules/bull-repl/node_modules/execa/index.js:231:11
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Promise.all (index 0) {
  code: 1,
  killed: false,
  stdout: '',
  stderr: "configure.ac:36: warning: macro 'AM_PROG_LIBTOOL' not found in library\n" +
    'configure.ac:36: error: possibly undefined macro: AM_PROG_LIBTOOL\n' +
    '      If this token and others are legitimate, please use m4_pattern_allow.\n' +
    '      See the Autoconf documentation.\n' +
    'autoreconf: /usr/bin/autoconf failed with exit status: 1\n',
  failed: true,
  signal: null,
  cmd: '/bin/sh -c autoreconf -fi',
  timedOut: false
}

This is too complicated.

Loving this tool, however I'm needing to pass in:

tls: {
      rejectUnauthorized: false
    }

to bull options (ie: not using cert with --cert) to suit my needs.
Perhaps you could add in a --reject-unauthorized flag to enable this feature?
Happy to make a PR if you agree this would be a useful feature.

Not critical but it feels like these shortcuts should be present as they are on other tools.

• CTRL-N should be the same as up arrow
• CTRL-P should be the same as down arrow

You can see that the node CLI uses these shortcuts 😊 (as well as python... and many others)

example use:

clean -s <failed> 999h

output:

Jobs cleaned

However, when I run again:

failed

It still shows all the jobs that should have been cleaned.

Ran the following commands:
completed -t 1d
completed -t '1d'

Both resulted in the same error:

/usr/local/lib/node_modules/bull-repl/node_modules/node-jq/lib/exec.js:29
                    return reject(Error(stderr));
                                  ^

Error: jq: error (at <stdin>:0): date "2024-02-21T16:08:10.245Z" does not match format "%Y-%m-%dT%H:%M:%S.%3Z"

    at ChildProcess.<anonymous> (/usr/local/lib/node_modules/bull-repl/node_modules/node-jq/lib/exec.js:29:35)
    at ChildProcess.emit (node:events:514:28)
    at maybeClose (node:internal/child_process:1091:16)
    at ChildProcess._handle.onexit (node:internal/child_process:302:5)

It would be nice to see all three timestamps: timestamp, processedOn, and finishedOn.

Especially with large jobs, listing jobs fills up the whole screen and it is difficult to show a list of multiple jobs. It would be nice to have the option to grep the output of completed, failed, etc with something like grep or jq https://stedolan.github.io/jq/manual/#Basicfilters

Hi!

thank you for your tool!
I was trying it, and figured out there is an error with slightly different use cases:

$ bull-repl connect --host $REDIS_HOST myqueue
BULL-REPL> 
connect ECONNREFUSED 127.0.0.1:6379
BULL-REPL> (node:322) UnhandledPromiseRejectionWarning: connect ECONNREFUSED 127.0.0.1:6379
(Use `node --trace-warnings ...` to show where the warning was created)
...

$ bull-repl
BULL-REPL> connect --host 10.170.165.172 myqueue
Connected to 10.170.165.172:6379, prefix: bull, queue: myqueue

BULL-REPL | bull.myqueue>

Do you have a clue on this?
Thank you,

It may be a vulnerability

cc @davestimpert

On v0.15.0 the following command would work:

BULL-REPL> connect -r redis://user:pass@host:port queue

Since v0.16.0 that command fails with:

Error: getaddrinfo ENOTFOUND redis
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:60:26)

I've replicated on all versions since v0.15.0