GitHubHelp

mgurlitz/cmd

A minimal Redis-backed job queue with workers, locking, retries, and filtering

★ 0Forks 0RustGitHub ↗Compare

README

cmd

A minimal Redis-backed job queue with workers, locking, retries, and filtering.

Features

  • Priority queues via Redis sorted sets (scores)
  • Distributed locking prevents duplicate processing
  • Per-worker filtering dynamically route jobs to specific workers
  • Retries with poison queue failed jobs retry up to N times, then move to dead-letter
  • Graceful shutdown handles SIGINT/SIGTERM cleanly
  • Metrics & dashboard Prometheus endpoint + web UI with editable filters

Quick Start

# Add items to a queue
cmd add myqueue item1 item2 item3

# Add with priority (lower scores processed first)
cmd add myqueue --score 10 high-priority-item
cmd add myqueue --score 100 low-priority-item

# Start a worker that runs "process.sh" for each item
cmd server myqueue ./process.sh

# List queue contents
cmd list myqueue
cmd list myqueue --detail  # shows which items are locked and by whom

# Start metrics dashboard
cmd metrics myqueue --port 9090

Commands

add - Add items to queue

cmd add <queue> [items...]
cmd add <queue> --score <n> [items...]    # priority (lower = first)
cmd add <queue> --transient [items...]    # pub/sub only, not persisted

list - List queue contents

cmd list <queue>
cmd list <queue> --detail          # show lock info
cmd list <queue> --filter <pat>    # filter by substring
cmd list <queue> --number          # show item count
cmd list <queue> --server          # list active workers instead

get - Check if item exists

cmd get <queue> <item>

remove - Remove items

cmd remove <queue> <item1> <item2>
cmd remove <queue> --all           # clear entire queue

server - Start a worker

cmd server <queue> <command> [args...]

# Options:
#   --filter <pat>       only process items matching pattern
#   --max-retries <n>    retry failed items N times (default: 5)
#   --delay-between <d>  wait between items (default: 3s)
#   --delay-start <d>    wait before starting
#   --reverse            process in reverse order
#   --stop-when-empty    exit when queue is empty
#   --transient          use pub/sub mode
#   --verbose            print debug info
#   --print              print each item being processed

Example:

# Process video files, only mp4s, with 10s between jobs
cmd server videos ./transcode.sh --filter .mp4 --delay-between 10s

set-filter - Change worker filter at runtime

cmd set-filter <queue> <server-id> <filter>
cmd set-filter <queue> <server-id> --remove

lock - Manage locks

cmd lock <queue> --list            # show locked items
cmd lock <queue> --list --detail   # with timestamps
cmd lock <queue> --add <items...>  # manually lock
cmd lock <queue> --remove <items...>
cmd lock <queue> --delete          # clear all locks
cmd lock <queue> --prune 1h        # remove locks older than 1 hour

stop - Signal workers to stop

cmd stop <queue>                   # stop all workers on this queue
cmd stop <queue> <hostname>        # stop workers on specific host
cmd stop <queue> --after 5m        # stop after 5 minutes

metrics - Start monitoring server

cmd metrics <queue1> [queue2...] --port 9090

Opens:

  • http://localhost:9090/ - Web dashboard
  • http://localhost:9090/metrics - Prometheus endpoint

Metrics

The /metrics endpoint exposes Prometheus-format metrics:

Metric Type Description
cmd_queue_size{queue="..."} gauge Total items in queue
cmd_queue_active{queue="..."} gauge Items currently being processed
cmd_queue_pending{queue="..."} gauge Items waiting to be processed
cmd_queue_poison{queue="..."} gauge Items in dead-letter queue
cmd_workers_active{queue="..."} gauge Number of active workers

Example Prometheus scrape config:

scrape_configs:
  - job_name: 'cmd'
    static_configs:
      - targets: ['localhost:9090']

Web Dashboard

The dashboard at / shows for each queue:

  • Metrics summary (size, pending, active, poison, workers)
  • Pending items (paginated)
  • Active items with server name and duration
  • Workers with editable filters and cancel buttons

Auto-refreshes every 10 seconds.

Redis Data Model

Key Type Purpose
{queue} SET or ZSET Queue items (ZSET if scores used)
{queue}-active HASH Locked items: item -> "server timestamp"
{queue}-server ZSET Active workers: server-id -> timestamp
{queue}-server-filter HASH Per-worker filters: server-id -> pattern
{queue}-poison ZSET Dead-letter queue: item -> timestamp
{queue}-stop HASH Stop signals
{queue}:{item} HASH Per-item retry counter

Development

nix develop
cargo build
cargo run -- --help

License

MIT

Contributors

mgurlitz

Issues