brianjarita / jobqueue

Jobqueue manages running and scheduling jobs (think Sidekiq or Resque for Go).

Jobqueue

Jobqueue manages running and scheduling jobs (think Sidekiq or Resque).

Prerequisites

You can choose between MySQL and MongoDB as a backend for persistent storage.

Getting started

Get the repository with go get github.com/olivere/jobqueue.

Example:

import (
	"github.com/olivere/jobqueue"
	"github.com/olivere/jobqueue/mysql"
)

// Create a MySQL-based persistent backend.
store, err := mysql.NewStore("root@tcp(127.0.0.1:3306)/jobqueue_e2e?loc=UTC&parseTime=true")
if err != nil {
	panic(err)
}

// Create a manager with the MySQL store and 10 concurrent workers.
m := jobqueue.New(
	jobqueue.SetStore(store),
	jobqueue.SetConcurrency(10),
)

// Register one or more topics and their processor
m.Register("clicks", func(args ...interface{}) error {
	// Handle "clicks" topic
})

// Start the manager
err := m.Start()
if err != nil {
	panic(err)
}

// Add a job: It'll be added to the store and processed eventually.
err = m.Add(&jobqueue.Add{Topic: "clicks", Args: []interface{}{640, 480}})
if err != nil {
	panic(err)
}

...

// Stop the manager, either via Stop/Close (which stops after all workers
// are finished) or CloseWithTimeout (which gracefully waits for a specified
// time span)
err = m.CloseWithTimeout(15 * time.Second) // wait for 15 seconds before forced stop
if err != nil {
	panic(err)
}

See the tests for more details on using jobqueue.

Tests and Web UI

Ensure the tests succeed with go test. You may have to install dependencies.

You can run a simulation of a real worker like so:

$ cd e2e
$ go run main.go

Play with the options: go run e2e/main.go -h.

Then open a second console and watch the worker doing its job:

$ cd ui
$ go run main.go

Then open your web browser at http://127.0.0.1:12345.

License

MIT License. See LICENSE file for details.