funcguy / jobrunr

An extremely easy way to perform background processing on the JVM. Backed by persistent storage. Open and free for commercial use.

The ultimate library to perform background processing on the JVM.
Dead simple API. Extensible. Reliable.
Distributed and backed by persistent storage.
Open and free for commercial use.

   
   
   
    

Overview

Incredibly easy way to perform fire-and-forget, delayed and recurring jobs inside Java applications using only Java 8 lambda's. CPU and I/O intensive, long-running and short-running jobs are supported. Persistent storage is done via either RDBMS (e.g. Postgres, MariaDB/MySQL and Oracle) or NoSQL (MongoDB and Redis).

JobRunr provides a unified programming model to handle background tasks in a reliable way and runs them on shared hosting, dedicated hosting or in the cloud (hello Kubernetes) within a JVM instance.

Features

• Simple: just use Java 8 lambda's to create a background job
• Distributed & cluster-friendly: guarantees execution by single scheduler instance using optimistic locking.
• Persistent jobs: using either a RDMBS (four tables and a view) or a noSQL data store.
• Embeddable: built to be embedded in existing applications.
• Minimal dependencies: (ASM, slf4j and either jackson and jackson-datatype-jsr310 or gson)

Usage scenario's

Some scenario's where it may be a good fit:

• within a REST api return response to client immediately and perform long-running job in the background
• mass notifications/newsletters
• calculations of wages and the creation of the resulting documents
• batch import from xml, csv or json
• creation of archives
• firing off web hooks
• image/video processing
• purging temporary files
• recurring automated reports
• database maintenance
• updating elasticsearch/solr after data changes
• …and so on

You can start small and process jobs within your webapp or scale horizontally and add as many background job servers as you want to handle a peak of jobs. JobRunr will distribute the load over all the servers for you. JobRunr is also fault-tolerant - is an external webservice down? No worries, the job is automatically retried 10-times with a smart back-off policy.

JobRunr is a Java alternative to HangFire, Resque, Sidekiq, delayed_job, Celery and is similar to Quartz and Spring Task Scheduler.

Screenshots

   
   
   

Usage

Fire-and-forget tasks

Dedicated worker pool threads execute queued background jobs as soon as possible, shortening your request's processing time.

BackgroundJob.enqueue(() -> System.out.println("Simple!"));

Delayed tasks

Scheduled background jobs are executed only after a given amount of time.

BackgroundJob.schedule(() -> System.out.println("Reliable!"), Instant.now().plusHours(5));

Recurring tasks

Recurring jobs have never been simpler; just call the following method to perform any kind of recurring task using the CRON expressions.

BackgroundJob.scheduleRecurringly("my-recurring-job", () -> service.doWork(), Cron.daily());

Process background tasks inside a web application…

You can process background tasks in any web application and we have thorough support for Spring - JobRunr is reliable to process your background jobs within a web application.

… or anywhere else

Like a Spring Console Application, wrapped in a docker container, that keeps running forever and polls for new background jobs.

See https://www.jobrunr.io for more info.

Installation

Using Maven?

JobRunr is available in Maven Central - all you need to do is add the following dependency:

<dependency>
   <groupId>org.jobrunr</groupId>
   <artifactId>jobrunr</artifactId>
   <version>0.9.7</version>
</dependency>

Using Gradle?

Just add the dependency to JobRunr:

implementation 'org.jobrunr:jobrunr:0.9.7'

Configuration

Do you like to work Spring based?

@Bean
public BackgroundJobServer backgroundJobServer(StorageProvider storageProvider, JobActivator jobActivator) {
    final BackgroundJobServer backgroundJobServer = new BackgroundJobServer(storageProvider, jobActivator);
    backgroundJobServer.start();
    return backgroundJobServer;
}

@Bean
public JobActivator jobActivator(ApplicationContext applicationContext) {
    return applicationContext::getBean;
}

@Bean
public JobScheduler jobScheduler(StorageProvider storageProvider) {
    return new JobScheduler(storageProvider);
}

@Bean
public StorageProvider storageProvider(JobMapper jobMapper) {
    final SQLiteDataSource dataSource = new SQLiteDataSource();
    dataSource.setUrl("jdbc:sqlite:" + Paths.get(System.getProperty("java.io.tmpdir"), "jobrunr.db"));
    final SqLiteStorageProvider sqLiteStorageProvider = new SqLiteStorageProvider(dataSource);
    sqLiteStorageProvider.setJobMapper(jobMapper);
    return sqLiteStorageProvider;
}

@Bean
public JobMapper jobMapper(JsonMapper jsonMapper) {
    return new JobMapper(jsonMapper);
}

@Bean
public JsonMapper jsonMapper() {
    return new JacksonJsonMapper(); // or GsonMapper()
}

Or do you prefer a fluent API?

Define a javax.sql.DataSource and put the following code on startup:

@SpringBootApplication
@Import(JobRunrStorageConfiguration.class)
public class WebApplication {

    public static void main(String[] args) {
        SpringApplication.run(WebApplication.class, args);
    }

    @Bean
    public JobScheduler initJobRunr(ApplicationContext applicationContext) {
        return JobRunr.configure()
                .useStorageProvider(SqlStorageProviderFactory
                          .using(applicationContext.getBean(DataSource.class)))
                .useJobActivator(applicationContext::getBean)
                .useDefaultBackgroundJobServer()
                .useDashboard()
                .initialize();
    }
}

Contributing

See CONTRIBUTING for details on submitting patches and the contribution workflow.

How can I contribute?

• Take a look at issues with tag called Good first issue
• Join the discussion on gitter.
• Answer questions on issues.
• Fix bugs reported on issues, and send us pull request.