ropensci / targets

Function-oriented Make-like declarative workflows for R

targets

Pipeline tools coordinate the pieces of computationally demanding analysis projects. The targets package is a Make-like pipeline tool for statistics and data science in R. The package skips costly runtime for tasks that are already up to date, orchestrates the necessary computation with implicit parallel computing, and abstracts files as R objects. If all the current output matches the current upstream code and data, then the whole pipeline is up to date, and the results are more trustworthy than otherwise.

Philosophy

A pipeline is a computational workflow that does statistics, analytics, or data science. Examples include forecasting customer behavior, simulating a clinical trial, and detecting differential expression from genomics data. A pipeline contains tasks to prepare datasets, run models, and summarize results for a business deliverable or research paper. The methods behind these tasks are user-defined R functions that live in R scripts, ideally in a folder called "R/" in the project. The tasks themselves are called “targets”, and they run the functions and return R objects. The targets package orchestrates the targets and stores the output objects to make your pipeline efficient, painless, and reproducible.

Prerequisites

1. Familiarity with the R programming language, covered in R for Data Science.
2. Data science workflow management techniques.
3. How to write functions to prepare data, analyze data, and summarize results in a data analysis project.

Installation

If you are using targets with crew for distributed computing, it is recommended to use crew version 0.4.0 or higher.

install.packages("crew")

There are multiple ways to install the targets package itself, and both the latest release and the development version are available.

Type	Source	Command
Release	CRAN	install.packages("targets")
Development	GitHub	remotes::install_github("ropensci/targets")
Development	rOpenSci	install.packages("targets", repos = "https://dev.ropensci.org")

Get started in 4 minutes

The 4-minute video at https://vimeo.com/700982360 demonstrates the example pipeline used in the walkthrough and functions chapters of the user manual. Visit https://github.com/wlandau/targets-four-minutes for the code and https://rstudio.cloud/project/3946303 to try out the code in a browser (no download or installation required).

Usage

To create a pipeline of your own:

1. Write R functions for a pipeline and save them to R scripts (ideally in the "R/" folder of your project).
2. Call use_targets() to write key files, including the vital _targets.R file which configures and defines the pipeline.
3. Follow the comments in _targets.R to fill in the details of your specific pipeline.
4. Check the pipeline with tar_visnetwork(), run it with tar_make(), and read output with tar_read(). More functions are available.

Documentation

• User manual: in-depth discussion about how to use targets. The most important chapters are the walkthrough, help guide, and debugging guide.
• Reference website: formal documentation of all user-side functions, the statement of need, and multiple design documents of the internal architecture.
• Developer documentation: software design documents for developers contributing to the deep internal architecture of targets.

Help

Please read the help guide to learn how best to ask for help using targets.

Courses

• Carpentries workshop by Joel Nitta
• Half-day interactive tutorial with a Keras example

Selected talks

English

• Get started with targets in 4 minutes (4:08)
• targets in Action with Joel Nitta and Eric Scott. rOpenSci Community Call (1:09:56).
• targets and crew for clinical trial simulation pipelines. R/Pharma 2023 (1:57:22).
• targets and stantargets for Bayesian model validation pipelines. R/Medicine 2021 (15:33)
• Reproducible computation at scale in R with targets New York Open Statistical Programming Meetup, December 2020 (1:54:28).
• ds-incubator series, 2021 by Mauro Lepore.

Español

• Introducción a targets. Irene Cruz, R-Ladies Barcelona, 2021-05-25 (1:25:12).

日本語

• Bio”Pack”athon, 2022-03-31 (1:04:10), Joel Nitta

Example projects

• Four-minute example
• Minimal example
• Machine learning with Keras
• Validate a minimal Stan model
• Using Target Markdown and stantargets to validate a Bayesian longitudinal model for clinical trial data analysis
• Shiny app that runs a pipeline
• Deploy a pipeline to RStudio Connect

Apps

• tar_watch(): a built-in Shiny app to visualize progress while a pipeline is running. Available as a Shiny module via tar_watch_ui() and tar_watch_server().
• targetsketch: a Shiny app to help sketch pipelines (app, source).

Deployment

• https://solutions.rstudio.com/r/workflows/ explains how to deploy a pipeline to RStudio Connect (example code).
• tar_github_actions() sets up a pipeline to run on GitHub Actions. The minimal example demonstrates this approach.

Extending and customizing targets

• R Targetopia: a collection of R packages that extend targets. These packages simplify pipeline construction for specific fields of Statistics and data science.
• Target factories: a programming technique to write specialized interfaces for custom pipelines. Posts here and here describe how.

Code of conduct

Please note that this package is released with a Contributor Code of Conduct.

Citation

citation("targets")
To cite targets in publications use:

  Landau, W. M., (2021). The targets R package: a dynamic Make-like
  function-oriented pipeline toolkit for reproducibility and
  high-performance computing. Journal of Open Source Software, 6(57),
  2959, https://doi.org/10.21105/joss.02959

A BibTeX entry for LaTeX users is

  @Article{,
    title = {The targets R package: a dynamic Make-like function-oriented pipeline toolkit for reproducibility and high-performance computing},
    author = {William Michael Landau},
    journal = {Journal of Open Source Software},
    year = {2021},
    volume = {6},
    number = {57},
    pages = {2959},
    url = {https://doi.org/10.21105/joss.02959},
  }

targets's Issues

Full runthrough of an example workflow

Implement a run_targets() function to run the correct targets and the correct order and store the results. Use the "speed" memory strategy from drake. Then move on to #21 and #22.

Ensure call objects load their packages

Mock up a plan with maps

We are almost ready for #21.

result class

re #4

Visualization

Decompose the results class into metrics and value

We spend a lot of time working with value without error messages etc. The latter awkwardly get in the way.

dynamic file format

Implement early on.

put interactive tests in different subfolder of tests/

Memory management at the target level

• New "targets_memory" class, a high-performant abstraction of an environment.
• New "targets_cache" class with multiple layers of "targets_memory" objects.
• Give each target its own "targets_cache" object so it exhibits crate-like behavior on HPC.

default file for targets

re #4, #6. format = "file" should signal that the path is the return value.

settings class

re #4

Update design vignette

• No bundles.
• We draw downstream edges from both buds and branches to everything downstream.

Skip up-to-date targets

That's the whole point, right? But it can't happen until #5 etc. Got pre-work to tackle first.

Remove downstream queue

And rename "upstream queue" to just "queue".

Functions to give the name of the target and default file path mid-build.

• tar_name()
• tar_path()

Powered by a value in a private pkg environment.

Cancel targets mid-build

Use custom error condition like in drake. This should avoid the need for fancy condition/change triggers.

Look for superfluous methods

Grep for methods that appear only once in R/*.R.

Progress monitoring

Needs #5.

target class

• cmd field.
• All dependency hashes:
  • cmd$hash().
  • Hashes of the dependencies of cmd from hash_envir().
  • Hashes of the targets directly upstream.
• Names of the targets directly upstream.
• Delegate pkgs() and eval() to cmd.
• Field for the return value.
• Other metadata such as runtime and seed. Need to think about organization.

Encapsulate return values

• Is a value NULL because it was not set yet or NULL because the user returned NULL?
• Should be responsible for formatting itself.

Test call objects

run with seed

Dependency-aware childlists

Patterns will need to know which dependencies go with which branches.

Subclasses of files

Dynamic files and static files need to behave differently.

$find_value() method for the target class

qs format

Implement early on

avoid duplicate validation code

Condense into reusable assertions.

Static analysis of functions and other globals

1. Create a dependency graph of functions and other globals in the environment using static code analysis.
2. Hash all the objects in topological order. The hash of a function is the hash of the deparsed body together with the hashes of all the immediate upstream neighbors.
3. Keep track of the result in a named character vector.

Load data from workers

Need some sort of bootstrapping

stem$produce_children()

Should use the childlist to produce buds.

DSL

Custom deps field and get_deps() method for branches

Should list the upstream children (buds and branches) so they get loaded into memory at the right time.

deplist class

internal data frame with columns "name", "direction", "branching" (logical), and "class". Plus methods to interrogate the dependency structure for the sake of build decisions.

Specialized database structure

For #5 and #30.

• Should append to the rows of a file like a txtq. But since the index of the head row is session-specific, we probably only need the database file itself (no lockfile either).
• We only append rows to make target-specific updates, so we know later rows are chronologically later.
• When updating targets, only transact every 1 or so seconds (or a customizable interval).
• Clean up at the beginning and end of make() sessions which just means reading in the DB and removing early rows to deduplicate targets.
• Cleaning and pruning will involve this data structure.

tar_knitr(): literate programming integration

One of the DSLs should be able to inline them into commands so we can detect dependencies.

Centralized metadata storage

For performance. Need some kind of database.

Buds

• Implement a new "targets_bud" class to make slices of stems into targets. This will help us map() and cross() over stems.
• Implement stem$produce_children() to create a named list of buds.

API

Options

• tar_option()
• tar_options()

Dependencies

• tar_knitr()
• tar_package()

Pipeline

• tar_cue()
• tar_pipeline()
• tar_target()
• tar_target_external()
• tar_validate()

Visuals

• tar_glimpse()
• tar_graph()

Run

• tar_make()
• tar_make_clustermq()
• tar_make_future()

Runtime

• tar_cancel()
• tar_name()
• tar_path()

Data

• tar_load()
• tar_meta()
• tar_progress()
• tar_read()

Clean

• tar_clean()
• tar_deduplicate()
• tar_destroy()
• tar_invalidate()
• tar_prune()

Memory strategies at the whole workflow level

Related: #19. We should think about when objects should be kept in memory or released. The advantage of keeping them for longer is we do not need to access storage, but that could make memory blow up.

R6 $finalize() methods

For memory efficiency.

Avoid the centralized store

Use qs on individual targets

Branching

In targets, all branching will be dynamic branching.

Let's tackle the infrastructure before we get too far ahead. One idea: we could use target iterators/generators. Maybe the abstract factory pattern?

Bundles

When we come to a pattern in a pipeline, we should do the following:

1. Add the branches to the graph and priority queue.
2. Replace the pattern with a bundle of the same name.

A bundle is a target whose rule is just to aggregate the children. We may want different kinds of bundles based on the iteration type: list() vs vec_c().

target by target memory strategies

The whole downstream queue thing is not working, and it's a black box.

Order of implementation for the basic components

1. data store. Should accept a target and store the value.
2. DSL
3. Priority queue
4. Naive runghtroughs
5. Target metadata
6. Metadata store
7. Skipping up-to-date stuff (last)

HPC

clustermq only. I would think about future, but right now it still totally relies on batchtools for distributed computing.

Put seed in cmd class

The target class has too much to worry about.

Scheduler

Requires a graph, but we should not need to hold on to the graph to operate the queue. That last point is a weakness of drake's internals.

Pipeline-level settings

Totally global:

• names
• reporter
• algorithm
• workers
• garbage collection
• template

Target-level defaults:

• packages
• library
• memory
• deployment
• storage
• retrieval
• format
• failure mode
• trigger

hex logo