hartl3y94 / cli-1

A tool for testing and debugging Dependabot update jobs.

The dependabot CLI is a tool for testing and debugging Dependabot update jobs.

Installation

You can download a pre-built binary from the releases page.

If you have the gh command available, you can install the latest release of dependabot using the following command (gist source):

$ gh gist view --raw e09e1ecd76d5573e0517a7622009f06f | bash

Requirements

• Docker

Contributing

Check out our contributing guidelines for instructions on building the project locally, sharing feedback, and submitting pull requests.

Usage

$ dependabot
Run Dependabot jobs from the command line.

Usage:
  dependabot [command]

Examples:
  $ dependabot update go_modules rsc/quote --dry-run
  $ dependabot test -f scenario.yaml

Available Commands:
  help        Help about any command
  test        Test scenarios
  update      Perform update job

Flags:
  -h, --help                   help for dependabot
  -v, --version                version for dependabot

Use "dependabot [command] --help" for more information about a command.

dependabot update

Run the update subcommand with the --dry-run flag to simulate a Dependabot update job for the provided ecosystem and repo (without actually creating any PRs).

$ dependabot update go_modules rsc/quote --dry-run
# ...
+----------------------------------------------------+
|        Changes to Dependabot Pull Requests         |
+---------+------------------------------------------+
| created | rsc.io/quote/v3 ( from 3.0.0 to 3.1.0 )  |
| created | rsc.io/sampler ( from 1.3.0 to 1.99.99 ) |
+---------+------------------------------------------+

The first argument specifies the package manager (e.g. go_modules, bundler, npm_and_yarn, or python). Available values are defined in dependabot-core; by convention, each ecosystem registers itself according to the name of its top-level subdirectory in the repo.

The second argument is the repository name with owner (e.g. dependabot/cli for this repo).

By default, repositories are fetched from GitHub.com. To override this, set the --provider / -p option to azure, bitbucket, codecommit, or gitlab.

To update dependencies in a subdirectory, specify a path with the --directory / -d option.

Set the LOCAL_GITHUB_ACCESS_TOKEN environment variable to a Personal Access Token (PAT), and the CLI will pass that token to the proxy to authenticate API requests to GitHub (for example, to access private repositories or packages).

Job description file

The command-line interface for the update subcommand provides only a subset of the available options for a Dependabot update job. To perform security updates or authenticate against a private registry, you can pass a job description to the update subcommand using the --file / -f option (this replaces the package manager and repository name arguments).

$ dependabot update -f job.yaml --dry-run

# job.yaml
job:
    package_manager: npm_and_yarn
    allowed_updates:
      - update-type: all
    security_advisories:
      - dependency-name: express
        affected-versions:
          - <5.0.0
        patched-versions: []
        unaffected-versions: []
    security_updates_only: true
    source:
        provider: github
        repo: dependabot/smoke-tests
        directory: /
        commit: 66115359e6f6cc3af6a661c5d5ae803720b98cb8
credentials:
  - type: npm_registry
    registry: https://npm.pkg.github.com
    token: $LOCAL_GITHUB_ACCESS_TOKEN

This example describes an update job responsive to a hypothetical security advisory affecting express package releases earlier than version 5.0.0. When performing this job, Dependabot will consult the private registry specified using the provided credentials instead of the default NPM registry.

Before running an update job, the dependabot CLI replaces any $-prefixed values in the YAML file with values from the environment. (e.g. $LOCAL_GITHUB_ACCESS_TOKEN).

Note

The job description file format isn't documented publicly, but you can find examples in the testdata directory and check out the Job class in dependabot-core.

How it works

When you run the update subcommand, the CLI does the following:

1. Pulls the updater and proxy images from the container registry
2. Creates and configures container networks so the updater communicates exclusively through the proxy
3. Starts the proxy
4. Starts the updater, using the job description as input
5. Records calls made by the updater to create and manage pull requests
6. Writes recorded calls as YAML (if --output / -o option is specified)

sequenceDiagram
    CLI->>Proxy: Starts the proxy
    CLI->>Updater: Starts the updater
    Updater->>GitHub: Fetches repo
    loop
    Updater->>Registry: Fetches package information
    Updater->>CLI: Records calls to create or updates PRs
    end
    CLI->>YAML file: Writes recorded calls to output file (if specified)

All network requests made by the updater go through the proxy. The proxy injects credentials into outbound requests so that the updater doesn't have access to secrets. This isolation is especially important for package managers that run untrusted code during an update job, such as when evaluating manifest files or executing install scripts.

dependabot test

Run the test subcommand with a scenario file specified by the --file / -f option to test the expected behavior for a Dependabot update job.

$ dependabot test -f scenario.yaml
# ...
+------------------------------------------+
|   Changes to Dependabot Pull Requests    |
+---------+--------------------------------+
| created | ubuntu ( from 17.04 to 22.04 ) |
+---------+--------------------------------+

time="2022-09-28T08:15:26Z" level=info msg="15/15 calls cached (100%)"

Scenario file

A scenario file describes the input and expected output of a Dependabot job.

# scenario.yaml
input:
    job:
        package-manager: docker
        allowed-updates:
          - update-type: all
        ignore-conditions:
          - dependency-name: ubuntu
            source: tests/smoke-docker.yaml
            version-requirement: '>22.04'
        source:
            provider: github
            repo: dependabot/smoke-tests
            directory: /
            commit: 832e37c1a7a4ef89feb9dc7cfa06f62205191994
output:
  - type: create_pull_request
    expect:
        data:
            base-commit-sha: 832e37c1a7a4ef89feb9dc7cfa06f62205191994
            dependencies:
              - name: ubuntu
                previous-requirements:
                  - file: Dockerfile
                    groups: []
                    requirement: null
                    source:
                        tag: "17.04"
                previous-version: "17.04"
                requirements:
                  - file: Dockerfile
                    groups: []
                    requirement: null
                    source:
                        tag: "22.04"
                version: "22.04"

This example scenario describes the expected behavior for Dependabot to update the base image of a Dockerfile from ubuntu:17.04 to ubuntu:22.04.

• The input field consists of a job and any credentials. (this is equivalent a job description file).
• The output field comprises an array of expectation objects. These correspond to requests made by the updater to the Dependabot API service when performing an update job.

Note

The scenario file format isn't documented publicly, but you can find examples in the smoke-tests repo and check the Job class in dependabot-core.

Producing a test

To produce a scenario file that tests Dependabot behavior for a given repo, run the update subcommand and set the --output / -o option to a file path.

$ dependabot update go_modules rsc/quote --dry-run -o go-scenario.yml

Run the test subcommand for the generated scenario file, specifying a cache directory with the --cache option.

$ dependabot test -f go-scenario.yml --cache ./tmp/cache

While performing the update job, the CLI writes cached responses to requests in the specified directory.

Run the above command a second time, and you should see a line that looks like this at the bottom of the output:

time="2022-09-28T08:14:01Z" level=info msg="117/117 calls cached (100%)"

When the cache coverage for a scenario is 100%, subsequent runs of the test subcommand are most likely to be fast and deterministic. Any cache misses indicate an external request made by the updater, which may cause tests to fail unexpectedly (for example, when a new version of a package is released).

Troubleshooting

"Docker daemon not running"

failed to pull ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest:
Error response from daemon: dial unix docker.raw.sock: connect: no such file or directory

The CLI requires Docker to be running on your machine. Follow the instructions on Docker's website to get the latest version of Docker installed and running.

You can verify that Docker is running locally with the following command:

$ docker --version

"Network internet is ambiguous"

failed to start container: Error response from daemon: network internet is ambiguous (2 matches found on name)

This error can occur when the CLI exits before having an opportunity to clean up (e.g. terminating with ^C). Run the following command to remove all unused networks:

$ docker network prune