isabella232 / circleci-orb

Install, cache and run Cypress.io tests on CircleCI with minimal configuration.

Cypress CircleCI Orb

The Cypress CircleCI Orb is a piece of configuration set in your circle.yml file to correctly install, cache and run Cypress.io tests on CircleCI with very little effort. See this orb in CircleCI Registry.

Contents

• How to enable orbs
• Lots of examples
  • simple
  • with recording on Dashboard
  • with parallel tests
  • other examples
• Recipes
  • Install, test and release
  • Install, then run two different test jobs
  • Custom test command
  • other recipes like multiple jobs, testing on Windows, etc.
• Naming
• Jobs and executors in this orb
• Orb versions
• Effective config
• Development and license

How to enable

Note ⚠️: To use CircleCI Orbs in your projects, you need to enable two settings:

• From organization settings allow using uncertified orbs Settings -> Security -> Allow uncertified orbs
• From the project's settings allow beta features Settings -> Advanced Settings -> Enable pipelines

See the official CircleCI documentation.

Examples

Each example below should be placed into circle.yml or .circleci/config.yml file:

simple

Install dependencies (using npm ci) and run all Cypress tests:

# to use orbs, must use version >= 2.1
version: 2.1
orbs:
  # import Cypress orb by specifying an exact version x.y.z
  # or the latest version 1.x.x using "@1" syntax
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      # "cypress" is the name of the imported orb
      # "run" is the name of the job defined in Cypress orb
      - cypress/run

See cypress-io/circleci-orb-example

record on Dashboard

Runs all Cypress tests and records them on the Cypress Dashboard:

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/run:
          record: true

Note: recording test results and artifacts requires Cypress Dashboard account. You should also set your record key as CYPRESS_RECORD_KEY environment variable in the CircleCI project.

parallel

A more complex project that needs to install dependencies, build an application and run tests across 4 CI machines in parallel may have:

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      # first get the source code and install npm dependencies
      - cypress/install:
          # run a custom app build step
          build: 'npm run build'
      - cypress/run:
          # make sure app has been installed and built
          # before running tests across multiple machines
          # this avoids installing same dependencies 10 times
          requires:
            - cypress/install
          record: true # record results on Cypress Dashboard
          parallel: true # split all specs across machines
          parallelism: 4 # use 4 CircleCI machines to finish quickly
          group: 'all tests' # name this group "all tests" on the dashboard
          start: 'npm start' # start server before running tests

In all cases, you are using run and install job definitions that Cypress provides inside the orb. Using the orb brings simplicity and static checks of parameters to CircleCI configuration.

Note: recording test results and spec parallelization requires Cypress Dashboard account. You should also set your record key as CYPRESS_RECORD_KEY environment variable in the CircleCI project.

other examples

• install dependencies using Yarn
• running tests using Node 14
• running tests using Chrome browser
• start server before running tests
• wait for server to respond before starting tests
• parallel run across two machines
• build application after install
• running several groups of tests
• running another job after tests
• building using orb on Mac and Linux
• use custom executor
• set additional environment variables
• install private NPM dependencies
• print info after install
• install extra tools
• complete NPM module publishing example
• store test reports on Circle
• store screenshots and videos on Circle
• store other folders as artifacts on Circle
• use your own custom command to run tests
• skip saving workspace
• run commands in a subfolder of a monorepo
• cache using custom key in a monorepo
• config
• custom configuration file
• tag recorded run
• run different tests depending on the branch
• turn on specific DEBUG logs
• use custom ci-build-id

All examples are in docs/examples.md and are generated from the src/orb.yml file.

Also take a look at cypress-io/cypress-example-circleci-orb and cypress-io/cypress-example-kitchensink. You can find more examples under GitHub topic cypress-orb-example.

Naming

When importing this orb, we suggest using local name "cypress" for consistency.

version: 2.1
orbs:
  #          ↱ official orb name in the registry (org + name)
  cypress: cypress-io/cypress@1
  #  ↳ your local name for the imported orb
workflows:
  build:
    jobs:
      #   ↱ local orb name
      - cypress/run
      #          ↳ job "run" defined in the orb

You can of course use another local name

version: 2.1
orbs:
  #             ↱ official orb name in the registry (org + name)
  e2eCypress: cypress-io/cypress@1
  #  ↳ your local name for the imported orb
workflows:
  build:
    jobs:
      #   ↱ local orb name
      - e2eCypress/run
      #             ↳ job "run" defined in the orb

We suggest importing the orb under the local name cypress and giving names to each job using the name parameter.

version: 2.1
orbs:
  #          ↱ official orb name in the registry (org + name)
  cypress: cypress-io/cypress@1
  #  ↳ your local name for the imported orb
workflows:
  build:
    jobs:
      - cypress/run:
          name: E2E tests

See Recipes for more examples.

Jobs and executors

See docs/jobs.md and docs/executors.md for a full list of public jobs and executors that this orb provides. For example, if you want to execute tests using Node 14

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/run:
          executor: cypress/base-14

For more examples, search for executor in the docs/examples.md page.

The CircleCI Orb exports the following job definitions to be used by the user projects:

run

This job allows you to run Cypress tests on a one or more machines, record screenshots and videos, use the custom Docker image, etc.

A typical example:

# to use orbs, must use version >= 2.1
version: 2.1
orbs:
  # import Cypress orb by specifying an exact version x.y.z
  # or the latest version 1.x.x
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      # checks out code, installs npm dependencies
      # and runs all Cypress tests and records results on Cypress Dashboard
      # cypress/run job comes from "cypress" orb imported above
      - cypress/run:
          record: true

See all its parameters at the cypress/run job example.

install

⚠️ Warning: this job is only necessary if you plan to execute the run job later. If you only want to run all tests on a single machine, then you do not need a separate install job.

Sometimes you need to install the project's npm dependencies and build the application once before running tests, especially if you run the tests on multiple machines in parallel. For example:

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      # install dependencies first (on 1 machine)
      - cypress/install
      # now run tests
      - cypress/run:
          # give this job a custom name for clarity
          name: 'end-to-end tests'
          requires:
            # use previously installed dependencies
            # to avoid installing them on each machine running tests
            - cypress/install
          record: true # record results to Cypress Dashboard
          parallel: true # run tests in parallel
          parallelism: 3 # use 3 CircleCI machines
          group: 3 machines # name this group "3 machines"

See available parameters at the cypress/install job example.

To better understand why we use a separate install job, take a look at the workflow diagram below.

The first job install runs on a single machine, and usually is very fast because it uses previously cached npm modules and Cypress binary to avoid reinstalling them. The second job run can run on multiple machines (in this case it runs on 3 machines), and uses workspace created by the install job to get 3 identical file systems before running tests. You can see the 3 parallel runs by clicking on the run job.

Versions

Cypress orb is versioned so you can be sure that the configuration will not suddenly change as we change orb commands. We follow semantic versioning to make sure you can upgrade project configuration to minor and patch versions without breaking changes. See CircleCI Orb versioning documentation.

You can find all changes and published orb versions for Cypress orb at cypress-io/circleci-orb/releases.

We are using cypress-io/cypress@1 version in our examples, so you get the latest published orb version 1.x.x. But we recommend locking it down to an exact version to prevent unexpected changes from suddenly breaking your builds.

Lock files

This orb requires the repository to use a lock file like package-lock.json or yarn.lock. These lock files are recommended for consistent repeatable installations of dependencies on the CI machines. If a lock file is not found, the Orb shows an error and stops.

Effective config

CircleCI expands orbs in your config file before running the workflows. You can see this effective config in their UI

If you install Circle local CLI, you can see the final effective configuration your project resolves to by running circleci config process <config filename> from the terminal.

For example, if your current CircleCI configuration file is .circleci/config.yml and it contains the following:

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/run

The fully resolved configuration will show:

$ circleci config process .circleci/config.yml
# Orb 'cypress-io/cypress@1' resolved to 'cypress-io/[email protected]'
version: 2
jobs:
  cypress/run:
    docker:
    - image: cypress/base:12.14.0
    parallelism: 1
    steps:
    - checkout
    - restore_cache:
        keys:
        - cache-{{ .Branch }}-{{ checksum "package.json" }}
    - run:
        name: Npm CI
        command: npm ci
    - run:
        command: npx cypress verify
    - save_cache:
        key: cache-{{ .Branch }}-{{ checksum "package.json" }}
        paths:
        - ~/.npm
        - ~/.cache
    - persist_to_workspace:
        root: ~/
        paths:
        - project
        - .cache/Cypress
    - attach_workspace:
        at: ~/
    - run:
        name: Run Cypress tests
        command: 'npx cypress run'
workflows:
  build:
    jobs:
    - cypress/run
  version: 2

Ejecting

If you want to customize the orb configuration, you can save and tweak the output of the circleci config process ... to suit your needs.

⚠️ Warning: there is no automated way to go from the "ejected" configuration back to using the orb.

Development

If you want to develop this orb and publish new versions, see our Contributing Guide.

License

This project is licensed under the terms of the MIT license.