Sauce Labs Testrunner Toolkit

Sauce Labs Testrunner Toolkit is a containerized testing solution that simplifies user setup, speeds up test execution time and supports native Javascript frameworks like Puppeteer and Playwright for running end-to-end web tests with Sauce Labs.

Requirements

Docker installed
Make sure the Docker daemon is running (e.g. docker info works in your terminal)
A Sauce Labs account; if you don't have one, start a free trial

Install

Using NPM:

npm i -g saucectl

Using brew:

brew tap saucelabs/saucectl
brew install saucectl

or via our one line installer:

curl -L https://saucelabs.github.io/saucectl/install | bash

Would you like to inspect the content of our one line installer? Download it, have a look and execute it:

curl -fsSL -o get_saucectl.sh https://saucelabs.github.io/saucectl/install
chmod 700 get_saucectl.sh
./get_saucectl.sh

Connect to Sauce Labs

A Sauce Labs user name and access key are needed to post the test results. You can get your SAUCE_ACCESS_KEY from Account > User Settings in Sauce Labs.

To authenticate yourself, the following environment variables need to be set:

SAUCE_USERNAME
SAUCE_ACCESS_KEY

You can export them as follows:

export SAUCE_USERNAME=<your-username>
export SAUCE_ACCESS_KEY=<your-access-key>

If you are using a cloud CI/CD tool, we strongly suggest protecting these values through secrets or context variables.

Getting started

saucectl new

This command will ask you to choose one of the frameworks:

After that, a ./sauce/config.yml file and an example test under the tests directory will be created, where you can start working from.

Run your first test

saucectl run

This command will run the example test based on the ./.sauce/config.yml file.

Using saucectl

To learn more about saucectl and its commands and flags, please visit the saucectl repository.

Quick demo

Configuration

saucectl requires a configuration file to know what tests to run and what framework to use. By default, .sauce/config.yml will be the place where saucectl will look for its configuration.

# Simple puppeteer example config
apiVersion: v1
metadata:
  name: Feature XYZ
  tags:
    - e2e
    - release team
    - other tag
  build: Release $CI_COMMIT_SHORT_SHA
files:
  - ./tests/**/*.js
image:
  base: saucelabs/stt-puppeteer-jest-node
  version: latest
sauce:
  region: us-west-1

If you wish to use more than one framework, or to configure different sets of tests separately, you could use any name for the configuration file, and specify it through saucectl run -c ./path/to/config.yml.

As an example, this repository uses two configurations for its pipeline. One for Puppeteer, and one for Playwright.

NOTE: Test files need to match (spec|test) in their file name so they will be automatically detected as testfiles.

Run a single file

files:
  - ./tests/file.spec.js

Run multiple files

files:
  - ./tests/*.spec.js

Run an entire directory

files:
  - ./tests/cypress/

Images

All images are hosted on Docker Hub.

Base image is called testrunner. It contains the tooling necessary to record videos, VNC etc. Plus Chrome, and a Firefox version.

Base image + Playwright contains saucectl with different versions of Playwright.

Base image + Puppeteer contains saucectl with different versions of Puppeteer.

Base image + TestCafe contains saucectl with different versions of TestCafe.

Base image + Cypress contains saucectl with different versions of Cypress.

Examples

The examples here show how Pipeline testing can be used. Try them and find your own use cases. Every testrunner image comes with a preconfigured setup that allows you to focus on writing tests instead of tweaking with the configurations. Our initial testrunner flavors come either with Puppeteer or Playwright as an automation framework. They will start the browser for you and expose the browser object (Puppeteer / Playwright) to the global scope for you to be accessible in the test:

Puppeteer Snippet:

Our Puppeteer testrunner image exposes browser into the global scope which represents an instance of its Browser class. The browser will be initiated and shutdown by the testrunner setup.

describe('saucectl demo test', () => {
	test('should verify title of the page', async () => {
		const page = (await browser.pages())[0]
		await page.goto('https://www.saucedemo.com/');
		expect(await page.title()).toBe('Swag Labs');
	});
});

Playwright Snippet:

The Playwright testrunner image also exposes a global browser variable that represents Playwright's Browser class. In addition to that you also have access to a pre-generated browser context via context as well as to a page frame via page.

describe('saucectl demo test', () => {
	test('should verify title of the page', async () => {
		await page.goto('https://www.saucedemo.com/');
		expect(await page.title()).toBe('Swag Labs');
	});
});

Testcafe Snippet

import { Selector } from 'testcafe';
fixture `Getting Started`
	.page `http://devexpress.github.io/testcafe/example`

const testName = 'testcafe test'
test(testName, async t => {
	await t
		.typeText('#developer-name', 'devx')
		.click('#submit-button')
		.expect(Selector('#article-header').innerText).eql('Thank you, devx!');
});

Cypress Snippet

context('Actions', () => {
	beforeEach(() => {
		cy.visit('https://example.cypress.io/commands/actions')
	})
	it('.type() - type into a DOM element', () => {
		// https://on.cypress.io/type
		cy.get('.action-email')
			.type('[email protected]').should('have.value', '[email protected]')
	})
})

More about the Sauce Labs Testrunner Toolkit

Native JavaScript testing is achieved through the combination of Sauce Labs, Jest, and the JavaScript framework of your choice. In the current beta, the toolkit supports Puppeteer, Playwright and TestCafe. Cypress. This approach gives you the power and expressiveness of different test frameworks with the dashboards, infrastructure, and analytics of Sauce Labs.

The specific framework you want to use to for testing should be based on the types of tests you need to run and the environment in which you are running them. In this beta you will be able to run tests in your existing CI pipeline and benefit from the low latency.

When tests are completed, logs, results, and videos will be uploaded to Sauce Labs to your account. After that, you can review, share, and analyze those results just as you would from any other test executed on Sauce Labs.

To learn more about:

Jest, visit https://jestjs.io/
The Google Puppeteer project, visit https://developers.google.com/web/tools/puppeteer
The Microsoft Playwright project, visit https://github.com/microsoft/playwright
TestCafe, visit https://devexpress.github.io/testcafe/
Cypress, visit https://github.com/cypress-io/cypress

Using `saucectl` in your CI/CD pipeline

This repository includes examples of CI/CD in GitHub Actions Workflows and CircleCI Pipelines. Although the GitHub Actions and CircleCI examples are included, the mechanism works with any CI/CD provider that supports containers.

Running tests in parallel across CI machines

To speed up the test execution in CI, you can parallelize the test execution across CI machines. The concrete setup will depend on your CI provider. Here's an example how to set it up for GitHub Actions.

Please visit here for more infos about the parallelization feature and its limitations.

Collaboration

There is a lot we can imagine doing next. It starts with hearing from you. Submit issues and features here - everything helps!

Contribution

The Sauce Labs Testrunner Toolkit is part of our commitment to a world of digital confidence where each of our digital lives and experiences are magical. If you are thinking about getting involved, please do. This repository is focused on helping people learn how to test their user experience. More is welcome.

nirvananimbusa / testrunner-toolkit Goto Github PK

testrunner-toolkit's Introduction