coderaiser / supertape

📼 Simplest high speed testing

📼 Supertape

📼 Supertape is Tape-inspired TAP-compatible simplest high speed test runner with superpowers. It's designed to be as compatible as possible with tape while still having some key improvements, such as:

• the ability to work with ESM Modules (take a look at mock-import for mocking and 🎩ESCover for coverage)

• a number of built-in pretty output formatters

• the ability to extend

• showing colored diff when using the t.equal() and t.deepEqual() assertion operators

• detailed stack traces for async functions

• multiple test.only's

• smart timeouts for long running tests 🏃‍♂️

• more natural assertions: expected, result -> result, expected:

  t.equal(error.message, 'hello world', `expected error.message to be 'hello world'`);

• ability to generate tests with ♨️Speca

📼 Supertape doesn't contain:

• assertion aliases, making the available operators far more concise;
• es3 code and lots of ponyfills;
• t.throws(), t.doesNotThrow() - use tryCatch or tryToCatch with t.equal() instead;
• t.plan();

For a list of all built-in assertions, see Operators.

How 📼Supertape test looks like?

You can use both CommonJS and ESM, here is ESM example:

import {test} from 'supertape';

const sump = (a, b) => a + b;

test('calc: sum', (t) => {
    const result = sum(1, 2);
    const expected = 3;
    
    t.equal(result, expected);
    t.end();
});

Install

npm i supertape -D

Usage

Usage: supertape [options] [path]
Options
   -h, --help                  display this help and exit
   -v, --version               output version information and exit
   -f, --format                use a specific output format - default: progress-bar/tap on CI
   -r, --require               require module
   --no-check-scopes           do not check that messages contains scope: 'scope: message'
   --no-check-assertions-count do not check that assertion count is no more then 1
   --no-check-duplicates       do not check messages for duplicates
   --no-worker                 disable worker thread

Environment variables

• SUPERTAPE_TIMEOUT - timeout for long running processes, defaults to 3000 (3 seconds);
• SUPERTAPE_CHECK_DUPLICATES - toggle check duplicates;
• SUPERTAPE_CHECK_SCOPES - check that test message has a scope: scope: subject;
• SUPERTAPE_CHECK_ASSERTIONS_COUNT - check that assertion count is no more then 1;
• SUPERTAPE_CHECK_SKIPED - check that skiped count equal to 0, exit with status code;
• SUPERTAPE_LOAD_LOOP_TIMEOUT - timeout for load tests, defaults to 5ms, when mocha used as runner - 50ms optimal;

test('tape: error', (t) => {
    t.equal(error.code, 'ENOENT');
    t.end();
});

🤷 How to migrate from tape?

🐊 + 📼 = ❤️

You can convert your codebase from Tape, or Jest to 📼Supertape with help of 🐊Putout, which has built-in @putout/plugin-tape, with a lots of rules that helps to write and maintain tests of the highest possible quality.

Here is result example.

ESLint rules

eslint-plugin-putout has a couple rules for 📼Supertape:

• ✅ remove-newline-before-t-end
• ✅ add-newline-before-assertion
• ✅ add-newline-between-tests

Validation checks

To up the quality of your tests even higher, 📼Supertape has built-in checks. When test not passes validation it marked as a new failed test.

Single t.end()

t.end() must not be used more than once. This check cannot be disabled and has auto fixable rule 🐊remove-useless-t-end.

❌ Example of incorrect code

test('hello: world', (t) => {
    t.end();
    t.end();
});

✅ Example of correct code

test('hello: world', (t) => {
    t.end();
});

Check duplicates

Check for duplicates in test messages. Can be disabled with:

• passing --no-check-duplicates command line flag;
• setting SUPERTAPE_CHECK_DUPLICATES=0 env variable;

❌ Example of incorrect code

test('hello: world', (t) => {
    t.equal(1, 1);
    t.end();
});

test('hello: world', (t) => {
    t.equal(2, 1);
    t.end();
});

Check scopes

Check that test message are divided on groups by colons. Can be disabled with:

• passing --no-check-scopes command line flag;
• setting SUPERTAPE_CHECK_SCOPES=0 env variable;

❌ Example of incorrect code

test('hello', (t) => {
    t.equal(1, 1);
    t.end();
});

✅ Example of correct code

test('hello: world', (t) => {
    t.equal(1, 1);
    t.end();
});

Check assertions count

Check that test contains exactly one assertion. Can be disabled with:

• passing --no-check-assertions-count command line flag;
• setting SUPERTAPE_CHECK_ASSERTIONS_COUNT=0 env variable;

❌ Example of incorrect code

test('hello: no assertion', (t) => {
    t.end();
});

test('hello: more then one assertion', (t) => {
    t.equal(1, 1);
    t.equal(2, 2);
    t.end();
});

✅ Example of correct code

test('hello: one', (t) => {
    t.equal(1, 1);
    t.end();
});

test('hello: two', (t) => {
    t.equal(2, 2);
    t.end();
});

Operators

The assertion methods of 📼 Supertape are heavily influenced by tape. However, to keep a minimal core of assertions, there are no aliases and some superfluous operators hasn't been implemented (such as t.throws()).

The following is a list of the base methods maintained by 📼 Supertape. Others, such as assertions for stubbing, are maintained in special operators. To add custom assertion operators, see Extending.

Core Operators

t.equal(result: any, expected: any, message?: string)

Asserts that result and expected are strictly equal. If message is provided, it will be outputted as a description of the assertion.

☝️ Note: uses Object.is(result, expected)

t.notEqual(result: any, expected: any, message?: string)

Asserts that result and expected are not strictly equal. If message is provided, it will be outputted as a description of the assertion.

☝️ Note: uses !Object.is(result, expected)

t.deepEqual(result: any, expected: any, message?: string)

Asserts that result and expected are equal, with the same structure and nested values. If message is provided, it will be outputted as a description of the assertion.

☝️ Note: uses node's isDeepStrictEqual() algorithm with strict comparisons (===) on leaf nodes

t.notDeepEqual(result: any, expected: any, message?: string)

Asserts that result and expected are not equal, with different structure and/or nested values. If message is provided, it will be outputted as a description of the assertion.

☝️ Note: uses node's isDeepStrictEqual() algorithm with strict comparisons (===) on leaf nodes

t.ok(result: boolean | any, message?: string)

Asserts that result is truthy. If message is provided, it will be outputted as a description of the assertion.

t.notOk(result: boolean | any, message?: string)

Asserts that result is falsy. If message is provided, it will be outputted as a description of the assertion.

t.pass(message: string)

Generates a passing assertion with message as a description.

t.fail(message: string)

Generates a failing assertion with message as a description.

t.end()

Declares the end of a test explicitly. Must be called exactly once per test. (See: Single Call to t.end()

t.match(result: string, pattern: string | RegExp, message?: string)

Asserts that result matches the regex pattern. If pattern is not a valid regex, the assertion fails. If message is provided, it will be outputted as a description of the assertion.

t.notMatch(result: string, pattern: string | RegExp, message?: string)

Asserts that result does not match the regex pattern. If pattern is not a valid regex, the assertion always fails. If message is provided, it will be outputted as a description of the assertion.

t.comment(message: string)

Print a message without breaking the TAP output. Useful when using a tap-reporter such as tap-colorize, where the output is buffered and console.log() will print in incorrect order vis-a-vis TAP output.

Special Operators

To simplify the core of 📼 Supertape, other operators are maintained in separate packages. The following is a list of all such packages:

Here is a list of built-int operators:

Package	Version
@supertape/operator-stub

Formatters

There is a list of built-int formatters to customize output:

Package	Version
@supertape/formatter-tap
@supertape/formatter-time
@supertape/formatter-fail
@supertape/formatter-short
@supertape/formatter-progress-bar
@supertape/formatter-json-lines

API

test(message: string, fn: (t: Test) => void, options?: TestOptions)

Create a new test with message string. fn(t) fires with the new test object t once all preceding tests have finished. Tests execute serially.

Here is Possible options similar to Environment Variables but relates to one test:

• checkDuplicates;
• checkScopes;-
• checkAssertionsCount;
• timeout;

test.only(message, fn, options?)

Like test(name, cb) except if you use .only this is the only test case that will run for the entire process, all other test cases using tape will be ignored.

test.skip(message, fn, options?)

Extend base assertions with more:

const {extend} = require('supertape');
const test = extend({
    transform: (operator) => (a, b, message = 'should transform') => {
        const {is, output} = operator.equal(a + 1, b - 1);
        
        return {
            is,
            output,
        };
    },
});

test('assertion', (t) => {
    t.transform(1, 3);
    t.end();
});

Example

const test = require('supertape');

test('lib: arguments', async (t) => {
    throw Error('hello');
    // will call t.fail with an error
    // will call t.end
});

test('lib: diff', (t) => {
    t.equal({}, {hello: 'world'}, 'should equal');
    t.end();
});

// output
`
- Expected
+ Received

- Object {}
+ Object {
+   "hello": "world",
+ }
`;

License

MIT

supertape's Issues

Is this useable with TypeScript?

I'm not very familiar with testing in the JS ecosystem, and while setting up a new project I found that I could use Tape to test with:

ts-node node_modules/tape/bin/tape tests/**/*.ts

I like the features of Supertape, but I can't seem to get it to run my tests. I've tried both ts-node and esno/esmo, but I only get a "Done in 0.XXs" output without showing the testing results. Is there something I'm doing wrong, or is this use case not viable?

My test needs startup time before supertape detects tests

(Thank you very much for making supertape, it's the only testing library that matches how my brain works!)

I'm having a problem where supertape is stopping execution before my tests start running. In my test file, I want to do some slow setup (download some files to use in the tests) before the tests execute. But supertape stops it early before it can finish the setup.

From digging into the code, it seems this is because supertape expects one test() to be called in the test file straight away, and then loops every 5 ms until new test()s stop being added. Then it runs all of those tests.

So my test file basically looks like this:

const {test} = require("supertape")
const fetch = require("node-fetch")

const file = await fetch("some url").then(res => res.buffer()) // supertape stops trying to detect tests and stops the tests while this is downloading

test("process: I can process the file", () => {
  t.equal(processFile(file), "expected result")
})

I'd love it if supertape would let me do some setup before I do the tests. I can think of three ways to do this:

• (a) supertape could wait for my test file to get to at least one test() rather than ending it early
• (b) OR, supertape could have a setup(async () => { ... }) function that I can put my setup code in, and supertape will wait for setup before detecting tests
• (c) OR, supertape could have a go() function that I call once I've finished my setup

Do you have any advice on what I can do to make my use case work, or would I have to figure out a different way of doing it?

Since v10.0.0 supertape no longer prints ok for passed tests

The problem seems to only happen with --format tap or --format short.

supertape version 9.5.0:

$ supertape --no-check-assertions-count --format tap test/test.js
TAP version 13
# migrate: migration works
ok 1 it did not throw an error
# migrate: migration works the second time
ok 2 it did not throw an error
# orm: select: get works
ok 3 should equal

supertape version 10.0.0 and 10.2.0:

$ supertape --no-check-assertions-count --format tap test/test.js
# migrate: migration works
# migrate: migration works the second time
# orm: select: get works

I expected the output to have lines starting with ok.

Changes to `--require`

In #3, you mentioned that the --require flag could be used for JIT transpilation:

How does the --require flag work? Tape says that it can be used for JIT compilation - does this mean that the supertape CLI could support TypeScript through a required module?

Yes [this could work]

A quick Google search brought me to this article about using tape with TypeScript by requiring ts-node/register/transpile-only (effectively, a module that transpiles TS without type checking). However, trying

$ supertape -r ts-node/register/transpile-only test.ts

Fails with an error message about the .ts extension:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for test.ts

Looking into tape's implementation for --require, it has an import-or-require.js script:

function importOrRequire(file) {
  const ext = extnamePath(file);

  if (ext === '.mjs' || (ext === '.js' && getPackageType.sync(file) === 'module'))
    return import(pathToFileURL(file).href);

  require(file);
};

Compared to Supertape's simple-import.js, which simply uses a dynamic import:

supertape/packages/supertape/lib/simple-import.js

Line 3 in dc655d0

module.exports.simpleImport = (a) => import(a);

I've tested out a simple copy and paste implementation and can successfully run a TypeScript test:

// lib/cli.js

// in function 'cli'
for (const file of files) {
  const resolved = resolvePath(cwd, file);
  promises.push(importOrRequire(resolved));
}

// test.ts
import test from "supertape";

test("test: from ts-node", async t => {
  t.pass("using 'supertape' binary");
  t.end();
});

$ supertape -r ts-node/register/transpile-only test.ts
TAP version 13

1..1
# tests 1
# pass 1

# ✅ ok

What do you think about implementing something like this?