jest-community / create-jest-runner

A highly opinionated way for creating Jest Runners

create-jest-runner

A highly opinionated way for creating Jest Runners

Install

yarn add create-jest-runner

Usage

create-jest-runner takes care of handling the appropriate parallelization and creating a worker farm for your runner.

You simply need two files:

• Entry file: Used by Jest as an entrypoint to your runner.
• Run file: Runs once per test file, and it encapsulates the logic of your runner

1) Create your entry file

// index.js
const { createJestRunner } = require('create-jest-runner');
module.exports = createJestRunner(require.resolve('./run'));

createJestRunner(pathToRunFile, config?: { getExtraOptions })

• pathToRunFile: path to your run file. This must be an absolute path or a file:// URL.
• config: Optional argument for configuring the runner.
  • getExtraOptions: () => object used for passing extra options to the runner. It needs to be a serializable object because it will be send to a different Node process.

2) Create your run file

module.exports = options => {};

Run File API

This file should export a function that receives one parameter with the options

options: { testPath, config, globalConfig }

• testPath: Path of the file that is going to be tests
• config: Jest Project config used by this file
• globalConfig: Jest global config
• extraOptions: The return value of the { getExtraOptions } argument of createJestRunner(...) the entry file.

You can return one of the following values:

• testResult: Needs to be an object of type https://github.com/facebook/jest/blob/4d3c1a187bd429fd8611f6b0f19e4aa486fa2a85/packages/jest-test-result/src/types.ts#L103-L135
• Promise<testResult|Error>: needs to be of above type.
• Error: good for reporting system error, not failed tests.

Example of a runner

This runner "blade-runner" makes sure that these two emojis ⚔️ 🏃 are present in every file

// index.js
const { createJestRunner } = require('create-jest-runner');
module.exports = createJestRunner(require.resolve('./run'));

// run.js
const fs = require('fs');
const { pass, fail } = require('create-jest-runner');

/** @type {import('create-jest-runner').RunTest} */
const runTest = ({ testPath }) => {
  const start = Date.now();
  const contents = fs.readFileSync(testPath, 'utf8');
  const end = Date.now();

  if (contents.includes('⚔️🏃')) {
    return pass({ start, end, test: { path: testPath } });
  }
  const errorMessage = 'Company policies require ⚔️ 🏃 in every file';
  return fail({
    start,
    end,
    test: { path: testPath, errorMessage, title: 'Check for ⚔️ 🏃' },
  });
};

module.exports = runTest;

Create runner from binary

yarn create jest-runner my-runner

# Or with npm
npm init jest-runner my-runner

Note: You will have to update the package name in package.json of the generated runner.

Add your runner to Jest config

Once you have your Jest runner you can add it to your Jest config.

In your package.json

{
  "jest": {
    "runner": "/path/to/my-runner"
  }
}

Or in jest.config.js

module.exports = {
  runner: '/path/to/my-runner',
};

Run Jest

yarn jest

create-jest-runner's Issues

Writing a runner in typescript

I'm struggling to figure out how one might create a runner in typescript.

Anyone know how to do this?

Could be cool to put in the readme if we can figure out a clean way.

I see #39 but this is a little different and I didn't want to scope-creep that issue.

Determine if a test is slow

By using runtime / 1000 > config.slowTestThreshold (from here)

Changelog for 0.5.0?

I'm pretty darn fast on this i know, blame that I just ran yarn outdated. Anyway I saw you tagged and released 0.5.0 but changelog.md hasn't been updated yet.

Add binary to support `yarn create` and `npm init`

npm@6 has support for npm init (and npm create as an alias) in the same way that yarn does.

We should add a binary to this package so people can run yarn create jest-runner my-runner and have a project be scaffolded for them.

Yarn docs: https://yarnpkg.com/lang/en/docs/cli/create/
npm docs: https://docs.npmjs.com/cli/init

Identifying the running process

I'm using successfully the create-jest-runner however the processes I launch needs a unique socket port. Because I'm not able to get the first available port as I do not control the code of the process being launched, however I can pass as argument a port. All my processes thus need a different port.

To do that with create-jest-runner I would need to know which instance I have to use the right port. I was thinking of modifying create-jest-runner so to add to the extra options or add another option that would provide the ID of the runner between 1 and maxWorkers.

Is this something that you would be interested in?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): lock file maintenance

---

Warning

Renovate failed to look up the following dependencies: Failed to look up npm package *.

Files affected: package.json

---

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency @types/node to v20
• chore(deps): update dependency eslint to v9
• chore(deps): update dependency eslint-plugin-jest to v28
• chore(deps): update dependency eslint-plugin-prettier to v5
• chore(deps): update dependency prettier to v3
• chore(deps): update typescript-eslint monorepo to v7 (major) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• chore(deps): update yarn to v4
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update dependency execa to v9
• chore(deps): update dependency strip-ansi to v7
• fix(deps): update dependency chalk to v5
• fix(deps): update dependency p-limit to v6

Detected dependencies

dockerfile

Dockerfile

• node 20

github-actions

.github/workflows/nodejs.yml

• actions/checkout v4
• actions/setup-node v4
• actions/checkout v4
• actions/setup-node v4
• actions/checkout v4
• actions/setup-node v4

npm

package.json

• chalk ^4.1.0
• jest-worker ^29.5.0
• p-limit ^3.1.0
• @babel/cli ^7.0.0
• @babel/core ^7.0.0
• @babel/preset-env ^7.0.0
• @babel/preset-typescript ^7.0.0
• @jest/test-result ^29.0.0
• @tsconfig/node14 ^14.0.0
• @types/node ^14.18.23
• @typescript-eslint/eslint-plugin ^6.0.0
• @typescript-eslint/parser ^6.0.0
• babel-jest ^29.0.0
• eslint ^8.10.0
• eslint-config-airbnb-base ^15.0.0
• eslint-config-prettier ^9.0.0
• eslint-plugin-import ^2.7.0
• eslint-plugin-jest ^27.0.0
• eslint-plugin-prettier ^4.0.0
• execa ^5.0.0
• jest ^29.0.0
• jest-runner ^29.0.0
• prettier ^2.0.5
• semver ^7.3.8
• strip-ansi ^6.0.0
• typescript ^5.0.0
• @jest/test-result ^28.0.0 || ^29.0.0
• jest-runner ^28.0.0 || ^29.0.0
• node ^14.15.0 || ^16.10.0 || >=18.0.0
• * ^14.18.23
• yarn 3.8.3

---

• Check this box to trigger a request for Renovate to run again on this repository

Rewrite in typescript

☑️ I am willing to work on this issue.

This is a proposal to rewrite the whole project in typescript.
The compilation will continue to use babel but with the addition of @babel/preset-typescript to keep changes minimal.

tsc will be run separately for type-checking and emitting type declaration files without emitting js files.

Here's a checklist of tasks for tracking the progress:

• Bump babel & related dependencies to latest, add @babel/preset-typescript (WIP in #40)
• Add tsconfig.json with strict type-checking enabled (WIP in #40)
• Add typecheck script to package.json#scripts (WIP in #40)
• Migrate files in lib one-by-one to typescript & make necessary changes for type-checking to pass (WIP in #40)
• Migrate files in integrationTests one-by-one to typescript & make necessary changes for type-checking to pass
• Add typescript related eslint rules

---

Issues to be discussed / resolved:

• What do the files in the generator folder do?

• The necessary changes for typescript migrate would likely result in some breaking changes for downstream users that relied on quirks of the current version of the project. Should we integrate tests for downstream projects (e.g. jest-runner-prettier, jest-runner-eslint just to name a few) or just go for a major version bump when releasing to indicate that there might breaking changes?

• Can we prefer passing errorMessage in fail.ts using options.test.errorMessage? There are currently two methods to pass the error message using the function fail. Here's A non-exclusive list of downstream packages I checked and users seemed to prefer using options.test.errorMessage over options.errorMessage.

  1. using options.errorMessage
jest-runner-prettier
  2. using options.test.errorMessage
jest-runner-eslint
jest-runner-stylelint
jest-runner-tsd
     AFAIK jest-runner-mocha, jest-runner-electron, jest-runner-groups are not downstream consumers of this project, and these are all the jest runner projects I am aware of.
     Please let me know if this should be opened as a separate issue.

feat: Allow passing `stats` object to `pass` and `fail` functions

It would be useful to allow passing stats object to pass and fail functions.

Use case. Some runners (for instance, jest-runner-tsd) are counting passing / failing tests per file. If I get it right, at the moment create-jest-runner assumes that the whole file passed or failed. Hence, jest-runner-tsd is implementing custom pass, fail and toTestResult functions. Looks like this is unnecessary boiler plate code.

Alternatively it would be possible to expose toTestResult from create-jest-runner. Not sure about this. I think it is better to extend pass, fail allowing to pass stats (with defaults as they are set now):

interface Stats {
  failures: number;
  passes: number;
  pending: number;
  todo: number;
}

Glad to work on this. Just wanted to agree on the directions first.

Support returning array of TestResult

The change is pretty small and useful.

The use case is that I have a runner that uses cosmiconfig to get some configuration, which also can be an array, so I basically return something like

Promise.all(
  configs.map((cfg) => {
    /* runner thing */
  })
)

Currently, we need to return an object only, something like

const results = await Promise.all(
  configs.map((cfg) => {
    /* runner thing */
  })
)

return results[0];

But this way it won't show/report in the terminal all the test results.

The only change needed is this:

create-jest-runner/lib/createJestRunner.js

Lines 142 to 148 in e17075b

	const runAllTests = Promise.all(
	tests.map(test =>
	runTestInWorker(test)
	.then(testResult => onResult(test, testResult))
	.catch(error => onError(error, test)),
	),
	);

to something like

const runAllTests = Promise.all(
  tests.map(test =>
    runTestInWorker(test)
      .then(testResult => {
        if (Array.isArray(testResult)) {
          testResult.forEach(result =>
            result.errorMessage && result.stats.failures > 0
              ? onError(new Error(result.errorMessage), test)
              : onResult(test, result),
          );
          return;
        }
        onResult(test, testResult);
      })
      .catch(err => onError(err, test)),
  ),
);

Tested. It is working and shows all the tests.

---

$ jest -c jest.build.config.js
 PASS   build  @tunnckocore/utils/dist/main/index.js
 PASS   build  @tunnckocore/utils/dist/module/index.js
 PASS   build  @tunnckocore/execa/dist/main/index.js
 PASS   build  @tunnckocore/execa/dist/module/index.js

previously was showing only

 PASS   build  @tunnckocore/utils/dist/main/index.js
 PASS   build  @tunnckocore/execa/dist/main/index.js

Remove mocha

Noticed mocha being added to my lock file, and this repo is the culprit. Not sure why this requires mocha when I see no usages? Nor why it's a production dep.

Passing data from CJR to the workers

This is a feature suggestions. I would love to have an agreement about the design, than I will implement it.

The problem - I have a use case that I need to do some expensive operation in the runner. I can do it once, but I don't have a way to pass the data to the workers. Solving it by repeating the operation inside each worker will reduce the performance that the jest runner approach will be useless.

Suggested solution - Naive approach. Let CJR (create jest runner) pass an optional data object. Each worker can pass the data object to the worker as an argument inside the OnStart function.

types folder not on npm

The types folder is not distributed on NPM.

It looks to me that if submitting another publish to npm should fix this.

Using custom testEnvironment

Can I use custom testEnvironment with created runner?
Tried this:
jest-config.js:

module.exports = {
    runner: './index.js',
    testEnvironment: './custom-env.js',
};

custom-env:

const NodeEnvironment = require('jest-environment-node');

class CustomEnvironment extends NodeEnvironment {
    constructor(config) {
        super(config)
    }

    async setup() {
        console.log('Setup Test Environment.');
        await super.setup();
    }

    async teardown() {
        console.log('Teardown Test Environment.');
        await super.teardown()
    }

    runScript(script) {
        return super.runScript(script)
    }
}

module.exports = CustomEnvironment;

I got no output from my CustomEnvironment, but it's ok when I comment out runner option

Feature request: provide type definition

Hi,

I am writing runner with TypeScript but I notice there is no type definition in create-jest-runner. I would be nice if create-jest-runner is shipped with type definition.

Runners built with `create-jest-runner` do not work with next.js (or presumably any other projects that have configurations that cannot be cloned)

So, this is a strange issue, and I'm not entirely sure what's going on yet, but I do have a minimal repro case that illustrates the issue.

Basically, given the following jest config:

const nextJest = require("next/jest");

const createJestConfig = nextJest({ dir: "./" });

module.exports = async () => ({
    projects: [
        await createJestConfig({
            displayName: "unit-test"
        })(),
        {
            displayName: "lint",
            runner: "jest-runner-eslint",
            testMatch: ["<rootDir>/pages/**/*.tsx"]
        }
    ]
});

Then running jest results in:

❯ npm run test

> [email protected] test
> jest

 FAIL   lint  pages/index.tsx
  ● Test suite failed to run

    ()=>null could not be cloned.

      at ChildProcessWorker.send (node_modules/jest-runner-eslint/node_modules/jest-worker/build/workers/ChildProcessWorker.js:279:17)

 FAIL   lint  pages/page2.tsx
  ● Test suite failed to run

    ()=>null could not be cloned.

      at ChildProcessWorker.send (node_modules/jest-runner-eslint/node_modules/jest-worker/build/workers/ChildProcessWorker.js:279:17)

Test Suites: 2 failed, 2 total
Tests:       0 total
Snapshots:   0 total
Time:        0.122 s
Ran all test suites.

I think that what's going on here is that create-jest-runner is trying to send the entire jest config (rather than just the project's config), to the jest-worker threads, which involves deep cloning the config object. However, it seems that next/jest includes closures that cannot be cloned in it's project config.

Presumably, I'm guessing this is also going to be broken if there's any other project in the jest config that includes any fields that cannot be cloned.

I'm not really sure as I don't know anything at all about how create-jest-runner or jest-worker work so I could be way off here. Would appreciate any tips / thoughts to help me investigate and potentially fix this one.

Tests are failing locally, because the colors

One way to fix it is to strip colors, another is to just prepend the test script with CI=true.

Passing --no-colors to the runJest seems to not work.

On a side note, linting is also failing, because the __fixtures__ and fixtures.

Can it be that wrong `.d.ts` files got published to `npm` with the latest version?

The .d.ts files look strange in latest version (0.11.0). Can it be something wrong got published to npm?

For instance, types.ts in the repo does not import from '@jest/types' anymore and it does not have types Milliseconds, or Path. But here is what I see installing the package and looking at types.d.ts (this is the whole file):

I was trying to use the new RunTest type, but it is not there.

The installed version is 0.11.0. I checked three times, but who knows (;