direct-adv-interfaces / mocha-headless-chrome

Run client-side mocha tests in the command line through headless Chrome.

mocha-headless-chrome

This is the tool which runs client-side mocha tests in the command line through headless Chrome (puppeteer is used).

Node 14.0.0+ and Mocha 2.3.0+ are supported.

Getting Started

First you need to install mocha-headless-chrome:

npm i mocha-headless-chrome

Then prepare the test page (see the example).

Note. It is necessary to add the <meta charset="utf-8"> tag. Otherwise browser may use another encoding and test results will be shown incorrectly.

Then run the CLI and specify your test page path using -f parameter.

mocha-headless-chrome -f test-page.html

Options

• -f, --file - Path or URL of the page which contains tests (required)
• -r, --reporter - Mocha reporter name (defaults to "spec")
• -o, --out - Path to the file where test result will be saved
• -c, --coverage - Path to the file where coverage info will be saved
• -e, --executablePath - Chrome executable path
• -v, --visible - Show Chrome window
• -a, --args - Chrome arguments ('--' prefix will be added)
• -w, --width - Viewport width
• -H, --height - Viewport height
• -t, --timeout - Timeout in ms (defaults to 60000)
• -p, --polling - Puppeteer polling mechanism
• -h, --help - Output usage information

Examples

Run test on the "test.html" page:

mocha-headless-chrome -f test.html

Run tests on the remote page:

mocha-headless-chrome -f http://localhost:8080

Output test results using "nyan" reporter:

mocha-headless-chrome -f test.html -r nyan

Pass the Chrome --no-sandbox and --disable-setuid-sandbox arguments:

mocha-headless-chrome -f test.html -a no-sandbox -a disable-setuid-sandbox

Mocha reporters

All mocha reporters are supported. Specify the reporter name through -r parameter. All reporter output (include cursor manipulations) will be redirected to stdout as like it works in console.

For usage of third-party reporter just include it's code to the page by <script> tag and specify it's name in the -r parameter.

Also special reporter named "none" is available which does not output anything. This reporter will be useful when you want to process test result without output to console (for example, when saving data to a file).

Programmatically usage

You can use mocha-headless-chrome programmatically. Just require the mocha-headless-chrome node module and pass proper parameters into it. Function result is a Promise.

const {runner} = require('mocha-headless-chrome');

const options = {
    file: 'test.html',                           // test page path
    reporter: 'dot',                             // mocha reporter name
    width: 800,                                  // viewport width
    height: 600,                                 // viewport height
    timeout: 120000,                             // timeout in ms
    polling: 'raf',                              // polling mechanism
    executablePath: '/usr/bin/chrome-unstable',  // chrome executable path
    visible: true,                               // show chrome window
    args: ['no-sandbox']                         // chrome arguments
};

runner(options)
    .then(result => {
        let json = JSON.stringify(result);
        console.log(json);
    });

See also the TypeScript example.

mocha-headless-chrome's Issues

Strange characters before test description in output

On MacOS X and Windows I see a strange set of characters before the description of each test:

  Color
    ✓ should parse strings

Should this be an emoji?

Coverage support

Hello. Does m-h-c requires that code for testing also is to be instrumented before using the -c / --coverage option. I was unable to produce coverage information (just empty files) with the -c option. Tnx.

Edit: I've found your explanation for this in closed issues section. This can be deleted.

Error: m.setup is not a function at shimMochaInstance

This library looks like just what I need! When I tried running it, I'm getting an error. Can you please point me in the right direction to sort it out?

$ node_modules/mocha-headless-chrome/bin/start -f http://localhost:8080
Error: TypeError: m.setup is not a function
    at shimMochaInstance (<anonymous>:10:11) 
    ...

Here's my test page, FWIW. It works fine in the browser.

<html>
<head>
  <meta charset="utf-8">
  <link href="https://cdn.rawgit.com/mochajs/mocha/2.2.5/mocha.css" rel="stylesheet" />
</head>
<body>
  <div id="mocha"></div>
  <script src="/static/build/lib.js"></script>
  <script src="/static/build/app.min.js"></script>
  <script src="http://chaijs.com/chai.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/sinon@3/pkg/sinon.js"></script>
  <script src="https://cdn.rawgit.com/mochajs/mocha/2.2.5/mocha.js"></script>
  <script>mocha.setup('bdd')</script>
  <script src="/static/build/unit-tests.js"></script>
  <script>
    mocha.run();
  </script>
</body>
</html>

Thank you for any help you can provide!

coverage doesn't write if directory to file doesn't already exist

It'd be nice if there were mkdirp-style logic if the --coverage directory path doesn't yet exist.

I'd be up for sending a PR to use mkdirp.sync, if you're up for it.

{ Error: ENOENT: no such file or directory, open 'C:\Code\Shenanigans\audioplayr\coverage\coverage.json'
    at Object.fs.openSync (fs.js:667:18)
    at Object.fs.writeFileSync (fs.js:1326:33)
    at runner.then.obj (C:\Code\Shenanigans\audioplayr\node_modules\mocha-headless-chrome\cli.js:38:28)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:118:7)
  errno: -4058,
  code: 'ENOENT',
  syscall: 'open',
  path: 'C:\\Code\\Shenanigans\\audioplayr\\coverage\\coverage.json' }

Support grep and invert options

At work, we use these options to run different parts of the test suite in parallel.

It would be nice if the runner supports this, or exposes some mechanism to configure general Mocha options.

Using already installed chrome

Hi, there is any way to pass executablePath option to puppeteer through CLI?

Hangs when grep results in no tests

If the grep query argument causes no tests to be run, the test runner’s end event is fired synchronously and can’t be caught as line 69 of runner.js. The tests output “0 passing”, but then they just hang instead of closing.

For example, if I have the following in browsertest.html:

<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
        <title>In-Browser Mocha Tests</title>
        <link rel="stylesheet" href="../node_modules/mocha/mocha.css">
    </head>
    <body>
        <div id="mocha"></div>
        <script src="../node_modules/mocha/mocha.js"></script>
        <script>
            mocha.setup({ui: 'bdd'});

            describe('Test suite', function () {
                it('A single passing test', function() {
                    // this passes
                });
            });

            mocha.run();
        </script>
    </body>
</html>

Running this hangs:

mocha-headless-chrome --file test-plugins/browsertest.html?grep=unmatchable

If you pass a callback directly to mocha.run(), that seems to reliably get called in this scenario.

Is height -h or -H?

It's the only arg in the readme that's uppercase. Is that intentional?

Enable Travis CI for test script

It would be good to have CI for PRs and commits.

Code coverage?

Could you provide a sample of how this be used with, say, istanbul/nyc please?

Add a changelog

Hi,

What is the changlog for v2.0.0.

• What did changed ?
• Why does this require a major version bump ?
• Is there any breaking changes ?

It would be useful to add a CHANGELOG.md file, and to add the same description to each GitHub release. See https://keepachangelog.com if you need any help.

Exception in runners.js

Mocha Version 8.1.0

Macha Headless Chrome Version 3.1.0

Something must have changed in Mocha with version 8.1.0. With the update to Mocha 8.1.0 (I checked: that is the culprit) Mocha Headless Chrome crashes on me in line 72 when it tries to access Mocha.reporters. "Mocha" is undefined at that point.

What do width and height default to?

It's not clear from the README.md or source code?

Why would I want this?

Got mocha and headless chrome to work just as such, so I'm wondering... what exactly is the use case for your repo?

The initial feeling I got from the README is that this would be needed to make mocha and puppeteer work. It's not. Please educate. Thanks.

Running in Travis

I've been using mocha-headless-chrome in travis for continuous integration for a while.

However, this week, the mocha-headless-chrome test runner has started failing in Travis.

At first, I thought it was an issue with puppeteer sandboxing so I tried starting mocha-headless-chrome as follows:

mocha-headless-chrome -a no-sandbox,disable-setuid-sandbox -f http://127.0.0.1:${port}/demo/mocha.html

That didn't work.

Has anyone else seen this issue?

Am I setting the puppeteer arguments correctly?

I'm getting errors like the following repeated:

ERROR: { Error: watch /home/travis/build/fs-webdev/idx-new-admin-app/node_modules/shebang-command/license ENOSPC
    at _errnoException (util.js:1022:11)
    at FSWatcher.start (fs.js:1374:19)
    at Object.fs.watch (fs.js:1400:11)
    at createFsWatchInstance (/home/travis/build/fs-webdev/idx-new-admin-app/node_modules/chokidar/lib/nodefs-handler.js:37:15)
    at setFsWatchListener (/home/travis/build/fs-webdev/idx-new-admin-app/node_modules/chokidar/lib/nodefs-handler.js:80:15)
    at FSWatcher.NodeFsHandler._watchWithNodeFs (/home/travis/build/fs-webdev/idx-new-admin-app/node_modules/chokidar/lib/nodefs-handler.js:228:14)
    at FSWatcher.NodeFsHandler._handleFile (/home/travis/build/fs-webdev/idx-new-admin-app/node_modules/chokidar/lib/nodefs-handler.js:255:21)
    at FSWatcher.<anonymous> (/home/travis/build/fs-webdev/idx-new-admin-app/node_modules/chokidar/lib/nodefs-handler.js:473:21)
    at FSReqWrap.oncomplete (fs.js:153:5)
  code: 'ENOSPC',
  errno: 'ENOSPC',
  syscall: 'watch /home/travis/build/fs-webdev/idx-new-admin-app/node_modules/shebang-command/license',
  filename: '/home/travis/build/fs-webdev/idx-new-admin-app/node_modules/shebang-command/license' }

Multiple input files support?

Hi, using the mocha-headless-chrome programmatically I am wondering how to pass multiple files at once for running with the same options? Is that something which is already possible? Or something for future versions? Or something you dont want to support at all?

Error: waiting for function failed: timeout 60000ms exceeded

I try to run this and always get this error:

$ mocha-headless-chrome -f MyTest.html
Error: waiting for function failed: timeout 60000ms exceeded
    at Timeout.WaitTask._timeoutTimer.setTimeout (C:\Users\tdo\AppData\Roaming\npm\node_modules\mocha-headless-chrome\node_modules\puppeteer\node6\lib\FrameManager.js:1413:60)
    at ontimeout (timers.js:380:14)
    at tryOnTimeout (timers.js:244:5)
    at Timer.listOnTimeout (timers.js:214:5)

If I run with the browser visible (-v), I can see that the tests run and pass.

Test result is not saved in the format of the reporter.

I was expecting to see the reporter output when using --out, but that's not the case. I use xunit reporter, and I see the xml in the terminal, but the file which save the results is a json object:

console:

<testsuite name="Mocha Tests" tests="1" failures="0" errors="0" skipped="0" timestamp="Wed, 13 Jun 2018 08:37:18 GMT" time="0.017">
<testcase classname="Button" name="must render a button with given content as label" time="0.014"/>
</testsuite>

saved file:

{"stats":{"tests":1,"passes":1,"pending":0,"failures":0,"start":"2018-06-13T08:37:18.093Z","end":"2018-06-13T08:37:18.110Z","duration":17},"tests":[{"title":"must render a button with given content as label","fullTitle":"Button must render a button with given content as label","duration":14,"err":{}}],"pending":[],"failures":[],"passes":[{"title":"must render a button with given content as label","fullTitle":"Button must render a button with given content as label","duration":14,"err":{}}]}

Tests don't run/fail with mocha-headless-chrome while they succeed in browser

We have started a lot of unit tests for ES6 modules. Our tests run fine in the browser but fail when using mocha-headless-chrome.

In some cases, mocha-headless-chrome attempts to run the tests but exists reporting 0 passing tests. Other times, it runs but all the tests fail because the web components apparently never get created.

Here's a repo that shows the first case. Any help would be greatly appreciatd.

Add option to run test suite in incognito mode

#28

In incognito mode, Chrome doesn't write any user data to disk. This can lead to significant speedups with test suites that use features like IndexedDB.

Note that when incognito is enabled, this will open two windows (one incognito and one non-incognito). I couldn't figured out how to suppress this behavior. It should be harmless though.

Can support TypeScript like mocha `-r` param?

Mocha

$ mocha -r ts-node/register -r ignore-styles \"test/**/*.spec.*\""

mocha-headless-chrome (expected)

$ mocha-headless-chrome -r ts-node/register -r ignore-styles -f test.html

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Mocha Tests</title>
        <meta charset="utf-8">
        <link rel="stylesheet" href="../node_modules/mocha/mocha.css">
        <script src="../node_modules/mocha/mocha.js"></script>
        <script src="../node_modules/chai/chai.js"></script>
    </head>
    <body>
        <div id="mocha"></div>
        <script>mocha.setup('bdd');</script>
        <!-- add typescript directly here -->
        <script src="example.ts"></script>
        <script>mocha.run();</script>
    </body>
</html>

Console output handler adds a lot of blank lines

Hi there. In our test setup we periodically have a lot of newlines in the output. This seems to be caused by the stdout checking behavior in this library, where if the log message doesn't come from mocha, a newline is inserted. (This toggles between funny and annoying.)

We looked at removing all the console statements from our project and this didn't fix the issue, so our guess is that this is in one of the libraries that we're using.

Looking more into the details it looks like the console event from puppeteer is coming with no arguments, which causes the "not standard out" logic to be triggered, which adds a newline. The empty list is blank so this ends up with a blank newline.

No cookes

My tests making same origin requests for some data. In headless mode server does not received any cookies. In real browser everything works.

What could be wrong with my setup?

Provide TypeScript typings?

https://github.com/DefinitelyTyped/DefinitelyTyped/
DefinitelyTyped/DefinitelyTyped#21760
DefinitelyTyped/DefinitelyTyped#21762

It's useful for TypeScript typings to be published as a part of JavaScript packages. That way they don't get out of sync with the packages themselves.

Failing to load tests doesn't cause test suite to fail

If Chrome has an issue loading a test file, it will raise some error and will attempt to run the rest of the suite.
mocha-headless-chrome will catch the error here, but doesn't do anything with it. This leads to the odd situation of Chrome running only a subset of the tests and mocha-headless-chrome reporting success if all tests in that subset pass.

The impact of this is that inattentive developers will get fooled, and CI won't detect the issue.

Reporter option is ignored

Steps to reproduce

Create a project with the following files:

package.json

{
  "name": "mhc-bug",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "mocha-headless-chrome -f tests.html -r teamcity"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chai": "^4.1.2",
    "mocha-headless-chrome": "^2.0.0",
    "mocha-teamcity-reporter": "^2.4.0"
  }
}

tests.html

<!DOCTYPE html>
<html>
    <head>
        <title>Mocha Tests</title>
        <meta charset="utf-8">
        <link rel="stylesheet" href="node_modules/mocha/mocha.css">
    </head>
    <body>
        <div id="mocha"></div>
        <script src="node_modules/mocha/mocha.js"></script>
        <script src="node_modules/chai/chai.js"></script>
        <script src="node_modules/mocha-teamcity-reporter/lib/teamcityBrowser.js"></script>
        <script>
            mocha.setup('bdd');
            // mocha.reporter('teamcity');
        </script>
        <script src="tests.js"></script>
        <script>mocha.run();</script>
    </body>
</html>

tests.js

'use strict';

const assert = this.chai ? this.chai.assert : require('assert');

describe('hello', function () {
    it('world', function () {
        console.log('!!!');
    });
});

Then run npm test.

Observed behavior

Mocha uses the default spec reporter, even though -r teamcity was specified on the command line.

Uncommenting the // mocha.reporter('teamcity'); line yields the expected behavior, so this is not a problem with the reporter itself.

Expected behavior

Expect test output in TeamCity format, as the -r teamcity option was passed to mocha-headless-chrome.

Suggested fix

I believe this issue is caused by this code in mocha-headless-chrome/lib/runner.js:

m.setup({ reporter: Mocha.reporters[reporter] || Mocha.reporters.spec });

This code executes as soon as the global mocha object is defined. It initializes the Mocha framework with the specified reporter.

However, at this point, any custom reporter wouldn't be defined yet. Hence the object lookup fails and it falls back to Mocha.reporters.spec.

I think this code should instead be:

const setup = m.setup.bind(m);

m.setup = (options) => {
    if (typeof options === 'string') {
        options = { ui: options };
    } else if (options === null || options === undefined) {
        options = {};
    }

    return setup(Object.assign({}, options, { reporter }));
};

This delays initialization until mocha.setup() is called from user code. By then, any custom reporters should be ready to use.

No output

Hi there!

I have a problem: when I run mocha-headless-chrome, I have no output:

$ node_modules/.bin/mocha-headless-chrome -f ./test.inbrowser/build/index.html -r tap
%cDownload the React DevTools for a better development experience: https://fb.me/react-devtools
You might need to use a local HTTP server (instead of file://): https://fb.me/react-devtools-faq font-weight:bold
$ 

If some tests are going to fail, I have just +1 line tests failing

$ node_modules/.bin/mocha-headless-chrome -f ./test.inbrowser/build/index.html -r tap
%cDownload the React DevTools for a better development experience: https://fb.me/react-devtools
You might need to use a local HTTP server (instead of file://): https://fb.me/react-devtools-faq font-weight:bold
Tests fails
$ 

I tried to change -r option to spec, dot – none of them works. How can I fix that?

npm list | grep mocha-headless-chrome
├─┬ [email protected]

--watch

Thanks for the package. It works great.

Are there any plans to add a watcher for development mode?

Question about coverage options

Hi,

This is a great project, using it to test custom elements as an alternative to WCT and it works pretty nicely.

I'd like to get the coverage option -c working, but I get an empty file.
Do I need to setup istandbul separately?
Any guidelines would be appreciated.

Thanks

Add coverage option for programmatic use

We ran into an issue when needing to add in coverage as an option while using it programmatically via node api. This is an option if you use it via the cli but not the node api.

Proposal Solution

const {runner} = require('mocha-headless-chrome');

const options = {
    file: 'test.html',
    timeout: 120000,
    coverage: 'build/coverage.json'
};

runner(options)
    .then(result => {
        let json = JSON.stringify(result);
        console.log(json);
    });

Don't close browser window

Is it possible to have a flag that doesn't close the browser window? My use case is, if some tests fails, I can instantly see what's wrong in the browser.

support mocha --reporter-options settings

some reporters (like xunit) have reporter-specific options to control its output. there doesn't seem to be any way to set this using mocha-headless-chrome though. one can workaround it by manually calling mocha.reporter(...), but that sort of defeats the point of having the --reporter pass-thru setting.

Cannot exec `npm install`

I could not execute npm install on my local environment and also on GitHub Actions.

https://github.com/mishina2228/mocha-headless-chrome/runs/6862661629?check_suite_focus=true

npm ERR! code ETIMEDOUT
npm ERR! syscall connect
npm ERR! errno ETIMEDOUT
npm ERR! network request to https://repo.br.hmheng.io/artifactory/api/npm/npm-remote/yocto-queue/-/yocto-queue-0.1.0.tgz failed, reason: connect ETIMEDOUT 10.35.19.146:443
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network 
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2022-06-13T13_52_15_594Z-debug-0.log

mocha-headless-chrome/package-lock.json

Lines 1738 to 1746 in 6c95d37

	"node_modules/yocto-queue": {
	"version": "0.1.0",
	"resolved": "https://repo.br.hmheng.io/artifactory/api/npm/npm-remote/yocto-queue/-/yocto-queue-0.1.0.tgz",
	"integrity": "sha1-ApTrPe4FAo0x7hpfosVWpqrxChs=",
	"dev": true,
	"license": "MIT",
	"engines": {
	"node": ">=10"
	},

It seems that repo.br.hmheng.io is not available.

Can we recreate package-lock.json?

Failed tests do not throw errors

What

When one of the tests fails, its report is visible in the console, however, it does not result into error.

Why

This causes failed tests to have no effect on the build pipeline (CI), making build successful even on tests failure.

Example

See one of my builds with failed test.

Expected behavior

Failed test should throw in the console to ensure proper CI.