sabberworm / karma-iframes

Lets you run each test in a separate context, loaded as an iframe.

What is karma-iframes?

It’s a Karma plugin that lets you run each test in a separate context, loaded as an iframe.

Essentially, it will let you designate a set of files as running each in a separate context. A test designated this way will run in a new iframe, isolated from every other test file designated thusly.

This means you can pollute the global namespace all you want in one test without affecting the other (see karma-runner/karma#412).

Why is this useful?

Preprocessors that package a test file plus all its dependencies into a single file (think TypeScript in its --outFile --module mode or karma-webpack) can sometimes end up packaging the same dependency multiple times. This plugin won’t fix that but it will mitigate its effects. For example, if you have a file that requires 'jquery' and then exports $ to window.$ and you include this file from multiple tests, only the one instance of jQuery that is loaded last will actually be global but it may or may not have all required jQuery plugins registered to it, depending on which entry file they were included in.

Also, some older frameworks (Ext, Prototype) might require certain globals to exist in specific places.

Are there any drawbacks?

Sure, as always:

• When focusing on a test (e.g. using Jasmine’s fit, or QUnit’s only), you’ll only be focusing on this test within the suite/file it belongs, not the whole test set. This might not be what you want.
• Creating a new context incurs some costs, both in the karma server as well as in the client code. You should be able to mitigate this by setting runInParent to true, to nest the iframes only 1 level deep instead of 2.
• The plugin messes with some karma internals and might not be compatible with all configurations/plugins.
• For it to work, all files you want separated have to not depend on each other. You can only include each file either in all iframes or only in one. Slicing arbitrarily is not supported.
• Code coverage support is not under unit test.

How does it work?

The plugin will prevent the test files from being directly included in the test runner context. Instead it will only add one adapter that loops through all the designated files, and create an iframe for each.

These iframes each load an HTML file similar to the normal karma context: it includes the normal test adapter and all ambient files but of the designated files it will only contain a single one. It will also include an additional file which I lovingly call the “reverse context”. This will provide the __karma__ global that is necessary for the test adapter to notify the test results.

This reverse context will talk to the iframe adapter on the parent context using the postMessage API.

Configuration

To use the plugin, install it first:

npm install --save-dev karma-iframes

Then, add it to the plugins section of your karma.conf.js:

	[…]
	plugins: [
		[…],
		'karma-iframes'
	],
	[…]

To use it, add it as a framework:

	[…]
	frameworks: [
		[…],
		'iframes'
	],
	[…]

Most likely you’ll want to list 'iframes' as the last item in frameworks. Any file included by a later framework will be included into the test runner context but not the iframe context. This might be needed in some cases, e.g. for frameworks that extend the way the karma client talks to the server, but it’s not the common case.

Lastly, mark the files you want separated with the iframes preprocessor:

	[…]
	preprocessors: {
		[…],
		'test/modules/**/*.js': ['iframes']
	},
	[…]

of course, you can combine this with other preprocessors:

	[…]
	preprocessors: {
		[…],
		'test/modules/**/*.js': ['webpack', 'iframes'],
		'test/modules/**/*.ts': ['typescript', 'iframes']
	},
	[…]

In the list of preprocessors for a pattern, I can’t think of a single reason why you’d not want the 'iframes' preprocessor to be last (unless some other plugin comes along, designed specifically to extend karma-iframes).

Acknowlegdements

• Thanks to @thealien for code coverage support.

karma-iframes's Issues

A巨下午好

こんばんは

'jasmine is not defined' after adding karma-iframes

After I added karma-iframes as documented I get the error: Uncaught ReferenceError: jasmine is not defined

Updating Karma to v5+ : PromiseContainer is not a constructor

Hello,

We are using karma-iframes for a while, with Karma v4.4.1

When trying to upgrade to Karma v5+ (tried several v5+ with no more success)
karma-server refuse to start with this error :

31 08 2020 11:30:29.879:ERROR [karma-server]: Server start failed on port 9877: TypeError: common.PromiseContainer is not a constructor npm ERR! code ELIFECYCLE npm ERR! errno 130

This doesn't occur if I remove "iframes" from our list of frameworks.

Is there a needed configuration change for karma-iframes when upgrading karma from v4 to v5 ?

Thanks for the help.

Edit:
here is our dev dependencies for karma plugins :

    "karma": "5.1.1",
    "karma-chrome-launcher": "^2.2.0",
    "karma-coverage-istanbul-reporter": "^2.0.4",
    "karma-iframes": "^1.2.2",
    "karma-junit-reporter": "^1.2.0",
    "karma-mocha": "^2.0.1",
    "karma-sinon-chai": "^2.0.2",
    "karma-sourcemap-loader": "^0.3.8",
    "karma-spec-reporter": "0.0.32",
    "karma-webpack": "^4.0.2",

relevant configuration :

  config.set({
    frameworks: ["mocha", "sinon-chai", "iframes"],
    reporters: ["spec", "coverage-istanbul", "junit"],
    files: [
      'test/**/*.spec.js'
    ],
    preprocessors: {
      'test/**/*.spec.js': ["webpack", "sourcemap", "iframes"]
    },

reverse-context.js added twice due to failing filter guard on Windows

Issue:
On Windows only, the reverse-context script is added twice, this causes the context to not be setup properly.

Cause:
In rewrite-middleware.js

.filter(file => file.originalPath !== REVERSE_CONTEXT);

Does not evaluate to through for reverse-context filepath on windows as require-resolve does not normalize path separators.

REVERSE_CONTEXT

Resolution:
In frameworks.js when require resolving REVERSE_CONTEXT

let REVERSE_CONTEXT = require.resolve('../static/reverse-context.js').split(path.sep).join('/');

And change rewrite-middleware.js Line 74

return (file.originalPath === REVERSE_CONTEXT) || (file.originalPath === REVERSE_CONTEXT.split(path.sep).join('/')));

to

  return file.originalPath === REVERSE_CONTEXT;

JS errors sent from the iframe to the parent context aren't serialised correctly

The callback of window.onerror is as follows:

window.onerror = function(message, source, lineno, colno, error) { ... }

The last parameter 'error' is an Error instance. When this error is caught and sent to the context (via the reverse context), postMessage is used but Error objects can't be serialised in postMessage and so the call fails and the error never makes it to the parent context. The error thrown in this case looks like:
Uncaught DOMException: Failed to execute 'postMessage' on 'Window': Error could not be cloned.

This seems to specifically be for errors in the JS document itself (syntax errors etc) rather than asserts in the tests themselves.

The offending code is in reverse-context.js, lines 30-36:

DIRECT_METHODS = ['error', 'log', 'complete', 'result'];
DIRECT_METHODS.forEach(method => {
	karma[method] = function() {
		callParentKarmaMethod(method, Array.prototype.slice.call(arguments));
	}
	karma[method].displayName = method+' (proxied)';
});

This could be corrected by adding a transform to the arguments array that converts Error objects into strings, e.g.

DIRECT_METHODS = ['error', 'log', 'complete', 'result'];
DIRECT_METHODS.forEach(method => {
	karma[method] = function() {
		var args = Array.prototype.slice.call(arguments);
		for (var i = 0, l = args.length; i < l; ++i) {
			if (args[i] instanceof Error) {
				args[i] = args[i].stack || args[i].toString();
			}
		}
		callParentKarmaMethod(method, args);
	}
	karma[method].displayName = method+' (proxied)';
});

This does mean however that karma never sees an Error object on the other end, only a string, but it gets sent right to karma.error so this might be ok.