istanbuljs-archived-repos / istanbul-lib-instrument Goto Github PK

When code gets instrumented entries in fnMap and branchMap don't have the line property as specified in https://github.com/gotwarlost/istanbul/blob/master/coverage.json.md .

I found that this breaks lcov reports generated by Intern.

insertFunctionCounter in visitor.js passes name(path.node.id ? path.node.id.name : path.node.name) as the name to this.cov.newFunction. However, when name is null, the result fnMap contains 'anonymous_' in their names, how can i get the correct name without 'anonymous_'?

name = name || '(anonymous_' + f + ')'; ---newFunction in source-coverage.js

/* istanbul ignore next */
export function doesntWork() {
  if (true) {
    console.log('covered');
  } else {
    console.log('not covered');
  }
}

/* istanbul ignore next */
function works() {
  if (true) {
    console.log('covered');
  } else {
    console.log('not covered');
  }
}
export { works };

After transforming it with Babel:

// ...

/* istanbul ignore next */
export function doesntWork() {
  ++cov_vic9ro8ua.f[0];
  ++cov_vic9ro8ua.s[0];

  if (true) {
    ++cov_vic9ro8ua.b[0][0];
    ++cov_vic9ro8ua.s[1];

    console.log('covered');
  } else {
    ++cov_vic9ro8ua.b[0][1];
    ++cov_vic9ro8ua.s[2];

    console.log('not covered');
  }
}

/* istanbul ignore next */
function works() {
  if (true) {
    console.log('covered');
  } else {
    console.log('not covered');
  }
}
export { works };

The ignore annotation is not respected.

It looks possible that the codeVisitor list needs to be updated.

instead of monkeying around with index locations and attaching comments in the instrumentation code.

The release of v1.3.1 caused a regression mentioned here: 590abf6#commitcomment-20304998

PhantomJS 2.1.1 (Linux 0.0.0) ERROR
SyntaxError: Cannot declare a parameter named 'area' in strict mode

Wanted to create an issue to track with a PR instead of just a commit comment.

Someone send a PR to node-tap that defends against tests that might mutate Object.prototype, essentially replacing some for (var i in obj) loops with Object.keys(obj).forEach(..), or adding a if(obj.hasOwnProperty(i)) guard.

Here's the test:

// test/has-own-property.js
Object.prototype.foo = 'bar'
require('../').pass('modifying Object.prototype did not break things')

Running with coverage does this:

$ nyc node test/has-own-property.js
Transformation error for /Users/isaacs/dev/js/tap/lib/root.js ; return original code
Cannot assign to read only property 'optional' of bar
Transformation error for /Users/isaacs/dev/js/tap/lib/test.js ; return original code
Cannot assign to read only property 'optional' of bar
Transformation error for /Users/isaacs/dev/js/tap/lib/stack.js ; return original code
Cannot assign to read only property 'optional' of bar
Transformation error for /Users/isaacs/dev/js/tap/lib/assert.js ; return original code
Cannot assign to read only property 'optional' of bar
Transformation error for /Users/isaacs/dev/js/tap/lib/synonyms.js ; return original code
Cannot assign to read only property 'optional' of bar
Transformation error for /Users/isaacs/dev/js/tap/lib/mocha.js ; return original code
Cannot assign to read only property 'optional' of bar
TAP version 13
ok 1 - modifying Object.prototype did not break things
1..1
# time=23.336ms
----------|----------|----------|----------|----------|----------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------|----------|----------|----------|----------|----------------|
All files |  Unknown |  Unknown |  Unknown |  Unknown |                |
----------|----------|----------|----------|----------|----------------|

I was able to make a simpler example but it has a different error:

// t.js
Object.prototype.foo = 'bar'
require('./x.js')

// x.js
var x = { baz: 1 }

$ nyc node t.js
failed to instrument /Users/isaacs/dev/js/tap/x.js with error: You passed `traverse()` a visitor object with the property "Program" that has the invalid property "foo"
----------|----------|----------|----------|----------|----------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------|----------|----------|----------|----------|----------------|
All files |      100 |      100 |      100 |      100 |                |
 t.js     |      100 |      100 |      100 |      100 |                |
----------|----------|----------|----------|----------|----------------|

The new logic added in #33 introduces issues with function names. Take a case like this:

let someFunction = function() {
  someFunction = function(){};
};

becomes

let someFunction = (coverageStuff, function someFunction() {
  someFunction = function(){};
});

which will fail because someFunction now references the internal function name binding, rather than the outer one.

This is causing Babel itself to fail to bootstrap because we use this plugin in our tests, and has also been reported by a third-party in babel/babel#5048.

Everyone's favorite test-coverage tool is now an organization \o/ would love help figuring out a logo.

consider using https://github.com/lerna/lerna

except maybe not for "nyc"?

Hey! I spent some time now with latest Alpha release, and I'm impressed! Great work!
However. I found a few strange things...

In the project I tried this I'm using TypeScript. First I figured out is that Ecmascript - ES6 - isn't working with this Istanbul release?

I know import & export statements isn't working in the node.js environment, and a possible workaround is to use sourcemap. But still "normal" ES6 syntax that Node v.6.x are accepting was throwing at me.
How to solve this? Sourcemap?

The most crazy thing I discovered was the UMD format.

With "normal" TS transpile without modules give me 100% correct coverage.
If I transpile down to ES5 with UMD as module, the instrumenter goes crazy!
The coverage result ending up to be 46% even if all lines are covered!!

The reported uncovered lines is also wrong if you do this. The .js file is only on 1 line, but get reported back that both line 6 and 7 are not covered!!

Functions are reported to be 2, but there are no functions.

So you better understand...

tsc --outDir ./build/es5 -t es5 -m umd

Transpile down from TS to ES5, and run coverage on the transpiled files.

If I do it like this

tsc --outDir ./build/es5 -t es5

coverage is back to 100% and instrumenter works perfect! EXCEPT if I add some more code to the main file, then the instrumenter reports wrong line number!

With TypeScript v. 2.0, dead code elimination for ES modules. Meaning you can set taget to ES5 and module to ES6.

The instrumenter doesn't seem to like that either? Is that something to be supported in a future release? Or I'm forced to transpile down to ES5 or use SourceMap?

This can be reproduced easy.
https://github.com/jeibo/workflow/blob/master/scripts/coveralls.sh#L27-L30

If you run this script, and change line 27.

And look at this file: https://github.com/jeibo/workflow/blob/master/src/jeibo.ts

If you run coverage on it, I get reported that line 7 isn't covered. In fact line 3, 4, 5 is the lines who isn't covered! Or line 5, 6, 7 in the transpiled code.

Coverage decreased (-33.3%) to 66.667%

Apologies for [a] being completely naive on the topics of calculating and applying coverage as well as [b] potentially asking this in the wrong forum.

I'm working with Typescript, which is being transpiled via webpack using awesome-typescript-loader. Showing coverage on the original *.ts files requires karma-remap-istanbul. However, it seems like it could/should be more native; plus my IDE (IntelliJ) only picks up on the coverage report from karma-coverage for the transpiled code, not the reported coverage from the remap.

I took a stab at updating istanbul-instrumenter-loader to 1.1.0-alpha.2, hoping it would quickly and easily fulfill my dreams: my changes. Sadly, it did not--coverage pertains to the transpiled code.

I'm unsure of where to continue fumbling around. It is to continue in the realm of the webpack loader? The karma-coverage instrumenter config? Both?

Thanks!

for some reason in version 1.4.0 this code cause error

export const copy = (from, to) => gulp.src(from).pipe(gulp.dest(to))

error output - https://travis-ci.org/thecotne/gulp-companion/builds/188722121#L956

releated issue - istanbuljs/babel-plugin-istanbul#78

my broken build - thecotne/gulp-companion#3

Hi there!

Two things first:

• Thanks a lot for this awesome tool and istanbul/nyc in general ❤️
• I'm not sure I'm on the right repo, though this one looks to me like the most fitting one related to this problem. Let me know if I should move this issue somewhere else

I have discovered a problem with a false positive in code coverage reports, when dealing with switch cases and fallthroughs.

I have created a repo demonstrating the problem and some additional explanations in the README https://github.com/jfmengels/incorrect-coverage. I hope that that is sufficient explanation, but let me know if you wish for me to provide more.

(cc @ericelliott)

After 5b401f5 commit

Ref main issue: istanbuljs/nyc#400.

I'll try to figure out what the problem is, if somebody help I would be glad

Continuation of gotwarlost/istanbul#694 will allow running coverage with almost no overhead, compared to the current situation of instrumented code running 4-5x slower than original, a huge problem in large projects.

The only question I have before I start a PR on this is what option to choose among the following:

1. Change the produced coverage JSON format so that statements: {'1': 1, '2': 1, '3': 0} become statements: [1, 1, 0]. How much of a breaking change this is and will it also require breaking changes in downstream modules?
2. Change the JSON format during instrumented run but convert it back to the old format in the end. Keeps compatibility but hackish and can lead to ugly code.
3. Try changing the format to use zero indexes (e.g. statements: {'0': 1, '1': 1, '2': 0}) — this might be enough to make V8 switch to "fast" mode where it treats it as array rather than dictionary — needs to be checked though.

I'd prefer option 1 but would appreciate hints on the scope of work here. cc @bcoe

update: went with option 3 (zero-based indices).

Thanks!

Coveralls.io and CodeCov both get the same coverage report, but show different result.

Coveralls.io - 66.667%
CodeCov - 71.43%

Could it be issue with comments in the code?

The boilerplate can be found here: https://github.com/Kflash/rachelle

Just click on the Coveralls or the CodeCov badge and see for yourself!

update

Code coverage staff wrOTE this regarding the issue:

Looks like the reports expressed line 1 as being covered. I'll review why this is the case and fix it accordingly. The issue is actually in the report and not with codecov processing.

I'll ping you back shortly and have this fixed up. Thank you for your patience ;)

Steve

Now that istanbul-lib-coverage has released 1.0.0, we should update to that version as it fixes a guard issue that affects babel-plugin-rewire.

More information: speedskater/babel-plugin-rewire#148

PR to follow shortly ...

/to @dtinth @bcoe
/cc @davglass

Hi guys,

I started to work on istanbul v1 after a 6 month hiatus of totally ignoring it. I have, like, 3 weeks
to get a reasonable "v1" out and I'm also looking for an exit strategy in terms of handing modules over
to people who can actually work on it and make it better. Nowadays, it is getting harder and harder for
me to work in this.

In the process, I discovered the new babel plugin-based instrumenter by @dtinth and there are a lot of
good ideas in it, not to mention the simplicity of the code since Babel handles the heavy lifting of
the AST transformation.

The thing that bothered me, though, is that it is yet another variant of code coverage transformations and has different semantics from istanbul's instrumenter (notably the inability to ignore code for instrumentation and a bunch of other smaller differences). Now there is istanbul 0.x, istanbul 1.x-alpha and the __coverage__ babel plugin each of which do subtly to wildly different things and cause friction for the end user who wants to move between ES5/ Typescript/ ES6.

So, @dtinth I see your "shamelessly copied from istanbul" code and raise you my version of shameless copies from your code.

The babel branch in this repo is a sketch of an implementation of instrumentation as a AST babel visitor. It implements "ignores" and steals/ incorporates all the good ideas from the __coverage__ implementation. (There may be some differences that I need to reconcile in a second pass based on what you guys think).

The istanbul instrumentation API is now witten in ES6, and adapted to use Babel under the covers. The programVisitor that is exposed can be be used by a babel plugin that implements source file filters and calls the visitor only when code needs to be instrumented. This, IMO, gives us some semblance of consistency for ES5/ ES6 code coverage.

I'll let you guys check out the code and play with it. All my current tests pass, although es6 tests are pretty light.

I would love to get some early feedback on whether this looks like a promising approach that can be taken to completion.

Also, we can open up this repo for commit access to more folks so you don't have to deal with the
"@gotwarlost is unresponsive" issue :)

Please add more people to the thread as you see fit. I also have some other ideas re: the CLI (TLDR; make nyc the default CLI for istanbul) that we can discuss as we go.

Thanks,
Krishnan

i'm trying to collect coverage for files that don't have tests at all.
The classic way of doing it is just to require a file, but that can become a performance problem and some modules also have side effects.

basically what i think i need is to transpile the file under test into https://github.com/istanbuljs/istanbul-lib-instrument/blob/master/src/visitor.js#L431 this, but without the rest of the file. Is there an easy way to do it right now?

Given the code:

() => A

insertFunctionCounter in visitor.js passes path.node.body.loc as the loc to this.cov.newFunction. However, path.node.body.loc is undefined, so the location ends up looking like:

{end: {}, start: {}}

which is crashing the HTML report downstream.