gotwarlost / istanbul Goto Github PK

A file path c:\a\b\c.js will emit a wrong key for the coverage object since the backslashes are not escaped in the generated code.

Thanks to @nowamasa for the pull request

With a javascript file test.js

(function () {
    "use strict";

    try {
        var obj = Object.freeze({});
        obj.foo = 1; // will throw
    } catch (e) {
        console.log("ERROR:", e);
        return;
    }
    console.log("Nothing thrown");
}());

Running node test.js prints out

ERROR: [TypeError: Can't add property foo, object is not extensible]

But with instrumented version istanbul instrument test.js > instrumented.js and node instrumented.js I get

Nothing thrown

I’m trying the same command I’m using for other projects to generate the HTML coverage reports:

istanbul cover --report html tests/tests.js

The files coverage.json, prettify.css, prettify.js are generated in the ./coverage directory (as expected), but it seems no HTML files are being created.

I’m using the latest available version of Istanbul. Any ideas?

If I use istanbul cover -- vows *.js and a test breaks (exit's 1), then istanbul should do the same or else it looks like the test is passing when in fact it's failing.

Example:
https://travis-ci.org/#!/yui/shifter/jobs/2854398

Hi, I'm investigating code coverage for node.js code.

You'll may find this dumb, but I cannot figure out on how to generate lcov file (i.e. format described here: http://ltp.sourceforge.net/coverage/lcov/geninfo.1.php).

Something like this:

SF:core/calculator.js
DA:1,1
DA:2,1
DA:10,2
DA:11,2
DA:14,2
DA:20,1
end_of_record

I use a quality dashboard (http://www.sonarsource.org/) to display code quality metrics. And lcov format is the only one accepted.

Regards.

Xavier

Hi

I'm trying to get a test coverage report from an end-to-end acceptance test run and I can't seem to find any information on how to get the coverage results from a running node instance. The goal here is to get a true indication of test coverage through combining the reports from the unit tests, and the end-to-end acceptance tests.

I've used the following:

istanbul instrument src -o ../build/

and I can see instrumented code that puts results into a "coverage" object. Is there any mechanism to collect the coverage for all objects whilst they're running, then to grab a report after all my acceptance tests have run?

Many thanks in advance for your time, and indeed the time you already put in to this fantastic tool!

Sam - Real Time Development

@davglass @reid @mfncooper @sdesai

Starting this thread to see how we can improve branch coverage in Istanbul.

Assumption: Default processing is just about ok in the sense of generality and hitting the cases we care about. If not, let me know.

Based on conversations with some of you here are some desires:

1. Ability to automatically exclude generally-impossible-to-test-without-a-lot-of-shennanigans branches. The big one here is if (object.hasOwnProperty('foo')) {} style branches where the else path should be considered non-existent and istanbul should probably not even consider this if statement as a branch.
2. Ability for user to tag certain branches as unimportant/ untestable. Potential implementation is via comments just before the line of code (similar to jslint). Feel free to send thoughts on what this interface should look like.
3. Ability to specify a reason along with the comment that is somehow shown in the HTML report
4. Ability (for managers) to ensure that people aren't commenting their way out of their responsibilities (perhaps by having "raw" numbers shown against the "processed" numbers when different?)

Anyway, ideas welcome. Just keep this issue updated and we can circle back 3-4 weeks from now and decide what needs to be done. I'll also need to do some research on how comments are made available in the parse tree by esprima before we decide the final solution.

Thanks for Istanbul, which I'm using in a Rails app with Teabag. Though it's a Rails app, I have lots of Javascript code for a Backbone app, and that's mostly in Coffeescript.

Is it possible or planned for Istanbul to report lines numbers on Coffescript source?

It looks like jasmine-coverage-javascript-coffeescript is part of the way there, using coffee-coverage.

Hi,

first of all, pretty awesome project :)
My problem is, that I´am trying to use it as a middleware to collect some coverage informations
from my client side javascript & i´am missing some kind of docs as a starting point how to integrate
everything properly.

I can generate & run instrumented code in the client, but I´am not really sure how to generate a report out of that information. Would be really cool, if one of you guys could add some information on how to do this.

Regards

Trying to instrument this results in an error:

var x = [, , 1];

I'm working on a project with quite a good number of files (298 as picked up by Istanbul to be precise) and it doesn't seem to be able to deal with them :(

Setup is OS X 10.7.5, Node v0.10.0, Istanbul 0.1.32 and using via Guard-Jasmine 1.13.2.

fs.js:413
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^
Error: EMFILE, too many open files 'path-removed.js'
    at Object.fs.openSync (fs.js:413:18)
    at Object.fs.readFileSync (fs.js:270:15)
    at LookupStore.Store.mix.get (/usr/local/share/npm/lib/node_modules/istanbul/lib/store/fslookup.js:39:19)
    at HtmlReport.Report.mix.writeDetailPage (/usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:358:71)
    at /usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:435:26
    at Object.FileWriter.writeFile (/usr/local/share/npm/lib/node_modules/istanbul/lib/util/file-writer.js:24:9)
    at /usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:434:24
    at Array.forEach (native)
    at HtmlReport.Report.mix.writeFiles (/usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:428:23)
    at /usr/local/share/npm/lib/node_modules/istanbul/lib/report/html.js:430:22

I tried using istanbul test in one of my smaller projects and when it executes the tests it reports all 30 are honored, but it just hangs at that point and never exits:

https://github.com/davglass/echoecho

@davglass said:

Also, you could add some flags to set the values of the warnings. Then if the code coverage report doesn't meat these numbers you can process.exit(1) to fail the build.

@gotwarlost said:

Istanbul refrains from explicitly exiting from the process since stdout/ stderr may not fully flush etc. if it gets into this business and could interfere with other exit handlers that the app under test could have set up. So some thinking on that is required before we do the process.exit(1) thing.

Moving the function declarations above the return statement makes istanbul mark them as covered.

For example: the following won't work

function doStuff() {
  return doMoreStuff()

  function doMoreStuff() { // this line is red
    /* .. */
  }
}

But this does work:

function doStuff() {
  function doMoreStuff() { // this line is green
    /* .. */
  }

  return doMoreStuff()
}

• Remove yui-specific processing and move to its own module (e.g. istanbul-yui)
• Use new API provided by @davglass for hooking into YUI in the other module
• Leave the post load hook mechanism in place for the other module to leverage
• Document post load hooking mechanism
• Open-source the new module
• Remove the undocumented --yui flag after all of the above is done

I'm attempting to rework the Whiskey project to make use of Istanbul, and in the interest of making as little changes to Whiskey as possible, am looking for an equivalent to _$jscoverage in Istanbul. In particular, I notice that instrumented files have a coverage object, but it doesn't appear to be exported anywhere like _$jscoverage appears to be. That Whiskey is using _$jscoverage (in a test callback context) suggests to me that it might be exported somehow. However, my gut tells me that I'm sorely misunderstanding something.

Would anyone familiar with jscoverage and Istanbul know how best to gain access to coverage for my purposes? Thanks!

It would be great if the reporter had a legend at the top to tell you what the I, E and highlights mean.

But in the mean time, it would be excellent if the reporter could at least add a title="Description" to them so that users can just mouseover them for a quick explanation.

This info can be useful in some cases.

This list is followed by two lines containing the number  of  functions
found and hit:

 FNF:<number of functions found>
 FNH:<number of function hit>

...

Branch coverage summaries are stored in two lines:

 BRF:<number of branches found>
 BRH:<number of branches hit>

...

At the end of a section, there is a summary about how many  lines  were
found and how many were actually instrumented:

 LH:<number of lines with a non-zero execution count>
 LF:<number of instrumented lines>

source: http://ltp.sourceforge.net/coverage/lcov/geninfo.1.php

Hi,

I've found some odd behaviour in generating the reports. Text and Text-Summary work perfectly.

Running result set over text-summary:

Using reporter [text-summary]

=============================== Coverage summary ===============================
Statements   : 89.36% ( 3803/4256 )
Branches     : 69.23% ( 333/481 )
Functions    : 86.9% ( 418/481 )
Lines        : 98.06% ( 152/155 )
================================================================================
Done

Same for HTML :

Using reporter [html]
Done

/folder/location/node_modules/istanbul/lib/report/html.js:252
                startCol = meta.start.column;
                               ^
TypeError: Cannot read property 'start' of undefined
    at /folder/location/node_modules/istanbul/lib/report/html.js:252:32
    at Array.forEach (native)
    at annotateBranches (/folder/location/node_modules/istanbul/lib/report/html.js:230:30)
    at HtmlReport.Report.mix.writeDetailPage (/folder/location/node_modules/istanbul/lib/report/html.js:374:9)
    at Report.mix.writeFiles (/folder/location/node_modules/istanbul/lib/report/html.js:436:26)
    at AsyncFileWriter.extend.processFile (/folder/location/node_modules/istanbul/lib/util/file-writer.js:93:9)
    at async.queue.q.process (/folder/location/node_modules/istanbul/node_modules/async/lib/async.js:725:21)
    at process.startup.processNextTick.process._tickCallback (node.js:244:9)

Have you seen anything like this before?

The only way I could get around this, and the coverage data still looks correct, is by updating the specific html.js file to handle no metaData attribute:

Updating the html.js file from this (line 246):

        if (sumCount > 0) { //only highlight if partial branches are missing
            for (i = 0; i < branchArray.length; i += 1) {
                count = branchArray[i];
                meta = metaArray[i];
                type = count > 0 ? 'yes' : 'no';

to this:

        if (sumCount > 0) { //only highlight if partial branches are missing
            for (i = 0; i < branchArray.length **&& metaArray[i]**; i += 1) {
                count = branchArray[i];
                meta = metaArray[i];
                type = count > 0 ? 'yes' : 'no';

(Haven't done it in a pull request, as i really have no idea if its a valid bug or not)

Cheers,

Tom

We are using jasmine-node to test our server side code. Because we read in many blogs, that istanbul code coverage is awesome we wanted to give it a try. Therefore we installed istanbul. Jasmine-node runs fine and istanbul with karma (testacular) also runs fine. But combining jasmine-node and istanbul doesn't work for us on Windows 7 64bit. When we run the following command:

nodes_modules.bin\istanbul.cmd cover node_modules.bin\jasmine-node.cmd test/server/nmc/unit

We get:
//-------------------------------------------------------------------------------------------------------------------------------
(function (exports, require, module, __filename, __dirname) { @if EXIST "%~dp0
^
No coverage information was collected, exit without writing coverage information

SyntaxError: Unexpected token ILLEGAL
at Module._compile (module.js:439:25)
at Module._extensions..js (module.js:474:10)
at Object.Module._extensions..js (D:\Studium\DiplomArbeit\git\nomocom\protot
yping\to-scaffold\node_modules\istanbul\lib\hook.js:101:13)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at runFn (D:\Studium\DiplomArbeit\git\nomocom\prototyping\to-scaffold\node_m
odules\istanbul\lib\command\common\run-with-cover.js:88:16)
at D:\Studium\DiplomArbeit\git\nomocom\prototyping\to-scaffold\node_modules
istanbul\lib\command\common\run-with-cover.js:191:17
at D:\Studium\DiplomArbeit\git\nomocom\prototyping\to-scaffold\node_modules
istanbul\lib\util\file-matcher.js:52:16
at D:\Studium\DiplomArbeit\git\nomocom\prototyping\to-scaffold\node_modules
istanbul\lib\util\file-matcher.js:35:9
//-------------------------------------------------------------------------------------------------------------------------------

Running the same command on Ubuntu works fine and we get a nice code coverage. On Ubuntu we use:

nodes_modules/.bin/istanbul cover node_modules/.bin/jasmine-node test/server/nmc/unit

Are we screwing something up, or is there a lack of compatibility with Win7? Does someone know how to fix this issue?

EDIT:
Our node version is: 0.10.5 and npm is on 1.2.18, if you need more information, let me know

Thank you
Tschoartschi

It would be nice if the options for text and text-summary were documented in the CLI. If you can point me to where to add this, I'd love to contribute.

RequireJS uses vm.runInThisContext to evaluate the AMD modules when running on node.js. I think the cover command should add hooks to runInThisContext and createScript automatically or at least have an easy way to set it.

Since transformFn isn't exposed to other modules I ended up editing istanbul/lib/hook.js directly. Not the most elegant solution but worked on my scenario (loading specs and source code with RequireJS):

diff --git a/node_modules/istanbul/lib/command/common/run-with-cover.js b/node_modules/istanbul/lib/command/common/run-with-cover.js
index a9e6e88..b43726d 100644
--- a/node_modules/istanbul/lib/command/common/run-with-cover.js
+++ b/node_modules/istanbul/lib/command/common/run-with-cover.js
@@ -1,3 +1,6 @@
+// this file was edited by Miller Medeiros to support RequireJS on node.js
+// changes are marked with [mm]
+
 /*
  Copyright (c) 2012, Yahoo! Inc.  All rights reserved.
  Copyrights licensed under the New BSD License. See the accompanying LICENSE file for terms.
@@ -124,6 +127,11 @@ function run(args, commandName, enableHooks, callback) {
                 if (opts['self-test']) {
                     hook.unloadRequireCache(matchFn);
                 }
+
+                // ---------- added [mm] -------------------- //
+                hook.hookRunInThisContext(matchFn, transformer, hookOpts);
+                // ------------------------------------------ //
+
                 hook.hookRequire(matchFn, transformer, hookOpts);
                 process.once('exit', function () {
                     var file = path.resolve(reportingDir, 'coverage.json'),
@@ -161,3 +169,4 @@ module.exports = {
     run: run,
     usage: usage
 };
+
diff --git a/node_modules/istanbul/lib/hook.js b/node_modules/istanbul/lib/hook.js
index a22a30a..f5a28f6 100644
--- a/node_modules/istanbul/lib/hook.js
+++ b/node_modules/istanbul/lib/hook.js
@@ -1,3 +1,7 @@
+// this file was edited by Miller Medeiros to support RequireJS on node.js
+// changes are marked with [mm]
+
+
 /*
  Copyright (c) 2012, Yahoo! Inc.  All rights reserved.
  Copyrights licensed under the New BSD License. See the accompanying LICENSE file for terms.
@@ -130,6 +134,21 @@ function hookCreateScript(matcher, transformer, opts) {
     };
 }

+// ======== edited [mm] ======== //
+
+var originalRunInThisContext = vm.runInThisContext;
+
+function hookRunInThisContext(matcher, transformer, opts) {
+    opts = opts || {};
+    var fn = transformFn(matcher, transformer, opts.verbose);
+    vm.runInThisContext = function (code, file) {
+        var ret = fn(code, file);
+        return originalRunInThisContext(ret.code, file);
+    };
+}
+
+// ========================== //
+
 /**
  * unhooks vm.createScript, restoring it to its original state.
  * @method unhookCreateScript
@@ -142,6 +161,7 @@ function unhookCreateScript() {
 module.exports = {
     hookRequire: hookRequire,
     unhookRequire: unhookRequire,
+    hookRunInThisContext : hookRunInThisContext, // added [mm]
     hookCreateScript: hookCreateScript,
     unhookCreateScript: unhookCreateScript,
     unloadRequireCache: unloadRequireCache

Not doing a pull request since I'm focused on other tasks and don't have time to write the tests. Just a FYI that other hooks might be important on different scenarios.

I have it working on the amd-utils travis build, so it can be used as a reference, see tests/runner.js and tests/spec folder for details about my tests: https://github.com/millermedeiros/amd-utils/blob/master/tests/ (source code and tests are written in AMD and are running on node.js and browser).

Cheers.

When a script contains an "if" without an "else" the file is flagged for not having covered the "else" that isn't there. This changes the total coverage of the branches incorrectly when the contents of the "if" are fully covered. The lone "if" should be detected correctly.

What is the current version number of Istanbul?

Reproducing Dav's comments from issue #1

However, it should be required that the code live in the json file and optted out for performance. That
way you can generate reports from the JSON file in the future without having the code provided. If all
the relevant data is in the JSON file, then istanbul can generate an lcov file and a directory structure
based on what's in the JSON file.

It's kind of a catch 22, the lcov file isn't good for long term storage if the files that it's instrumented are
different and are required to generate the report. So storing the code in the JSON file as a permanent
place to restore that report from seems like it's a better option.

Personally, I'm not a big fan of packing code into the coverage object by default. Especially for the browser testing use case where, IMO, the goal should be for the instrumented code to be as small as possible to reduce the test latency.

The approach I favor is to support a new zip command, that is run after all the tests which, at a high level, stores enough state to reproduce all the reports at a future point in time. Note that I did not say "packs code into the coverage object" since with hundreds of files all the code + coverage may not fit into memory.

The state could, instead, be a zip file containing a bunch of JSON coverage files and a bunch of source files all packed in. The file paths in the coverage objects are mangled so that they relatively reference the source files.

I must admit a lot more thinking/ discussion is required on my part to crystallize this. Feedback is most welcome.

I want to run my tests using node --harmony

I tried istanbul cover ./test/index.js -- --harmony but that does not work.

So I'm hoping that the issue is simple and is just me not using istanbul correctly.

I'm using node.js to run Istanbul via the command line.
Using dojo version 1.8.

The problem is Istanbul doesn't seem to be indexing all of the javascript files that are run.

I have two scenarios that show the problem.

Scenario #1: I use dojo's require method to run a javascript file called "test.js". In this scenario, dojo's declare method works and all of the lines in test.js execute (I know this through console.log).
Scenario #1 Problem: Istanbul does not include test.js or show the coverage for this file.

Scenario #2: I use the normal require to run a javascript file called "test.js". In this scenario, dojo's declare method partially works. The second argument (a function) is never executed.

Scenario #2 Problem: Istanbul includes test.js but the function in the declare is not executed.

Here is the test.js.

global.require([], function(){
console.log("Hello World!");
});
global.define([],function(){
console.log('yey');
});

console.log('boo');

Scenario #1 (istanbul-test.js)

dojoConfig = {
async: 1, // We want to make sure we are using the "modern" loader
hasCache: {
"host-node": 1, // Ensure we "force" the loader into Node.js mode
"dom": 0 // Ensure that none of the code assumes we have a DOM
},
packages: [{name: "dojo",location: "dojo"}]
};
require('./dojo/dojo.js');
//global.require is dojo's require
global.require(dojoConfig, ["./test.js"], function(){
console.log('It is finished!');
});

Scenario #2 (istanbul-test2.js)

dojoConfig = {
async: 1, // We want to make sure we are using the "modern" loader
hasCache: {
"host-node": 1, // Ensure we "force" the loader into Node.js mode
"dom": 0 // Ensure that none of the code assumes we have a DOM
},
packages: [{name: "dojo",location: "dojo"}]
};
var dojo = require('./dojo/dojo.js');
require( "./test.js");

When using istanbul on ES6 code it prints a bunch of errors

Transformation error; return original code
{ [Error: Line 7: Unexpected token *] index: 174, lineNumber: 7, column: 9 }
Transformation error; return original code
{ [Error: Line 11: Unexpected token *] index: 266, lineNumber: 11, column: 9 }
Transformation error; return original code
{ [Error: Line 11: Unexpected token *] index: 323, lineNumber: 11, column: 9 }
Transformation error; return original code
{ [Error: Line 10: Unexpected token *] index: 241, lineNumber: 10, column: 9 }
Transformation error; return original code
{ [Error: Line 9: Unexpected token *] index: 206, lineNumber: 9, column: 9 }
Transformation error; return original code
{ [Error: Line 8: Unexpected token *] index: 223, lineNumber: 8, column: 9 }
Transformation error; return original code
{ [Error: Line 12: Unexpected token *] index: 280, lineNumber: 12, column: 9 }
Transformation error; return original code
{ [Error: Line 8: Unexpected token *] index: 174, lineNumber: 8, column: 9 }
Transformation error; return original code
{ [Error: Line 9: Unexpected token *] index: 194, lineNumber: 9, column: 9 }
Transformation error; return original code
{ [Error: Line 11: Unexpected token *] index: 260, lineNumber: 11, column: 13 }
Transformation error; return original code
{ [Error: Line 12: Unexpected token *] index: 302, lineNumber: 12, column: 13 }

The result is that every piece of code I have that uses generators is not instrumented.

Since all reports default to async, it would be very handy if they supported callbacks to tell us when the reports have finished rendering.

I think this is the right spot for the request.

Istanbul was recently integrated into Testacular, and it's pretty fantastic. It would be nice to see a format that Jenkins can natively handle, rather than lcov and/or HTML.

It looks like people tend to favor using the Cobertura plugin... and if they have incoming LCov, they convert.
I found an lcov-tocobertura tool, but it's in Python :( Not sure if there are other / better options here I just haven't come across.

Also see post on SO regarding lcov in Hudson / Jenkins.

/cc @taichi

I opened an issue at Testacular to track this as well.
https://github.com/vojtajina/testacular/issues/164#issuecomment-10298818

Thanks for the consideration!

I have commited istanbul module in a SVN repository.
When I try to generate a HTML report I have the error :

Fatal error: EISDIR, illegal operation on a directory

The cause is the presence of the .svn repository

I have lots of projects that need to work in browsers but also in Node, Narwhal.js, Rhino, and other JavaScript environments. Example coverage report: http://rawgithub.com/mathiasbynens/esrever/master/coverage/esrever/esrever.js.html
The main code starts at line 20 and ends at line 41. Everything else is the export logic that covers all those environments.

Of course, since Istanbul uses Node to run the tests, the code paths for the other environments won’t be reached. But this is also causing the coverage % of the file to drop.

It would be nice if it was possible to somehow configure Istanbul to ignore some parts of the code and just consider them as covered, or just display them greyed out in the report (to signify that they were ignored while calculating the coverage % of the file). Thoughts?

I am using testacular/istanbul to generate a js coverage report, I can get the report, however, the chinese charater within the code has been displayed wrong.

describe("Hello world", function() {
it("HelloWorldTest", function() {
expect(helloWorld()).toEqual("���Һ�!");
});
});

should be

describe("Hello world", function() {
it("HelloWorldTest", function() {
expect(helloWorld()).toEqual("大家好!");
});
});

Although Jenkins is able to report the coverage metrics on a per-file basis, it is not able to display the original source code from the current coverage-cobertura.xml format. The Jenkins cobertura plugin reports:

Source code is unavailable. Some possible reasons are:

• This is not the most recent build (to save on disk space, this plugin only keeps the most recent build’s source code).
• Cobertura found the source code but did not provide enough information to locate the source code.
• Cobertura could not find the source code, so this plugin has no hope of finding it.
• You do not have sufficient permissions to view this file.

Currently the XML report outputs only an empty <sources/> node, and each <file/> node contains the full, absolute path to the file. In order to get Jenkins to display the source code for a file, I have to re-create the full absolute path as if it is relative to the job's cobertura/ directory. Consider this coverage.xml excerpt:

<sources/>
...
<class name="my_file.js" filename="/var/lib/jenkins/workspace/MyProject/lib/my_file.js">
...

Given this, the Jenkins plugin cannot display source, and the only way to get it to show the source is by:

$ cd /var/lib/jenkins/jobs/MyProject/cobertura/
$ mkdir -p ./var/lib/jenkins/workspace/MyProject/lib/
$ cp /var/lib/jenkins/workspace/MyProject/lib/my_file.js /var/lib/jenkins/jobs/MyProject/cobertura/var/lib/jenkins/workspace/MyProject/lib/my_file.js

That allows Jenkins to render the source code properly. I believe instead, the report should generate the coverage.xml file with:

<sources>
  <source>/var/lib/jenkins/workspace/MyProject</source>
</sources>
...
<class name="my_file.js" filename="lib/my_file.js">
...

This allows the Jenkins plugin to archive the source files relative to the <sources/> paths. Looking at the Jenkins plugin, it only looks at the <sources/> nodes for finding the source paths (see https://github.com/jenkinsci/cobertura-plugin/blob/master/src/main/java/hudson/plugins/cobertura/CoberturaPublisher.java#L377). If the sourcePaths set is empty, it has no paths or files to look for. Also see https://github.com/jenkinsci/cobertura-plugin/blob/master/src/main/java/hudson/plugins/cobertura/CoberturaCoverageParser.java#L164 for the actual SAX parser that collects the sourcePaths set from the XML file.

When using istanbul from your package.json, it would be nice if it could print a small report at the end.

This way, when using Travis, we have a record of the coverage numbers after the tests are run. Since Travis doesn't track build assets and it only tracks the log output. Putting a simple report in the output would be handy.

Also, you could add some flags to set the values of the warnings. Then if the code coverage report doesn't meat these numbers you can process.exit(1) to fail the build.

When i try to cover javascript code, written for browser - i am getting errors for browser supported variables such as document is not defined. etc. is there something i should set as config, am i missing something ?

i was trying like this from Command Line Interface "istanbul cover myjs.js"

Moreover, i am not sure, if it is the right place to ask such questions.

that seem to fail for random reasons that I cannot reproduce locally.

If you have multiple files being covered, the text/ html reports remove the common prefix. However when a project has a single file the whole directory path shows up.

We're adding coverage to an existing project, and many files are not covered. Since the coverage json generated from the instrumented code essentially reports 'undefined' for files that weren't touched during the testing, our coverage stats are inaccurate (i.e. way too high).

My proposal is to provide an option in the instrument CLI to generate a 'coverage-baseline.json' file for the files instrumented, such that if you ran a report on the generated file, you'd see all the files, with 0% coverage.

Seems that the report generator assumes that the code for the file being rendered is always relative to where it's being run from.

When it's generating foo/foo.js.html, it seems that it's looking for foo/foo.html relatively. There are a couple of issues with this.

• If you are doing this dynamically and you tell it to put the code in the results, then that should take precedence over the file on disk.
• This means that you have to know where the code is hosted and do a hard coded process.chdir() to just above the source tree in order to render the reports.

I have some tests that all pass when run without istanbul, but fail under coverage testing.

steps to reproduce:

git clone [email protected]:isao/mojito-cli.git --branch istanbul
cd mojito-cli/
npm i
npm test
# all pass
npm run cover
#32 tests fail

The failing tests are ones that examine buffered log statements using npmlog. The when run under istanbul, the message strings sometimes are not formatted (using util.format internally), or have leading spaces.

      FOUND:   Usage: mojito version [app] [mojit <moj
      WANTED: Usage: mojito version [app] [mojit <moji
              ^ (at position = 0)
...
      FOUND:   no package.json found at %s mojits/foo
      WANTED: no package.json found at mojits/foo
              ^ (at position = 0)

istanbul version:0.1.34
node v0.10.4

After installing istanbul with 'sudo npm -g istanbul', using it throws the following error:

/usr/local/lib/node_modules/istanbul/node_modules/escodegen/package.json:2
"name": "escodegen",
^
Unexpected token :
SyntaxError: Unexpected token :
at Module._compile (module.js:406:25)
at Object..js (module.js:417:10)
at Module.load (module.js:343:31)
at Function._load (module.js:302:12)
at require (module.js:355:19)
at /usr/local/lib/node_modules/istanbul/node_modules/escodegen/escodegen.js:2249:23
at Object. (/usr/local/lib/node_modules/istanbul/node_modules/escodegen/escodegen.js:2253:1)
at Module._compile (module.js:411:26)
at Object..js (module.js:417:10)
at Module.load (module.js:343:31)

node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Could not register command from file instrument.js
at /usr/local/lib/node_modules/istanbul/lib/util/factory.js:60:27
at Array.forEach (native)
at Object.loadStandard (/usr/local/lib/node_modules/istanbul/lib/util/factory.js:53:29)
at Function.loadAll (native)
at Object. (/usr/local/lib/node_modules/istanbul/lib/register-plugins.js:12:9)
at Module._compile (module.js:411:26)
at Object..js (module.js:417:10)
at Module.load (module.js:343:31)
at Function._load (module.js:302:12)
at require (module.js:355:19)

My istanbul cover commands fail on windows, but the vows --spec ./test/foo.js works fine.

I get this error:

YUI@WINDOWS7 ~/Desktop/play/cpr (master)
$ npm test

> [email protected] pretest c:\Users\YUI\Desktop\play\cpr
> jshint --config ./node_modules/yui-lint/jshint.json ./lib/


> [email protected] test c:\Users\YUI\Desktop\play\cpr
> istanbul cover --print both -- vows --spec ./tests/full.js


c:\Users\YUI\Desktop\play\cpr\node_modules\.bin\vows.CMD:1
(function (exports, require, module, __filename, __dirname) { :: Created by np
                                                              ^
Unexpected token :
SyntaxError: Unexpected token :
    at Module._compile (module.js:437:25)
    at Module._extensions..js (module.js:467:10)
    at Object.Module._extensions..js (c:\Users\YUI\Desktop\play\cpr\node_modules
\istanbul\lib\hook.js:101:13)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:492:10)
    at runFn (c:\Users\YUI\Desktop\play\cpr\node_modules\istanbul\lib\command\co
mmon\run-with-cover.js:80:16)
    at c:\Users\YUI\Desktop\play\cpr\node_modules\istanbul\lib\command\common\ru
n-with-cover.js:148:17
    at c:\Users\YUI\Desktop\play\cpr\node_modules\istanbul\lib\util\file-matcher
.js:52:16
    at c:\Users\YUI\Desktop\play\cpr\node_modules\istanbul\lib\util\file-matcher
.js:35:9
No coverage information was collected, exit without writing coverage information

npm ERR! Test failed.  See above for more details.
npm ERR! not ok code 0

YUI@WINDOWS7 ~/Desktop/play/cpr (master)
$

For reference, that vows is running:

YUI@WINDOWS7 ~/Desktop/play/cpr (master)
$ ./node_modules/.bin/vows --spec ./tests/full.js

  ♢ CPR Tests

  should be loaded
    ✓ should have cpr method
  should fail on non-existant from dir
    ✓ should return an error in the callback
  should be loaded and should NOT copy node_modules
    ✓ does not have ./out/1
    ✓ and threw an error
  should be loaded and should not copy graceful-fs from function
    ✓ and has ./out/3
    ✓ and dirs are not equal
    ✓ and from directory has graceful-fs dir
    ✓ and to directory does not have graceful-fs dir
  should be loaded and should copy node_modules
    ✓ has ./out/0
    ✓ and dirs are equal
    ✓ and from directory has graceful-fs dir
    ✓ and to directory has graceful-fs dir
  should be loaded and should not copy yui-lint from regex
    ✓ and has ./out/2
    ✓ and dirs are not equal
    ✓ and from directory has yui-lint dir
    ✓ and to directory does not have yui-lint dir
  should be loaded and should copy node_modules with overwrite flag
    ✓ has ./out/0
    ✓ and dirs are equal
    ✓ and from directory has graceful-fs dir
    ✓ and to directory has graceful-fs dir

✓ OK » 20 honored (10.885s)


YUI@WINDOWS7 ~/Desktop/play/cpr (master)
$

It would be really nice if the coverage-check was done on a per file basis. (I want to make it part of the CI build on a semi-large team project)
If I have 100 files around 90% an one at 5% I would like a report detailing the culprit file.

It would also be great, you you could specify a default threshhold (say 90%), but at the same time be able to.

• ignore files (patterns?)
• set a special threshold for individual files (patterns?)

HTML report should support source maps. This would allow the reports to be far more digestible when running against browserify bundles (https://github.com/substack/node-browserify).

Istanbul works flawlessly with jasmine-node, using the command: `istanbul cover jasmine-node test'

Is it possible to use istanbul with mocha, in a similar way. I get the following:

$ istanbul cover mocha -u exports -R spec

fs.js:684
  return binding.stat(pathModule._makeLong(path));
                 ^
Error: ENOENT, no such file or directory 'exports.js'
    at Object.fs.statSync (fs.js:684:18)
    at lookupFiles (/usr/local/share/npm/lib/node_modules/mocha/bin/_mocha:390:17)
    at spinner (/usr/local/share/npm/lib/node_modules/mocha/bin/_mocha:268:24)
    at Array.forEach (native)
    at Object.<anonymous> (/usr/local/share/npm/lib/node_modules/mocha/bin/_mocha:267:6)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3
No coverage information was collected, exit without writing coverage information

I tried using https://github.com/arikon/mocha-istanbul as the reporter but got a similar error.

It would be really nice to drive some things in Istanbul with a json file.

• Custom tolerances for coverage warnings
• Output directory
• Possible theme location

Istanbul should also walk up the directory tree looking for the json file, if it's not found then use the defaults.

Potentially, it could also look for a package.json file, if found, parse it and check for an istanbul key and use that as the configuration.

It could be "minimum uncovered unit". For example, in the case of statement coverage, enforcing 4 out 5 statements can be in the form of:

istanbul check-coverage --statement -1

because only 1 statement should be left uncovered. If someone adds a new line which won't be covered, this triggers the warning.

For some projects, including Esprima itself, this is much more convenient than a percentage-based threshold.