constellation / ibrik Goto Github PK

CoffeeScript meets Istanbul - code coverage tool for CoffeeScript

License: BSD 2-Clause "Simplified" License

JavaScript 19.52% CoffeeScript 19.96% CSS 18.43% HTML 42.09%

ibrik's Introduction

Ibrik - CoffeeScript meets Istanbul

Ibrik (ibrik) is code coverage tool for CoffeeScript. It uses istanbul and CoffeeScript sourceMaps. Internally, it uses Mozilla JS AST as its IR.

Install

ibrik can be used in in a Node.js application via the package manager:

npm install -g ibrik

Usage

like this

ibrik cover script.coffee
istanbul report html
open coverage/index.html

License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

ibrik's People

Contributors

Stargazers

Watchers

Forkers

jstamerj binchensjtu cuongvo zeno imater onesysadmin scotdalton ardelio juristat 59naga samuelhorwitz lucienh npstobie apoex yalab barionleg

ibrik's Issues

Compiling not with redux

Are you looking forward to support not only that compiler? Because i get some errors trying to get the coverage for code working totally fine.

Coffee-React

Is it possible to support coffee-react?

Ignored exit-code by `ibrik cover _mocha -- test/**/*.spec.coffee`

$ ibrik cover _mocha -- test/**/*.spec.coffee
// -> 0

Exit code should be inherited of mocha's exit code.
Because, result is "build passed" if run this command in travis.

Example:

https://travis-ci.org/59naga/jasminetea/builds/65717291#L189

追伸：
TravisCIでテスト書いてnpm testでテストしながらカバレッジ取ってるんですが、終了コードで1が返らないようです。
試しに、process.exitイベントを監視して、lib/command.jsのコールバック内で受け取った値を返してプロセスを終了するようにしたら、1で落ちるようになりました（下記ビルド65723859#L170を参照）。

Can't parse prototype

Ibrik can't parse code that using prototype, raising Syntax error. Here is example,

@myFunc(@::, 'foo')

Error: Syntax error on line XX, column XX: unexpected ',' (\u002C)

dependencies out of date.

$ npm install david --global
$ david

dependencies

┌───────────────┬─────────┬────────┐
│ Name          │ Package │ Latest │
├───────────────┼─────────┼────────┤
│ esprima       │ 1.2.x   │ 2.2.0  │
├───────────────┼─────────┼────────┤
│ estraverse    │ ^1.9.0  │ 4.1.0  │
├───────────────┼─────────┼────────┤
│ which         │ ~1.0.5  │ 1.1.1  │
├───────────────┼─────────┼────────┤
│ coffee-script │ ~1.8.0  │ 1.9.3  │
├───────────────┼─────────┼────────┤
│ lodash        │ ~2.4.1  │ 3.9.3  │
└───────────────┴─────────┴────────┘

npm install --save [email protected] [email protected] [email protected] [email protected] [email protected]

devDependencies

┌─────────────────┬─────────┬────────┐
│ Name            │ Package │ Latest │
├─────────────────┼─────────┼────────┤
│ chalk           │ ^0.5.1  │ 1.0.0  │
├─────────────────┼─────────┼────────┤
│ mocha           │ ~2.0.1  │ 2.2.5  │
├─────────────────┼─────────┼────────┤
│ chai            │ ~1.10.0 │ 3.0.0  │
├─────────────────┼─────────┼────────┤
│ gulp-coffeelint │ ^0.4.0  │ 0.5.0  │
└─────────────────┴─────────┴────────┘

npm install --save-dev [email protected] [email protected] [email protected] [email protected]

How can ibrik be used with mocha?

We have a suite of Mocha tests and I'm looking for a coverage tool to determine how thorough our tests are. Any idea how Mocha can integrate with ibrik? Thanks.

ibrik fails with class without constructor

testcase:

# cat test.coffee
class Foo
    bar: 1

Foo()
# ibrik cover test.coffee
Transformation error; return original code
TypeError: Cannot read property 'loc' of undefined
    at Controller.<anonymous> (/usr/local/lib/node_modules/ibrik/lib/instrumenter.js:88:38)
    at Controller.estraverse.traverse.leave (/usr/local/lib/node_modules/ibrik/lib/instrumenter.js:125:15)
    at Controller.__execute (/usr/local/lib/node_modules/ibrik/node_modules/estraverse/estraverse.js:318:31)
    at Controller.traverse (/usr/local/lib/node_modules/ibrik/node_modules/estraverse/estraverse.js:384:28)
    at Object.traverse (/usr/local/lib/node_modules/ibrik/node_modules/estraverse/estraverse.js:556:27)
    at Instrumenter.fixupLoc (/usr/local/lib/node_modules/ibrik/lib/instrumenter.js:69:25)
    at Instrumenter.instrumentSync (/usr/local/lib/node_modules/ibrik/lib/instrumenter.js:63:12)
    at /usr/local/lib/node_modules/ibrik/lib/hook.js:20:25
    at Object.require.extensions..coffee (/usr/local/lib/node_modules/ibrik/lib/hook.js:48:13)
    at Module.load (module.js:356:32)
[TypeError: Cannot read property 'loc' of undefined]
No coverage information was collected, exit without writing coverage information

rootcause: such class generates empty BlockStatement (node.body = [])

Karma-coverage install error in Windows 7.0

When I tried to install Karma-coverage version 0.1.3, I get following error

[email protected] build C:\Users\AppData\Roaming\npm\node_modules\karma-covera
ge\node_modules\ibrik
make -f tools/task.mk build

'make' is not recognized as an internal or external command,
operable program or batch file.

npm ERR! [email protected] build: make -f tools/task.mk build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is most likely a problem with the ibrik package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! make -f tools/task.mk build
npm ERR! You can get their info via:
npm ERR! npm owner ls ibrik
npm ERR! There is likely additional logging output above.
npm ERR! System Windows_NT 6.1.7600
npm ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nod
ejs\node_modules\npm\bin\npm-cli.js" "run-script" "build"
npm ERR! cwd C:\Users\Vijay\AppData\Roaming\npm\node_modules\karma-coverage\node
_modules\ibrik
npm ERR! node -v v0.10.22
npm ERR! npm -v 1.3.14
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! C:\Users\Vijay\AppData\Roaming\npm\node_modules\karma-coverage\node
_modules\ibrik\npm-debug.log
npm ERR! not ok code 0
npm ERR! [email protected] install: npm run-script build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the ibrik package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! npm run-script build
npm ERR! You can get their info via:
npm ERR! npm owner ls ibrik
npm ERR! There is likely additional logging output above.

npm ERR! System Windows_NT 6.1.7600
npm ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nod
ejs\node_modules\npm\bin\npm-cli.js" "install" "-g" "karma-coverage"
npm ERR! cwd C:\Users\Vijay\karma-coverage-karma-0.11\karma-coverage-karma-0.11
npm ERR! node -v v0.10.22
npm ERR! npm -v 1.3.14
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! C:\Users\Vijay\karma-coverage-karma-0.11\karma-coverage-karma-0.11
npm-debug.log
npm ERR! not ok code 0

This is because of ibrik dependency specifying its dep through git protocol:
https://github.com/Constellation/ibrik/blob/master/package.json

I think they should use https rather than git, you can file issue at https://github.com/Constellation/ibrik/issues/new
That said, it installs fine on Mac/Linux.

In the meantime, you can install previous version of karma-coverage - npm install -g [email protected] (as long as you don't need to generate coverage for coffee script, which has been added in 0.1.3).

Ibrik not correctly instrumenting some classes.

It seems Ibrik has problems with instrumenting code with empty prototype bodies:

myclass.coffee

class MyClass
    constructor: ->
        @foo()
    foo: ->

myInst = new MyClass()

$ ./node_modules/.bin/ibrik cover temp.coffee
Transformation error; return original code
TypeError: Cannot read property 'loc' of undefined
    at Controller.<anonymous> (./node_modules/ibrik/lib/instrumenter.js:88:38)
    at Controller.estraverse.traverse.leave (./node_modules/ibrik/lib/instrumenter.js:125:15)
    at Controller.__execute (./node_modules/ibrik/node_modules/estraverse/estraverse.js:317:31)
    at Controller.traverse (./node_modules/ibrik/node_modules/estraverse/estraverse.js:383:28)
    at Object.traverse (./node_modules/ibrik/node_modules/estraverse/estraverse.js:555:27)
    at Instrumenter.fixupLoc (./node_modules/ibrik/lib/instrumenter.js:69:25)
    at Instrumenter.instrumentSync (./node_modules/ibrik/lib/instrumenter.js:63:12)
    at ./node_modules/ibrik/lib/hook.js:20:25
    at Object.require.extensions..coffee (./node_modules/ibrik/lib/hook.js:48:13)
    at Module.load (module.js:356:32)
[TypeError: Cannot read property 'loc' of undefined]
No coverage information was collected, exit without writing coverage information

Possible bug fix for reporting coverage with grunt-karma, karma-jasmine and karma-coverage

Hi,

I've found that in ibrik/instrumenter.js, the code that is passed into Istanbul is compiled without the bare : true option.

 Instrumenter.prototype.instrumentSync = function(code, filename) {
      var e, program;
      filename = filename || ("" + (Date.now()) + ".js");
      if (typeof code !== 'string') {
        throw new Error('Code must be string');
      }
      try {
        code = coffee.compile(code, {
            sourceMap: true,
            *** bare : true ***
        });
        program = esprima.parse(code.js, {
          loc: true
        });
        this.fixupLoc(program, code.sourceMap);
        return this.instrumentASTSync(program, filename, code);
      } catch (_error) {
        e = _error;
        e.message = "Error compiling " + filename + ": " + e.message;
        throw e;
      }
    };

Karma kept reporting that my classes in CoffeeScript were undefined until I added that compiler flag in the ibrik source!

Switch Statement Coverage Not Seen

Ibrik is showing this switch statement as "Branch not taken" and the red as "Statement not covered":

But the console.log's logs shown are being hit:

Am I missing something?

outdated lodash - improving performance

There is a newer version of lodash (3.3.1) which improved performance 20-40% overall by better utilizing the JIT in JavaScript engines, using internal helper functions that avoid optimization disqualifications & increase the likelihood of function inlining.

lodash 3.0.0 is 40% (1.40x) faster than lodash 2.4.1.

In v3 also introduced were lazily evaluated chaining for massive performance wins in certain scenarios.

Any chance to update to the latest release?

Installation Error on Windows

Hi,

When attempting to install Ibrik on Windows 8, 64-bits. npm install ibrik results in:

npm WARN `git config --get remote.origin.url` returned wrong result (https://git
hub.com/michaelficarra/CoffeeScriptRedux.git)
npm ERR! git clone https://github.com/michaelficarra/CoffeeScriptRedux.git
npm ERR! Error: spawn ENOENT
npm ERR!     at errnoException (child_process.js:980:11)
npm ERR!     at Process.ChildProcess._handle.onexit (child_process.js:771:34)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Windows_NT 6.2.9200
npm ERR! command "C:\\Users\\x\AppData\\Local\\scoop\\apps\\nodejs\\0.1
0.21\\\\node.exe" "C:\\Users\\x\AppData\\Local\\scoop\\apps\\nodejs\\0.
10.21\\node_modules\\npm\\bin\\npm-cli.js" "install" "ibrik"
npm ERR! cwd C:\Projects
npm ERR! node -v v0.10.21
npm ERR! npm -v 1.3.9
npm ERR! syscall spawn
npm ERR! code ENOENT
npm ERR! errno ENOENT
npm http 304 https://registry.npmjs.org/mkdirp
npm http 304 https://registry.npmjs.org/which
npm http 304 https://registry.npmjs.org/escodegen
npm http 304 https://registry.npmjs.org/istanbul
npm http 304 https://registry.npmjs.org/optimist
npm http 304 https://registry.npmjs.org/estraverse
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     C:\Projects\npm-debug.log
npm ERR! not ok code 0

Question about coffeescriptredux

The coffeescriptredux compiler doesn't seem to support a bunch of coffeescript features. Is it necessary to use this compiler over the official one?

Problem when installing karma-coverage

Hello,

As it has been suggested here : karma-runner/karma-coverage#37, is it possible to change your git protocol to https ?

It causes some trouble when I want to install karma-coverage on a windows 7 installation.

Thanks

CoffeeScriptRedux `extends` compilation handling breaking coverage

michaelficarra/CoffeeScriptRedux#314

I don't know if this is on Ibrik's end or CSR's end, but I've submitted the issue to CSR since it originates in their code. On Ibrik's end, it may be necessary (depending on what CSR says is expected behavior) to somehow clean out the CSR instance between files.

To sum up the other issue, essentially having a file with extends will cause every subsequently processed Coffeescript file to include the helper functions that extends needs when compiled to JS. This causes code coverage to report otherwise 100% covered files as being incompletely covered despite every line in the reports being green because the boilerplate functions like isOwn$ and extends$ are never reached by files that don't actually need to use these.

how can ibrik be used with jasmine-node?

I have figured out how to use istanbul with coffee-coverage to get coverage reports of the compiled version of my code by doing something like this:

coffeecoverage lib lib-cov --bare (note the bare option was recently added to the master branch of the project, it's not published to npm yet)
change spec files to require lib-cov versions
istanbul cover jasmine-node -- test --coffee

However, it would be much better to get the reports to be based on coffee src files as ibrik seems to be able to do. Also, some of the coffee-coverage instrumentation code is being picked up as uncovered, which messes up the stats.

If I try to use ibrik as a drop-in replacement for istanbul (ibrik cover jasmine-node -- test --coffee with spec files pointing to lib) the tests run, but at the end I get No coverage information was collected, exit without writing coverage information.

Any suggestions?

Karma + RequireJs is hanging.

I am using [email protected] and my config is as follows which is hanging. It works fine without requirejs framework.

module.exports = function(config) {
config.set({
basePath: '',
autoWatch: true,
frameworks: ['jasmine','requirejs'],
files: [

  {pattern: '**/nonRequire/*.js', included: true},
  {pattern: '**/require/*.js', included: false},
  'test/spec/*.js'
],
browsers: ['PhantomJS'],

reporters: ['progress', 'coverage'],
preprocessors: { '*.js': ['coverage'] },

coverageReporter : {
    type : 'html',
    dir : 'coverage/'
},

singleRun: true

});
};

"when" clauses in for loops have incorrect line/column branch mapping

The result is a very hard to track down branch coverage shortcoming with no visual indication in the reports. Once stepping through the coverage report JSON, you find a line 0/column 0 branch.

Test Case

Code

window.foo = (arr) ->
    for val in arr when val?
        window.handleExisting val

    return

window.handleExisting = (i) ->
    return i + 1

Test

describe 'Foo', ->
    it 'window.foo', ->
        spyOn window, 'handleExisting'

        window.foo [1, 2, 3]

        expect(window.handleExisting).toHaveBeenCalledWith 1
        expect(window.handleExisting).toHaveBeenCalledWith 2
        expect(window.handleExisting).toHaveBeenCalledWith 3

    it 'window.handleExisting', ->
        expect(window.handleExisting 1).toEqual 2

The coverage reports incomplete branch coverage. In this simple code, it is obvious why: when val? never gets hit with an undefined value. However, this does not get indicated in the coverage report.

Further investigation shows the following JSON (relevant parts only):

{
    "branchMap": {
        "1":{
            "line":0,
            "type":"if",
            "locations":[
                {
                    "start":{
                        "line":0,
                        "column":0
                    },
                    "end":{
                        "line":0,
                        "column":0
                    }
                },
                {
                    "start":{
                        "line":0,
                        "column":0
                    },
                    "end":{
                        "line":0,
                        "column":0
                    }
                }
            ]
        }
    },
    "b": {
        "1": [3,0]
    }
}

Obviously, this branching is not occurring on the non-existent line 0.

To other people finding this issue and need a quick way to track down the uncovered branch:

Go into debug mode of your test runner (I use Karma), or somehow otherwise get ahold of the coverage JSON object
On the coverage object, look through the b key. This key contains branch coverage totals. The b key holds an object of key/value pairs where the key is the branch ID internal to the coverage reporter, and the value is an array of coverage hits. Find the value in b where there is at least one coverage hit of 0 (meaning the branch was never reached). Then take the corresponding ID (in the case above, "1") and access the corresponding branchMap (eg, as above, branchMap["1"]). This branch map metadata will be unhelpful on it's own, as it misreports the line and column, but the branchMap numbering is in order of top of file to bottom of file so if you look at the branches immediately before and after, you will find correct branch coverage data including real line numbers, and your missing branch should lie between these two line numbers. For example, if you are looking at branchMap["32"] because the b["32"] contains a value like [4, 0] (meaning the secondary part of the branch was hit 0 times and thus is the culprit), you should also look at branchMap["31"] and branchMap["33"]. If branchMap["31"] is on line number 180 and branchMap["33"] is on line number 210, then you know branchMap["32"] lies somewhere between 180 and 210, even though it has the erroneous line number of 0.

compile with CoffeeScriptRedux and don't distribute compiled output

CoffeeScriptRedux is a dev dependency, but it looks like the library was being compiled with CoffeeScript 1.4.0. You should add a Makefile with a task for compiling /src/* to /lib with CoffeeScriptRedux. Also, you should add /lib to a .gitignore and require the library to be built before being used.

Error: Syntax error: unexpected '\''

I generated some code using Yeoman (Angular generator with coffeescript). Karma tests are running, I'm trying to add the code coverage task.

It looks like ibrik cannot parse properly the generated code. Why? Is this a bug?

The output is:

Transformation error; return original code
Error: Syntax error on line 11, column 14: unexpected '\'' (\u0027)
08 :  # Controller of the app
09 : ###
10 : angular.module('app')
11 :   .controller 'MycontrollerCtrl', ($scope) ->
^^ :~~~~~~~~~~~~~^
12 :     $scope.awesomeThings = [
13 :       'HTML5 Boilerplate'
14 :       'AngularJS'
    at Object.CoffeeScript.parse (/usr/local/lib/node_modules/ibrik/node_modules/coffee-script-redux/lib/module.js:61:13)
    at Instrumenter.instrumentSync (/usr/local/lib/node_modules/ibrik/lib/instrumenter.js:58:22)
    at /usr/local/lib/node_modules/ibrik/lib/hook.js:20:25
    at Object.require.extensions..coffee (/usr/local/lib/node_modules/ibrik/lib/hook.js:48:13)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at runFn (/usr/local/lib/node_modules/ibrik/lib/cover.js:50:21)
    at /usr/local/lib/node_modules/ibrik/lib/cover.js:102:14
    at /usr/local/lib/node_modules/ibrik/node_modules/istanbul/lib/util/file-matcher.js:56:16
No coverage information was collected, exit without writing coverage information

How to ignore blocks of code

Istanbul has /* istanbul ignore next */. Whats the ibrik way of doing it? Or is that even supported?

Update coffee-script-redux dependency to use a release version over a specific commit

We are looking at caching npm modules for our build process and are using .tgz files to do so.

Since the coffee-script-redux dependency points to a specific commit and does a git clone, this becomes more difficult.

CoffeeScriptRedux has releases, why not just point to one of those?

What is your opinion of this pull request on istanbul?

gotwarlost/istanbul#167

How to use ibrik as a module from outside code

Hi,

Great initial code, but is there a way to use ibrik as a plugin module, for example from a unit testing framework? I have my own unit testing framework called gt that uses istanbul. Right now it can use coffee files as tests or main code, but does not instrument them. It would be great if there was a way I could include ibrik, let it hook into node's require loader and then generate annotated source report on command when unit tests are done.

Here is istanbul module: https://github.com/bahmutov/gt/blob/master/lib/coverage.js

Gleb

how can i use "ibrik instrument" by command line

Separating the instrumenting step from the "coverage" step is quite helpful, p.e. if you want to run integration tests against an instrumented web application.

What we are doing is: instrument our code with istanbul, start a webapp with the intrumented code, then run our integration tests against it, afterwards we generate the reports.

it would be great if we could extend or test setup with coffeescript support and your ibrik istanbul extension looks great and doing almost the job we need.

Do you see a good way to introduce an separate instrument command to ibrik?