check coverage on a per-file basis about istanbul

[gotwarlost]

This is a very good idea but managing this on the command line is going to be a disaster :)

This can be done after #3 is fixed.

[ebdrup]

agreed 👍

[BryanDonovan]

This would be great... I'm willing to try to work on it.

I think the per-file checks would just be off by default. If you want them on, you pass in the command-line options. It's not too hard to put that in a script.. I mean who types out

--lines 90 --statements 85 --branches 60 --functions 85

every time they run istanbul anyway? We're programmers, we can put that in a Makefile or other script :)

[gotwarlost]

sure, glad to have contributions. Please note that I'm on vacation for the next 3 weeks so will probably not respond to questions, if any.

Thanks!

[markwongsk]

I'm going to assume from this and the declined PR that this issue hasn't been solved?

[geekdave]

@gotwarlost : Jumping in late to this thread. Does this issue represent the latest on where Istanbul stands with regard to per-file coverage?

We're looking for a solution that will allow us to enforce code coverage on a per-file level. The reason is that our project has thousands of JS files. If if we only enforce an average threshold, it will allow lazy devs to sneak in untested changes, and bank on the good code coverage produced by other developers tests.

I wrote the grunt-blanket-mocha task to achieve this same result, using BlanketJS under the hood. But since BlanketJS does all the coverage instrumentation live in-memory, we're starting to hit errors with our huge codebase.

Any perspectives from you on the direction that you'd like to see Istanbul go with this would be very helpful. We'd be willing to work on implementation if it's a feature that's likely to be accepted (though I do see a rejected PR for this, so just wanted to make sure).

/cc @ryan-roemer

[gotwarlost]

#3 (comment)

#91 (comment)

istanbul now has a configuration YAML that could be used for adding these kinds of checks. Possibly as a new check section as a top-level key.

This could have a subsection for the checks that istanbul does implement today (e.g. global) and another (every-file ?) for the kinds of checks you want.

What I would really like to see is pattern-specific matches such that I can say: lib/foo/* must have at least 95% line-coverage but that might take while to implement correctly and arguably is a more complex use-case.

If we just implemented the 2 sections one for global and another for every-file that would allow us to represent the current checks in config, add the checks you want and still leave the door open for more sophisticated checks later (e.g. a patterns subsection), that would work for me too.

[geekdave]

Thanks @gotwarlost - Will attempt a stab at this in the coming weeks.

[ryan-roemer]

@gotwarlost -- I'm just starting to kick the tires on this. If you don't mind reviewing the following before I start, here's what I'm thinking:

• No new command line flags (route all through YML config). No particular reason for this, just easier to iron out capabilities in YML first, then maybe add these later.
• Add following checks: each-file, patterns-global, patterns-each-file

YML looks like:

check:
  # Existing coverage checks.
  # For all files as an *aggregate*.
  global:
    statements: 70
    lines: 70
    branches: 70
    functions: 70

  # For each and every file on an individual basis.
  each:
    statements: 60
    lines: 60
    branches: 60
    functions: 60

  # Patterns
  # For *groups* of files as an *aggregate*
  patterns-global:
    lib/foo/*:
      statements: 90
      lines: 90
      branches: 90
      functions: 90
    lib/bar/*:
      # ...

  # For *groups* of files on a per-file basis for each file in the group.
  patterns-each:
    lib/foo/*:
      statements: 90
      lines: 90
      branches: 90
      functions: 90
    lib/bar/*:
      # ...

Does that sound roughly right?

@geekdave -- I've started hacking this in https://github.com/FormidableLabs/istanbul/compare/feature-per-file-coverage (just notes, no work). Let me know if you want read/write access...

[ryan-roemer]

@gotwarlost @geekdave -- Have a PR out at #268 for the global and each (per-file) version of this (now config-based). Hope to see if we can work this up in shape to merge in!

[gustavnikolaj]

@gotwarlost, this issue seems to be resolved after #268 was merged, or is there something I'm missing?

[MattiSG]

I also understand that this is solved with the each part of the config.

[ryan-roemer]

The only thing I think is missing from the original issue request and my strawman above is pattern matching configuration params like patterns-global and patterns-each or something.

But may be appropriate for a new ticket if we want to cap off this particular one since it has an implementation PR already.