scoverage / sbt-scoverage

sbt plugin for scoverage

sbt-scoverage

sbt-scoverage is an sbt plugin that offers support for Scala code coverage using scoverage. This plugin supports Scala 2.12, 2.13, and 3.

NOTE: that ScalaJS and Scala Native support is limited to Scala 2. NOTE: that Scala 3 support starts with 3.2.x.

Setup

Requirements: Requires sbt 1.2.8 or above

In project/plugins.sbt:

addSbtPlugin("org.scoverage" % "sbt-scoverage" % "x.x.x")

If you are in an enterprise environment, and the above does not work, try:

libraryDependencies += "org.scoverage" % "sbt-scoverage_2.12_1.0" % "x.x.x"

Usage

Run the tests with enabled coverage:

sbt clean coverage test

or if you have integration tests as well

sbt clean coverage it:test

To enable coverage directly in your build, use:

coverageEnabled := true

To generate the coverage reports run

sbt coverageReport

Coverage reports will be in your target/scala-<scala-version>/scoverage-report directory. There are HTML and XML reports. The XML is useful if you need to programatically use the results, or if you're writing a tool.

NOTE: If you're running the coverage reports from within an sbt console session (as opposed to one command per sbt launch), then the coverage command is sticky. To turn it back off when you're done running reports, use the coverageOff command or reset coverageEnabled with set coverageEnabled := false.

Multi project reports

By default, scoverage will generate reports for each project separately. You can merge them into an aggregated report by using the following:

$ sbt coverageAggregate

NOTE: You do not need to run coverageReport before coverageAggregate; it aggregates over the sub-projects' coverage data directly, not the report xml.

Exclude classes and packages and files

You can exclude classes from being considered for coverage measurement by providing semicolon-separated list of regular expressions.

coverageExcludedPackages := "<empty>;Reverse.*;.*AuthService.*;models\\.data\\..*"

The regular expressions are matched against the fully qualified class name, and must match the entire string to take effect. Any matched classes will not be instrumented or included in the coverage report.

You can also exclude files and file paths.

coverageExcludedFiles := ".*\\/two\\/GoodCoverage;.*\\/three\\/.*"

Note: The .scala file extension needs to be omitted from the filename, if one is given.

Note: These two options only work for Scala2. Right now Scala3 does not support a way to exclude packages or files from being instrumented.

You can also mark sections of code with comments like:

  // $COVERAGE-OFF$Disabling highlighting by default until a workaround for https://issues.scala-lang.org/browse/SI-8596 is found
  ...
  // $COVERAGE-ON$

Any code between two such comments will not be instrumented or included in the coverage report.

Minimum coverage

Based on minimum coverage, you can fail the build with the following keys:

coverageFailOnMinimum := true
coverageMinimumStmtTotal := 90
coverageMinimumBranchTotal := 90
coverageMinimumStmtPerPackage := 90
coverageMinimumBranchPerPackage := 85
coverageMinimumStmtPerFile := 85
coverageMinimumBranchPerFile := 80

These settings will be enforced when the reports are generated. If you generate an aggregate report using coverageAggregate then these settings will apply to that report.

Override Location for Coverage Data And Report

If desired, one could override the default location for generating the sbt report and data through setting coverageDataDir:

Example in data-dir test:

coverageDataDir := target.value / "custom-test"

Can also be set through the sbt set directive

set coverageDataDir := file("/tmp")

Trouble-shooting failing tests

scoverage does a lot of file writing behind the scenes in order to track which statements have been executed. If you are running into a scenario where your tests normally pass, but fail when scoverage is enabled, then the culprit can be one of the following:

• timing issues on futures and other async operations, try upping the timeouts by an order of magnitude.
• tests are run in a sandbox mode (such as with java.security.PrivilegedAction<T>), try running the tests outside of the sandbox.

Example project

the scoverage samples project.

Integrations

Codacy

Codacy integrates with your favorite coverage tool to provide an in-depth overlook of your project status. scoverage information can be integrated into Codacy through the codacy-coverage-reporter.

Coveralls

If you have an open source project then you can add code coverage metrics with the Coveralls. scoverage will integrate with coveralls using the sbt-coveralls plugin.

Codecov

You can integrate with Codecov easily sending your reports there via your CI. You can see an example of this here in codecov/example-scala.

Plugin for SonarQube

If you want to visually browse statement coverage reports then use this plugin for SonarQube. It allows you to review overall project statement coverage as well as dig deeper into sub-modules, directories and source code files to see uncovered statements. Statement coverage measurement can become an integral part of your team's continuous integration process and a required quality standard.

Release Notes

For any information on releases and upgrading, please refer to the release page.

sbt-scoverage's Issues

add more functional tests

Now that #23 is done, we have a framework (albeit clunky) for running functional tests.

Some stuff off the top of my head:

• check that coverage reports get generated
• check that classes get ignored

Integration with Play Framework 2.3: OutOfMemoryError: GC overhead limit exceeded

After the instrumentation, I am getting GC exception. I have both "-Xms2048M" and "-Xmx2048M" allocate 2GB of memory which I think is enough. Do anyone have any idea how to get around this?

[scoverage]: Instrumentation completed [50838 statements]
[scoverage]: Written instrumentation file [/Users/mayumi.liyanage/wm_browse/target/scala-2.10/scoverage-data/scoverage.coverage.xml]
[scoverage]: Writing measurements data to [/Users/mayumi.liyanage/wm_browse/target/scala-2.10/scoverage-data]
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at sbt.ConcurrentRestrictions$$anon$4.take(ConcurrentRestrictions.scala:196)
.........

Thanks

HTML Report not generated when Tests are exluded via Filter

Hey,

i originally opened the following bug, which i am still stuggling with: #52
This bug caused our Tests to fail, which were relying on Browser Interaction with Selenium. Therefore i wanted to simply exclude those tests for now until the bug might get fixed. As they should not run at all, i used the following:

instrumentSettings

testOptions in ScoverageTest := Seq(Tests.Filter(s => !s.endsWith("MediumSpec")))

The affected Tests end with the suffix MediumSpec. The tests are indeed not run when we call scoverage:test, but in the end the HTML Report does not get generated. I can see the folder scoverage-data in my target directory. But the report folder is missing. If i remove this line, everything works as expected.

add class level threshold checks

So now that #11 is done, it would be great to have a threshold that is applied on a per class basis. As it stands, on a large codebase one could have a new class with no tests and everything could be hunky-dory.

Create a cross build for both 2.11 and 2.10

It would be nice if the plugin would work with both scala versions.

`coverageExcludedFiles` does not seem to have the expected behavior

Please feel free to close this if it's me missing the point, but when using coverageExcludedFiles I would expect the files matching the RegEx not to be include in the coverage report.

However, they seem to still be included in the coverage calculation and overall main summary page, but then, when clicking on the link they give essentially a 404.

Again, not sure that I'm fully understanding what the "excluded files" behavior is supposed to be, but, as a user, I'd say that the current behavior is, at least, surprising.

This a screenshot of the summary page for the controllers.ref package:

if then one clicks on any of the .scala classes on the right (eg, the ReverseApiController): this is what comes out:

java.lang.IllegalArgumentException: Cannot add dependency

I encounter an exception while adding instrumentSettings on a submodule (and it works for other submodule).

Here is the exception
java.lang.IllegalArgumentException: Cannot add dependency 'org.scoverage#scalac-scoverage-plugin_2.10;0.99.5' to configuration 'scoverage' of module XXX because this configuration doesn't exist!

`sbt scoverage:test` no longer runs the unit tests

These lines mean that the default action for sbt scoverage:test is now to run the integration tests, rather than the more conventional unit tests:

        // copy the test task into compile so we can do scoverage:test instead of scoverage-test:test etc
        test in ScoverageCompile <<= (test in ScoverageTest),
        test in ScoverageCompile <<= (test in ScoverageITest)

It should be more like:

        // copy the test task into compile so we can do scoverage:test instead of scoverage-test:test etc
        test in ScoverageCompile <<= (test in ScoverageTest)

(I can't seem to submit a PR -- github thinks my repos isn't a fork of this. Strange.)

compile failure ... can't isolate it

With 0.99.8 we get this compile failure:

https://travis-ci.org/ensime/ensime-server/builds/35143601

(the code is fine). Works with 0.99.7.1.

😕

Playframework: scoverage seems to break the AssetsController

We are using scoverage in our Play based project and are really happy with the plugin so far. Recently we started writing functional tests with selenium. The test work fine if we call sbt's normal test task. However if we call scoverage:test our selenium tests start failing because the javascript files delivered by the AssetsController start to return 404s. The AssetsController provided by Play gets the assets via a classloader. I have seen that scoverage seems to instrument some classpath settings. Could this be the cause of the issue? I am really a beginner with sbt and don't know, if this is maybe a really simple issue. It would be absolutely great if you could help us out on this one :)

sbt-scoverage 0.99.2.1 hangs build with Scala 2.11.0

My project was building fine with Scala 2.11.0 but I upgraded to new scoverage (commit: pathikrit/scalgos@47d16d9) and now the build is hanging (link to full build-log)

Add support for auto plugins in sbt 0.13.5

Add support for auto plugins in sbt 0.13.5

Add Apache 2 license file

I would like to contribute to this project, but until there is a license file I am not sure I can.

scoverage 0.99.5 published to maven repository

In Readme.MD, the instruction to add scoverage to sbt uses version 0.99.4.1

addSbtPlugin("org.scoverage" %% "sbt-scoverage" % "0.99.4.1")

However, the scoverage release page shows 0.99.5 released a couple of hours after 0.99.4.1

The central Maven repository also shows 0.99.5 available

Adding scoverage to sbt

Based on the maven repository search result above, will this line work?

addSbtPlugin("org.scoverage" %% "sbt-scoverage" % "0.99.4.1")

Seems the published versions are tied to scala version and uses the name scalac-scoverage-plugin.

Searching at mvnrepository shows even older version of scoverage published.

Is the latest version of scoverage published to Maven and how can I include it in an sbt project?

"excludedPackages" not work

Source code: https://github.com/jilen/srethink/tree/ignore_generated_class
Travis CI build: https://travis-ci.org/jilen/srethink/builds/30035348
Coveralls job: https://coveralls.io/jobs/2284078
I have ignored some package
build.sbt

ScoverageKeys.excludedPackages in ScoverageCompile := "srethink\\.protocol\\..*"

But the coveralls report still contains classess in that package

Min code coverage

Feature request for min coverage was added in 0.99.2.2
See updated readme for details.

test options

It's possible to run a subset of tests in sbt, e.g. with scalatest

test-only -- -l SlowTest

but I don't see how to pass this to scoverage:test.

Can you please add support for passing options to the test runner?

ScoverageKeys.excludedPackages not working

Hi I have following settings override:
.settings(ScoverageKeys.excludedPackages := ";./twirl/.;.*template.scala;")

Basically I want to ignore all the files under package twirl and all the classes which ends with
template.scala. However I am getting errors below. It is obviously looking at the file which ends with template.scala. Why isn't excludePagages not working?

[scoverage]: Instrumentation completed [50838 statements]
[error]
[error] while compiling: /Users/ph/target/scala-2.10/twirl/main/views/html/testsAndPocs/dashboard.template.scala
[error] during phase: scoverage-instrumentation
[error] library version: version 2.10.4

Error in Checking Min Coverage

There was an error in checking code coverage:

// check for default minimum
if (min > 0) {
if (min < coverage.statementCoverage) {
s.log.error(s"[scoverage] Coverage is below minimum [${coverage.statementCoverageFormatted}% < $min%]")
if (failOnMin)
throw new RuntimeException("Coverage minimum was not reached")
} else {
s.log.info(s"[scoverage] Coverage is above minimum [${coverage.statementCoverageFormatted}% > $min%]")
}
}

The statement of if(min < coverage.statementCoverage) should be if(min > coverage.statementCoverage).

Docs still say that highlighting is broken.

It looks like scala/scala#3799 has been resolved. Is it safe to enable highlighting?

`sbt scoverage:test` no longer instruments the compiled code

A recent change has broken sbt scoverage:test so that it no longer calls scoverage:compile, which means it doesn't really work at all.

$ sbt clean scoverage:test
Loading /cygdrive/c/Program Files (x86)/sbt/bin/sbt-launch-lib.bash
[info] Loading project definition from E:\Work\scoverage-samples\project
[info] Set current project to scoverage-samples (in build file:/E:/Work/scoverage-samples/)
[success] Total time: 0 s, completed 24-Jul-2014 19:13:54
[info] [scoverage] Waiting for measurement data to sync...
[info] Updating {file:/E:/Work/scoverage-samples/}scoverage-samples...
[info] Resolving org.scala-lang#scala-library;2.11.1 ...
[info] Done updating.
[info] Compiling 17 Scala sources to E:\Work\scoverage-samples\target\scala-2.11\classes...
[info] [scoverage] Reading scoverage instrumentation [E:\Work\scoverage-samples\target\scala-2.11\scoverage-data\scoverage.coverage.xml]
[info] [scoverage] Reading scoverage measurements...
[warn] E:\Work\scoverage-samples\src\main\scala\com\sksamuel\scoverage\samples\Debugger.scala:9: macro defs must have explicitly specified return types (inference of Unit from macro impl's c.Expr[Unit] is deprecated and is going to stop working in 2.12)
[warn]   def debug(params: Any*) = macro debugImpl
[warn]       ^
[warn] E:\Work\scoverage-samples\src\main\scala\com\sksamuel\scoverage\samples\Debugger.scala:14: type Context in package macros is deprecated: Use blackbox.Context or whitebox.Context instead
[warn]   def debugImpl(c: Context)(params: c.Expr[Any]*) = {
[warn]                    ^
[warn] there were 5 feature warnings; re-run with -feature for details
[warn] three warnings found
[info] Compiling 10 Scala sources to E:\Work\scoverage-samples\target\scala-2.11\test-classes...
[warn] there were 5 feature warnings; re-run with -feature for details
[warn] one warning found
3
4
[info] SimpleObject2Test:

Notice that there's no "[scoverage]: Begin instrumentation phase" message.
Compare that output to sbt scoverage:compile

I think the bug is that these lines:

        test in ScoverageTest := {
          (test in Test).value
          scoverageReport.value
        },

cause test in ScoverageTest to rely on "test in Test", which relies on compile in Test instead of the more correct compile in ScoverageTest.

The fix should be to revert it to something more like:

        test in ScoverageTest := {
          (test in ScoverageTest).value
          scoverageReport.value
        },

Overlapping Trees Issue

Looks like an issue in the compiler.
https://issues.scala-lang.org/browse/SI-8596

I've added a feature to turn off range highlighting. Just do
ScoverageKeys.highlighting := false
in version 0.99.4

You won't get pretty statement level highlighting but it should work.

travis and classes.bak

I see this a lot

[error] (scoverage:compile) Could not create directory /home/travis/build/ensime/ensime-server/target/scala-2.11/classes.bak

But I'm also creating a lot of directories in my CI, so I know that travis doesn't just barf when directories are created.

Does the SBT file API perhaps throw if the dir already exists? This might just need a little bit of error handling to avoid this common race / io.

Why can logs not be covered as a part of tests in code coverage report?

I have play.api.Logger in my scala files. When I run sbt clean scoverage:test, logs don't get covered.

Scoverage report shows them as red. When I comment out these logs, my code coverage is increased.

Is there any way to cover these logs in code coverage report without commenting them?

scala 2.9.3 with sbt 0.13.5 support

I'd love to use scoverage in the ensime build, but I can't use your latest releases for our scala 2.9.3 build because you haven't shipped any binaries :-(

What is the most stable version that I can use?

Scoverage crashes for code that contains "\uFFFF" (or Char.MaxValue)

If the code that we want to test contains the character Char.MaxValue the generation of the code always crashes due to a corrupt XML (invalid character).

1.0.0 breaking change to sbt coveralls plugin

Changing the command from scoverage:test to coverage test is a breaking change and now prevents the sbt coveralls command to run:

[error] No such setting/task
[error] scoverage:test
[error]               ^

I have also filed an issue with the coveralls plugin - but not sure where the fix must go (intuitively in the coveralls one, I'd guess, but that one seems less actively developed: @sksamuel change of adding the coverallsFailBuildOnError key, AFAICT, has still not made it to a new release).

I am currently unable to find where the sbt task to run is set in the coveralls plugin.

[error] bad option: '-P:scoverage:excludedPackages:'

I'm switching from play 2.2.4 to play 2.3.5.

my build .sbt contains

// -- Code Coverage

instrumentSettings

publishArtifact in Test := false

parallelExecution in Test := false

//ScoverageKeys.excludedPackages in ScoverageCompile := "<empty>;views.html.*;"

I'm launching the tests using ./activator clean scoverage:test

The only error I get:

[error] bad option: '-P:scoverage:excludedPackages:'

This command:

./activator clean test 

works fine

scala-2.9 branch

we're tied to scala 2.9 at my work. It would be fantastic if you could create a scala-2.9 branch with at least some basic functionality.

`sbt scoverage:test` attempts to compile the coverage report before the tests are run

These lines mean that the coverage report code is run in parallel with the unit test code, rather than after it:

        test in ScoverageTest := {
          (test in Test).value
          scoverageReport.value
        },

(The effect is somewhat masked by the "sleep(3000)" at the start of "scoverageReport")

There are ways to get sequential execution in SBT, but the above requests parallel execution.

Sorry I nuked all the issues :(

scalac-scoverage-plugin is automatically appended to our artifact since 0.99.9

Skinny Framework is very happy with scoverage. However, unfortunately recent updates gave us the following issue.

Skinny 1.3.2 was built with sbt-scoverage 0.99.7.1. Skinny 1.3.3 was built with 0.99.9.

Since sbt-scoverage 0.99.9, it automatically appends scalac-scoverage-plugin to our artifact. I had a same issue with scct, so I forked scct to work around like this.

This hehavior can be a pitfall for OSS developers. For now, we go back to 0.99.7. Would you please consider changing the behavior?

1.3.2 artifact with scoverage 0.99.7

https://oss.sonatype.org/content/repositories/releases/org/skinny-framework/skinny-framework_2.11/1.3.2/skinny-framework_2.11-1.3.2.pom

1.3.3 artifact with scoverage 0.99.9

https://oss.sonatype.org/content/repositories/releases/org/skinny-framework/skinny-framework_2.11/1.3.3/skinny-framework_2.11-1.3.3.pom

diff

$ diff skinny-framework_2.11-1.3.2.pom skinny-framework_2.11-1.3.3.pom

8c8
<     <version>1.3.2</version>

---
>     <version>1.3.3</version>
71c71
<             <version>1.3.2</version>

---
>             <version>1.3.3</version>
76c76
<             <version>1.3.2</version>

---
>             <version>1.3.3</version>
81c81
<             <version>1.3.2</version>

---
>             <version>1.3.3</version>
86c86
<             <version>1.3.2</version>

---
>             <version>1.3.3</version>
91c91
<             <version>1.3.2</version>

---
>             <version>1.3.3</version>
96c96
<             <version>1.3.2</version>

---
>             <version>1.3.3</version>
198a199,203
>         <dependency>
>             <groupId>org.scoverage</groupId>
>             <artifactId>scalac-scoverage-plugin_2.11</artifactId>
>             <version>0.99.9</version>
>         </dependency>

export "org.scoverage:scalac-scoverage-plugin" as dependency when publishing lib

Hi,

I have a library that I publish with sbt.

When I add 'scoverage.ScoverageSbtPlugin.instrumentSettings' to the settings, 'org.scoverage:scalac-scoverage-plugin' is after that a dependency on the published library.

sbt dependencyTree delivers:

[info]   +-mylib:mylib_2.11:0.0.1-SNAPSHOT [S]
[info]   | +-org.scoverage:scalac-scoverage-plugin_2.11:0.99.9 [S]
[info]   |   +-commons-io:commons-io:2.4
[info]   |   +-org.scala-lang.modules:scala-xml_2.11:1.0.1 [S]
[info]   |   | +-org.scala-lang:scala-library:2.11.0 (evicted by: 2.11.2)
[info]   |   | +-org.scala-lang:scala-library:2.11.1 (evicted by: 2.11.2)
[info]   |   | 
[info]   |   +-org.scala-lang:scala-library:2.11.1 (evicted by: 2.11.2)

Getting java.lang.RuntimeException: Duplicate mappings for javascript files

I am attempting to use sbt-scoverage for the application written in Play! Framework.
Problem is that I get error as following. My guess is that its trying to transfer service-main.js from somewhere in the target under scoverage-classes. I do not intend to run scoverage against javascript so I included following in ScoverageKeys.excludedPackages in ScoverageCompile := ".views.;.javascripts.;.assets.;.js.;.resource_managed." however I am getting exactly the same error. How do I instruct scoverage to ignore anything to do with javascript files?

[scoverage]: Instrumentation completed [35097 statements]
[scoverage]: Written instrumentation file [/Users/project/target/scala-2.10/scoverage-data/scoverage.coverage.xml]
[scoverage]: Writing measurements data to [/Users/project/target/scala-2.10/scoverage-data]
[warn] /Users/project/app/utils/conf/JsonConfiguration.scala:14: match may not be exhaustive.
[warn] It would fail on the following input: JsUndefined()
[warn] def deserialize(json: JsValue): Any = json match {
[warn] ^
[warn] 35 warnings found
[warn] one warning found
[info] Compiling 49 Scala sources to /Users/project/target/scala-2.10/test-classes...
java.lang.RuntimeException: Duplicate mappings:
/Users/project/target/scala-2.10/scoverage-classes/services-main.js
from
/Users/project/target/scala-2.10/resource_managed/main/public/javascripts/global/services/services-main.js
/Users/project/target/scala-2.10/resource_managed/main/public/javascripts/kiosk/global/services/services-main.js
/Users/project/target/scala-2.10/scoverage-classes/global.min.css
from......

coverageAggregate returns different results each time rerun

Just upgraded to 1.0.0 and am very excited about the improvements, especially coverageAggregate.
Currently when I run: sbt coverageAggregate it correctly finds all of my subproject's scoverage.xml files, but then doesn't seem to correctly combine them. One a small number of my classes show up in the aggregate file. (One subfile is 1900 lines, the final aggregate is only 92 lines).
Perhaps an unrelated bug, but the links within the aggregate HTML also don't point to the correct place for me.
Let me know what if any debug information that would be helpful.

Any chance at a way to combine sub-project reports into one?

I've just started using the sbt-scoverage plugin and my first impression is that it is much better than jacoco or scct. Great work!

My current project has multiple sub-projects and I was wondering whether you see a way to combine the outputs of all the individual tests into one big report across all projects. Do you think this is possible?

I'm more than willing to help add this feature. At this point I'm just not sure how hard/easy it would be and where to start.

Again, great work!
Age

Wierd compilation error when using 0.99.8

Something weird is going on with 0.99.8

When I a clean scovarge:test I get a compilation error:

> scoverage:test
[info] Updating {file:/workspace/ensime-src/}ensime-src...
[info] Formatting 14 Scala sources {file:/workspace/ensime-src/}ensime-src(test) ...
[info] Resolving org.scala-lang.modules#scala-xml_2.11;1.0.1 ...
[info] Done updating.
[info] Formatting 42 Scala sources {file:/workspace/ensime-src/}ensime-src(compile) ...
[info] [scapegoat] setting output dir to [/workspace/ensime-src/target/scala-2.11/scapegoat-report]
[info] Compiling 42 Scala sources and 1 Java source to /workspace/ensime-src/target/scala-2.11/classes...
[info] [scapegoat] 93 activated inspections
[info] [scapegoat] Beginning anaylsis...
[warn] [scapegoat] Anaylsis complete - 41 errors 160 warns 59 infos
[info] [scapegoat] Written HTML report [/workspace/ensime-src/target/scala-2.11/scapegoat-report/scapegoat.html]
[info] [scapegoat] Written XML report [/workspace/ensime-src/target/scala-2.11/scapegoat-report/scapegoat.xml]
[info] Compiling 14 Scala sources to /workspace/ensime-src/target/scala-2.11/test-classes...
[info] Compiling 14 Scala sources to /workspace/ensime-src/target/scala-2.11/scoverage-test-classes...
[error] /workspace/ensime-src/src/test/scala/org/ensime/test/ClassFileIndexSpec.scala:3: object config is not a member of package org.ensime
[error] import org.ensime.config.ProjectConfig
[error]                   ^
[error] 344 errors found
[error] (scoverage-test:compile) Compilation failed
[error] Total time: 34 s, completed 18-Aug-2014 21:21:49
Does not occur with 0.99.7.1 (scoverage version is the only change).
You can see it by trying my branch (and updating the scoverage version.

https://github.com/rorygraves/ensime-src/tree/swank_refactor2

javaOptions in Test not working, JVM not forked

This issue: scoverage/scalac-scoverage-plugin#15) was closed with a merge that added javaOptions from Test/Compile into the corresponding Scoverage configurations. This unfortunately does not fix the problem, which I've just confirmed persists as a java option that is necessary for some of my tests to run doesn't get applied when I run scoverage:test. As discussed in the sbt guide javaOptions only gets applied when you run the test under a forked jvm : http://www.scala-sbt.org/0.13.2/docs/Detailed-Topics/Forking.html

Currently it doesn't seem possible to fork the Scoverage test. Looking at the code here:

sbt-scoverage/src/main/scala/scoverage/ScoverageSbtPlugin.scala

Line 25 in 63b0fdf

val ScoverageTest: Configuration = config("scoverage-test") extend ScoverageCompile

it seems ScoverageTest extends from ScoverageCompile, which itself extends the sbt Compile configuration and doesn't have a fork option. Shouldn't this be extending the Test configuration instead? I'm guessing once we can fork the ScoverageTest the javaOptions will take effect.

Straw poll on sbt plugin usage

Do you run a seperate job for coverage? Like a coverage build in teamcity or jenkins or whatever. Or do you have just a single job that you use for both test coverage as well as building artifacts (that last bit is what I'm most interested in).

@D-Roch @rorygraves @stacycurl @RadoBuransky @fommil

option not to output HTML reports

In our CI, we're only interested in the minimum coverage, so we'd be happy to save CPU cycles. Also, the HTML outputting causes stability issues on travis, so we'd be keen to bypass the problem by avoiding the need to create these files.

@stacycurl also sees this.

Publish for Scala 2.11

Granted I am a bit mixed up and might be missing something.

Where is version 0.99.7.1 that is mentioned on the home page hosted?

Is it compiled both for Scala 2.10 and 2.11?

Broken links in coverage report if directory structure doesn't match package structure

The directory structure of the highlighted-source HTML files follows the files' declared package, but the URLs in the coverage report page follow the original source files' directory structure. For example, if I have a file at src/main/foo.scala with a package com.example.foo declaration, the resulting highlighted-source file will be generated at target/scoverage-report/com/example/foo.scala.html. However, the HTML report at target/scoverage-report/index.html will contain a broken link pointing at target/scoverage-report/foo.scala.

Beta 3 No Errors But No Report Either

plugins.sbt
addSbtPlugin("org.scoverage" %% "sbt-scoverage" % "1.0.0.BETA3")
In Build .sbt
scalaVersion := "2.11.3"
sbt.version=0.13.5
No coveragekeys specified as sbt fails if any key specified
sbt clean coverage test succeeds
No scoverage-report generated in target or in /target/scala-2.11
Any Ideas?

Replacement for ScoverageCompile

In the latest versions (0.99.9), ScoverageCompile no longer exists.
What should we used instead ?

the minimal coverage is not correctly checked

I tested the minimal coverage option, and this is the result:

[info] [scoverage] Waiting for measurement data to sync...
[info] [scoverage] Reading scoverage instrumentation [/home/simon/ePost/project/apps/MUI/app/target/scala-2.10/scoverage-data/scoverage.coverage.xml]
[info] [scoverage] Reading scoverage measurements...
[info] [scoverage] Generating Cobertura report [/home/simon/ePost/project/apps/MUI/app/target/scala-2.10/coverage-report/cobertura.xml]
[info] [scoverage] Generating XML report [/home/simon/ePost/project/apps/MUI/app/target/scala-2.10/scoverage-report/scoverage.xml]
[info] [scoverage] Generating XML report [/home/simon/ePost/project/apps/MUI/app/target/scala-2.10/scoverage-report/index.html]
[info] [scoverage] Reports completed
[error] [scoverage] Coverage is below minimum [78,40% < 80%]
java.lang.RuntimeException: Coverage minimum was not reached
    at scoverage.ScoverageSbtPlugin$$anonfun$testsCleanup$1$$anonfun$apply$2.apply$mcV$sp(ScoverageSbtPlugin.scala:150)
    at sbt.Tests$$anonfun$Cleanup$1.apply(Tests.scala:53)
    at sbt.Tests$$anonfun$Cleanup$1.apply(Tests.scala:53)
    at sbt.Tests$$anonfun$sbt$Tests$$partApp$1$1$$anonfun$apply$1.apply$mcV$sp(Tests.scala:160)
    at sbt.Tests$$anonfun$sbt$Tests$$fj$1$1.apply(Tests.scala:159)
    at sbt.Tests$$anonfun$sbt$Tests$$fj$1$1.apply(Tests.scala:159)
    at sbt.std.TaskExtra$$anon$1$$anonfun$fork$1$$anonfun$apply$1.apply(TaskExtra.scala:99)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:45)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:45)
    at sbt.std.Transform$$anon$4.work(System.scala:64)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:244)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
[error] (scoverage-test:executeTests) Coverage minimum was not reached

support for sbt's Integration phase

We'd really love to see unit test coverage and integration test coverage separately. Can you please let me know how to do this, or add this as a feature?

Great tool! I love being able to specify a minimum coverage level in our CI so builds fail when PRs don't have tests :-)

Unable to run `scoverage:test` with Scala 2.11.4 and sbt-scoverage 0.99.9 (or 0.99.7.1) in multi module project

I have a multi module project with a single project that defines custom package exclusions

ScoverageKeys.excludedPackages in Scoverage := """<empty>;.*Spec;.*\.steps\..*"""

and some common scoverage settings

val scoverageSettings = instrumentSettings ++ Seq(
  ScoverageKeys.highlighting := true,
  parallelExecution in ScoverageTest := false,
  testOptions in ScoverageTest += Tests.Argument(TestFrameworks.ScalaTest, "-l", "SlowTest"))

When using Scala 2.11.2 the whole project build fine and the scoverage:test task would run ad generate the desired reports etc.

Changing the Scala version to 2.11.4, without any other changes, caused the projects without the custom exclude rule to fail with the error

[error] bad option: '-P:scoverage:excludedPackages:'

Trying to set an exclude rule in the common scoverage settings of

ScoverageKeys.excludedPackages in Scoverage := "<empty>"

removing all exclusions or changing the scope to ScoverageTest does not fix this but reverting back to Scala 2.11.2 with the same configuration does.

The same behaviour happens with 0.99.7.1 and 0.99.9.

Macros are triggering compile errors

When I compile my project with scoverage I get the following error:

[error] project/core/src/main/scala/org/specs2/specification/create/AutoExamples.scala:15: erroneous or inaccessible type
[error]   implicit def eg[T : AsResult](code: =>T): Fragments = macro AutoExamples.create[T]
[error]                                                                                  ^

Ambiguous measurement from branch coverage

I have noticed that all the tests with no control-flow (such as if, else, switch, etc) shows 100% branch coverage due to no branch present in the code block. For example
def foo = {
println("Hello World!")
}
Shows 100% branch coverage and 0 % statement coverage. This is due to 0/0 branch covered. From my eyes, I see one branch in above test case, so shouldn't there be one branch even if there is no control flow?

Thanks

scala.reflect.api

I have a test that passed with test but failed with scoverage:test

It seems there is an issue with scala.reflect.api

Here is the stack
[info] - serialize/deserialize template metadata *** FAILED ***
[info] java.lang.AssertionError: assertion failed: List(object JSONUtils, object JSONUtils)
[info] at scala.reflect.internal.Symbols$Symbol.suchThat(Symbols.scala:1678)
[info] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:44)
[info] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
[info] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
[info] at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:72)
[info] at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:119)
[info] at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:21)
[info] at xxx.JSONUtils$SimpleProductReflection$$typecreator1$1.apply(JSONUtils.scala:78)
[info] at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:231)
[info] at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:231)