PitHelpError : tests did not pass without mutation when calculating the line coverage, Mutation testing requires a green suite. about pitest

[hcoles]

If you put together a minimal executable project that reproduces the issue, I can take a look.

[AjithGowda]

If you put together a minimal executable project that reproduces the issue, I can take a look.

Hi @hcoles

Here's is the project : https://github.com/AjithGowda/pitSample

Branch : master

steps :

1. Clean build the project
2. Run the test from gradle task ( Executes successfully )
3. Run pitest task ( failed to execute)

Looking forward for your response ,

Thank you

[hcoles]

Thanks for the minimal project.

The problem is that this project is using the module path, while pitest is still using the old style classpath. This is a known issue which need resolving.

If you update the project to use the latest version of pitest and the junit plugin, the problem becomes clearly visible in the log

pitest{
    pitestVersion = '1.15.3'
    targetClasses=["com.*"]
    junit5PluginVersion = '1.2.1'
    verbose=true
}

13:59:45 PIT >> INFO : MINION :         at com.org.sample.Renderer.<init>(Renderer.java:10)
13:59:45 PIT >> INFO : MINION :         at com.org.sample.RendererTest.CustomizeCellRenderer(RendererTest.groovy:19)
13:59:45 PIT >> INFO : MINION : Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make static java.awt.Graphics javax.swing.JComponent.safelyGetGraphics(java.awt.Component) accessible: module java.desktop does not "opens javax.swing" to unnamed module @15994b0b

You can work round it for now by opening any necessary modules in the pitest config.

pitest{
    pitestVersion = '1.15.3'
    targetClasses=["com.*"]
    junit5PluginVersion = '1.2.1'
    verbose=true
    jvmArgs.add("--add-opens=java.base/java.lang=ALL-UNNAMED")
    jvmArgs.add("--add-opens=java.desktop/javax.swing=ALL-UNNAMED")
}

[AjithGowda]

Thanks for the minimal project.

The problem is that this project is using the module path, while pitest is still using the old style classpath. This is a known issue which need resolving.

If you update the project to use the latest version of pitest and the junit plugin, the problem becomes clearly visible in the log

pitest{
    pitestVersion = '1.15.3'
    targetClasses=["com.*"]
    junit5PluginVersion = '1.2.1'
    verbose=true
}

13:59:45 PIT >> INFO : MINION :         at com.org.sample.Renderer.<init>(Renderer.java:10)
13:59:45 PIT >> INFO : MINION :         at com.org.sample.RendererTest.CustomizeCellRenderer(RendererTest.groovy:19)
13:59:45 PIT >> INFO : MINION : Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make static java.awt.Graphics javax.swing.JComponent.safelyGetGraphics(java.awt.Component) accessible: module java.desktop does not "opens javax.swing" to unnamed module @15994b0b

You can work round it for now by opening any necessary modules in the pitest config.

pitest{
    pitestVersion = '1.15.3'
    targetClasses=["com.*"]
    junit5PluginVersion = '1.2.1'
    verbose=true
    jvmArgs.add("--add-opens=java.base/java.lang=ALL-UNNAMED")
    jvmArgs.add("--add-opens=java.desktop/javax.swing=ALL-UNNAMED")
}

@hcoles

Thanks for the prompt response.

Is there a workaround to fix this issue via command line as well ?

[hcoles]

@AjithGowda I'm not sure what you mean. If you update the build file as suggested, it will work when running gradle from the commandline.

[AjithGowda]

@AjithGowda I'm not sure what you mean. If you update the build file as suggested, it will work when running gradle from the commandline.

Sorry if i had not conveyed it right , What i mean is

Can we add the jvm params ( jvmArgs.add("--add-opens=java.base/java.lang=ALL-UNNAMED"), when we run using org.pitest.mutationtest.commandline.MutationCoverageReport ?

If yes , could you help us with that ?

[hcoles]

If you are doing something custom calling MutationCoverageReport programatically, I'm afraid I'm not able to offer free consultancy. Paid consultancy is available via arcmuate.com

[AjithGowda]

If you are doing something custom calling MutationCoverageReport programatically, I'm afraid I'm not able to offer free consultancy. Paid consultancy is available via arcmuate.com

@hcoles Thanks for the input will try it out via configuration ,Have one more question.

I see there are difference in report when we run pitest using old and new pitest versions. when we run pitest with old pit version ( 1.11.6) with junit plugin (1.1.2) then most of the mutants are reported under killed or survived and only less under no coverage. where as if i run the same code and test with pitest version (1.15.3) and junit plugin (1.2.1), then most of the mutants are under no coverage section. May i know why there is difference ?

[hcoles]

Hi @AjithGowda

The sample project has only one mutant marked as no coverage, this is in an else branch that is never executed by a test.

If you are seeing mutants marked as no coverage which you believe are covered by a test, I can take a look if you upload a minimal executable project that reproduces the issue.

[AjithGowda]

Hi @AjithGowda

The sample project has only one mutant marked as no coverage, this is in an else branch that is never executed by a test.

If you are seeing mutants marked as no coverage which you believe are covered by a test, I can take a look if you upload a minimal executable project that reproduces the issue.

Hi @hcoles ,

will work on the minimal executable project.

Also the workaround you suggested above --jvmArgs.add("--add-opens=java.base/java.lang=ALL-UNNAMED") --jvmArgs.add("--add-opens=java.desktop/javax.swing=ALL-UNNAMED") , is it specific to the project we sent or this will fix for all the issues of this type error ?

[AjithGowda]

Hi @AjithGowda
The sample project has only one mutant marked as no coverage, this is in an else branch that is never executed by a test.
If you are seeing mutants marked as no coverage which you believe are covered by a test, I can take a look if you upload a minimal executable project that reproduces the issue.

Hi @hcoles ,

will work on the minimal executable project.

Also the workaround you suggested above --jvmArgs.add("--add-opens=java.base/java.lang=ALL-UNNAMED") --jvmArgs.add("--add-opens=java.desktop/javax.swing=ALL-UNNAMED") , is it specific to the project we sent or this will fix for all the issues of this type error ?

@hcoles Could you help us with the above query please ?

[hcoles]

Can you provide a little more detail about your project? It appears to be a mutation testing plugin for Intelij. Is it open source?

[AjithGowda]

Can you provide a little more detail about your project? It appears to be a mutation testing plugin for Intelij. Is it open source?

Hi @hcoles

Its not open source . Its a light project building for small set of people to run via command line.

We would like to know , the workaround you suggested to add the jvmargs is it specific to the minimal executable project i sent to you or will it fix all similar kind of issues.

Also could this be resolved in latest releases to piTest ?

[hcoles]

You can read about add-opens and the module path in the oracle java 9 migration guide

https://docs.oracle.com/en/java/javase/17/migrate/migrating-jdk-8-later-jdk-releases.html

As for whether this will be fixed for you in a future release of pitest. If you are calling MutationCoverReport programatically it's not possible to say as it will depend on what your code does. Future releases of pitest will however look at module path support when run via maven or gradle.