jqwik-team / jqwik Goto Github PK

View Code? Open in Web Editor NEW

550.0 13.0 63.0 24.37 MB

Property-Based Testing on the JUnit Platform

Home Page: http://jqwik.net

License: Eclipse Public License 2.0

Java 96.43% Kotlin 3.57%

quickcheck junit junit5 hacktoberfest

jqwik's People

Contributors

Stargazers

Watchers

jqwik's Issues

Do not depend on junit-platform-engine for the API

jqwik comes with a dependency on junit-platform-engine:

compile("org.junit.platform:junit-platform-engine:${junitPlatformVersion}")

A little surprsingly, this may cause trouble. I have a Maven project that also uses JUnit Jupiter tests and only has the dependency on junit-jupiter-engine in the Surefire configuration (Jupiter, in contrast to jqwik, comes with two dependencies, one for the API and one for the engine).

When running tests through IntelliJ in this project with 3 engines, often IntelliJ cannot find the tests. The details are not too important (issue raised with JetBrains), but the solution is to exclude the dependency on the platform engine in the POM.

My request here is to split jqwik into API and engine as Jupiter does. Alternatively, exluding junit-platform-engine from the exported dependencies may help, and a note in the docs to include it for the few users that use jqwik exclusively and thus do not get the dependency in any other way.

Build fails on JDK 10

Testing Problem

Mockito needs an update
Test classes with private constructors are not (no longer?) supported?!
Gradle 4.7-rc-1 supports "JDK-11-ea" partly...

Log snippets:

[...]

  JQwik Test Engine:GenericPropertyTests:collectStatistics
    MethodSource [className = 'net.jqwik.properties.GenericPropertyTests', methodName = 'collectStatistics', methodParameterTypes = '']
    => org.mockito.exceptions.base.MockitoException:
Mockito cannot mock this class: interface java.util.function.Consumer.

Mockito can only mock non-private & non-final classes.
If you're not sure why you're getting this error, please report to the mailing list.

Java               : 10
JVM vendor name    : "Oracle Corporation"
JVM vendor version : 10+46
JVM name           : Java HotSpot(TM) 64-Bit Server VM
JVM version        : 10+46
JVM info           : mixed mode
OS name            : Windows 10
OS version         : 10.0

Underlying exception : java.lang.UnsupportedOperationException: Cannot define class using reflection

[...]

  JQwik Test Engine:TestEngineIntegrationTests:runTestsFromMethod
    MethodSource [className = 'net.jqwik.TestEngineIntegrationTests', methodName = 'runTestsFromMethod', methodParameterTypes = '']
    => net.jqwik.JqwikException: Cannot create instance of class 'class net.jqwik.TestEngineIntegrationTests'. Maybe it has no default constructor?


[...]

Test run finished after 12476 ms
[        83 containers found      ]
[         0 containers skipped    ]
[        83 containers started    ]
[         0 containers aborted    ]
[        83 containers successful ]
[         0 containers failed     ]
[       395 tests found           ]
[         0 tests skipped         ]
[       395 tests started         ]
[         0 tests aborted         ]
[       365 tests successful      ]
[        30 tests failed          ]

Support pattern(regex) for StringArbitrary

Testing Problem

I want to generate a string of sophisticated patterns.
It would be nice to support string patterning like REGEX.

Discussion

jqwik with JMockit

Hi,

I tried to use jqwik with JMockit. In this way:

import static org.junit.jupiter.api.Assertions.assertNotNull;
import org.junit.jupiter.api.Test;
import mockit.Mocked;
import net.jqwik.api.Example;

public class JqikWithJMockit {

@Mocked
public Object object;

@Test
void test() {
	assertNotNull(object);	
}

@Example
boolean whenObjectIsMockedThenItShouldBeNotNull() {
	return object != null;
}

The object is mocked for the JUnit5 test but null for jqwik test. Any idea? Is it possible to use any mock frameworks with jqwik?

Thanks
Sash

Arbitraries containing null don't work correctly when used exhaustively

We want to exhaustively test parameters that can be either null or zero. This seems to be not possible at the moment. In the cases where jqwik automatically selects exhaustive generation (nullOrZero 1, 2 and 4), only null or zero is ever generated, depending on how the arbitrary is created. Forcing random generation on the other hand leads to the generation of both null and zero.

public class NullOrZeroTest {

  @Property @Report(Reporting.GENERATED)
  boolean nullOrZero1(@ForAll("nullOrZero1") BigDecimal bd) {
    System.out.println("nullOrZero1 " + bd);
    return bd == null || bd.compareTo(BigDecimal.ZERO) == 0;
  }

  @Property @Report(Reporting.GENERATED)
  boolean nullOrZero2(@ForAll("nullOrZero2") BigDecimal bd) {
    System.out.println("nullOrZero2 " + bd);
    return bd == null || bd.compareTo(BigDecimal.ZERO) == 0;
  }

  @Property(generation = GenerationMode.EXHAUSTIVE) @Report(Reporting.GENERATED)
  boolean nullOrZero3(@ForAll("nullOrZero3") BigDecimal bd) {
    System.out.println("nullOrZero3 " + bd);
    return bd == null || bd.compareTo(BigDecimal.ZERO) == 0;
  }

  @Property @Report(Reporting.GENERATED)
  boolean nullOrZero4(@ForAll("nullOrZero4") BigDecimal bd) {
    System.out.println("nullOrZero4 " + bd);
    return bd == null || bd.compareTo(BigDecimal.ZERO) == 0;
  }

  @Provide Arbitrary<BigDecimal> nullOrZero1() {
    return Arbitraries.of(null, BigDecimal.ZERO);
  }

  @Provide Arbitrary<BigDecimal> nullOrZero2() {
    return Arbitraries.of(BigDecimal.ZERO, null);
  }

  @Provide Arbitrary<BigDecimal> nullOrZero3() {
    return Arbitraries.bigDecimals().between(BigDecimal.ZERO, BigDecimal.ZERO)
        .withSamples((BigDecimal) null);
  }

  @Provide Arbitrary<BigDecimal> nullOrZero4() {
    return Arbitraries.of(new BigDecimal("0"), new BigDecimal("0.0"), new BigDecimal("0.00"))
        .withSamples((BigDecimal) null);
  }
}

Test output:

Running NullOrZeroTest
timestamp = 2019-12-12T20:26:00.925608, generated = [null]

nullOrZero1 null
timestamp = 2019-12-12T20:26:00.944175, generated = [0]

nullOrZero2 0
timestamp = 2019-12-12T20:26:00.944756, generated = [null]

nullOrZero2 null
timestamp = 2019-12-12T20:26:00.951872, NullOrZeroTest:nullOrZero2 = 
                              |-------------------jqwik-------------------
tries = 2                     | # of calls to property
checks = 2                    | # of not rejected calls
generation-mode = EXHAUSTIVE  | parameters are exhaustively generated
after-failure = PREVIOUS_SEED | use the previous seed
seed = 4566159385613353648    | random seed to reproduce generated values


timestamp = 2019-12-12T20:26:00.973788, generated = [null]

nullOrZero4 null
Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.152 s <<< FAILURE! - in NullOrZeroTest
nullOrZero3  Time elapsed: 0.015 s  <<< ERROR!
net.jqwik.api.JqwikException: EXHAUSTIVE generation is not possible. Maybe too many potential examples?

╷
├─ jqwik (JUnit Platform) ✔
│  └─ NullOrZeroTest ✔
│     ├─ nullOrZero1 ✔
│     ├─ nullOrZero2 ✔
│     ├─ nullOrZero3 ✘ EXHAUSTIVE generation is not possible. Maybe too many potential examples?
│     └─ nullOrZero4 ✔
└─ JUnit Jupiter ✔

Results:

Errors: 
  NullOrZeroTest.nullOrZero3 » Jqwik EXHAUSTIVE generation is not possible. Mayb...

Tests run: 4, Failures: 0, Errors: 1, Skipped: 0

Edit: The nullOrZero2 combination actually does work, but the others not. So it seems null must not be the first value created.
Edit2: Even then we do not get the full test matrix when testing two parameters:

Running NullOrZeroTest
timestamp = 2019-12-12T20:49:32.218276, generated = [0, 0]

timestamp = 2019-12-12T20:49:32.220822, generated = [0, null]

timestamp = 2019-12-12T20:49:32.221372, generated = [null, 0]

timestamp = 2019-12-12T20:49:32.231031, NullOrZeroTest:nullOrZero2 = 
                              |-------------------jqwik-------------------
tries = 3                     | # of calls to property
checks = 3                    | # of not rejected calls
generation-mode = EXHAUSTIVE  | parameters are exhaustively generated
after-failure = PREVIOUS_SEED | use the previous seed
seed = 8110854518611637746    | random seed to reproduce generated values

More convenience for String Arbritraries

Testing Problem

I often end up writing code like this.

Arbitraries.strings().alpha().withChars(new char[]{' ', '"', ',', ';', '.', ':'});

Test Execution Problem

Under certain circumstances we get the following warning

net.jqwik.discovery.HierarchicalJavaResolver resolveClass
WARNUNG: Class 'x.y.z' could not be resolved

without understanding why we get it. All tests seemingly were executed successfully.
This happens only with a very specific configuration of maven-surefire, implicating that the root cause probably lies on their ground.
I created a demo project with which the warning can be triggered: https://github.com/tmohme/jqwik-warning
For reproduction please checkout the project and build it with 'mvn clean test "-Dtest=org.example.*.java"'.
Any change to the class-loader related configuration of the maven-surefire-plugin lets the warning disappear. Also there's no warning, when you simply execute 'mvn clean test'.
Maybe this issue should be "forwarded" to the maven-surefire-plugin.

Desired behaviour

In the best case, we shouldn't get the warning.

Discussion

If the root cause lies in wrong/weird class loading behaviour of surefire, the warning should be accompanied by a hint what the possible consequences (for the test execution) are.
Maybe documenting this restriction would help.

Support for ad hoc usage

Testing Problem

Hi, I'm wondering if there's a straightforward way to use jqwik in an ad-hoc manner. By this I mean being able to use its random data generation utilities without having to use the jqwik junit engine to run tests. The specific use case I have is running integration tests in Spring while wanting to reuse the domain-specific arbitrary providers I've written.

Suggested Solution

An API for creating a single sample of a given Arbitrary<T>, or an @ArgumentsSource for injecting values via @ParametrizedTest.

Thanks for this library, it's great!

[Support] Running jqwik with pitest

Im using pitest for mutation testing, this causes spam like this on stderr

[serra@archlinux de.serra.condorcat]$ env JAVA_HOME=/usr/lib/jvm/default/ mvn -e --no-transfer-progress -T1C clean verify site > build.log
09:04:22 PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.
09:04:22 PIT >> INFO : Sending 75 test classes to minion
09:04:22 PIT >> INFO : Sent tests to minion
09:04:23 PIT >> INFO : MINION : 09:04:23 PIT >> INFO : Checking environment

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:23 PIT >> INFO : MINION : Juli 25, 2019 9:04:23 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.
Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [j
09:04:24 PIT >> INFO : MINION : qwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:24 PIT >> INFO : MINION : 09:04:24 PIT >> INFO : Found  14 tests

09:04:24 PIT >> INFO : MINION : 09:04:24 PIT >> INFO : Dependency analysis reduced number of potential tests by 0

09:04:24 PIT >> INFO : MINION : 09:04:24 PIT >> INFO : 14 tests received

09:04:24 PIT >> INFO : MINION : Juli 25, 2019 9:04:24 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:36 PIT >> INFO : MINION : Juli 25, 2019 9:04:36 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:46 PIT >> INFO : MINION : Juli 25, 2019 9:04:46 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:46 PIT >> INFO : MINION : Juli 25, 2019 9:04:46 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itCreates()]' as test or
09:04:46 PIT >> INFO : MINION :  test container but could not fulfill it

09:04:46 PIT >> INFO : MINION : Juli 25, 2019 9:04:46 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:46 PIT >> INFO : MINION : Juli 25, 2019 9:04:46 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicateChoice
09:04:46 PIT >> INFO : MINION : sInVote()]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicatedChoic
09:04:47 PIT >> INFO : MINION : es()]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicatedVotes
09:04:47 PIT >> INFO : MINION : ()]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicates()]' 
09:04:47 PIT >> INFO : MINION : as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsInvalidBallotRe
09:04:47 PIT >> INFO : MINION : ference()]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsInvalidChoices(
09:04:47 PIT >> INFO : MINION : )]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.BallotStoreBallotsTest]/[method:testBallotStoreBal
09:04:47 PIT >> INFO : MINION : lots()]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.PersistedBallotTest]/[method:itCantBeCreatedTwice(
09:04:47 PIT >> INFO : MINION : )]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.PersistedBallotTest]/[method:itCreatesEntry()]' as
09:04:47 PIT >> INFO : MINION :  test or test container but could not fulfill it

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:04:47 PIT >> INFO : MINION : Juli 25, 2019 9:04:47 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.PersistedBallotTest]/[method:itSetsCorredHasAlread
09:04:47 PIT >> INFO : MINION : yVotedFlag()]' as test or test container but could not fulfill it

09:04:47 PIT >> INFO : Calculated coverage in 25 seconds.
09:04:48 PIT >> INFO : Created  47 mutation test units
09:05:04 PIT >> INFO : Completed in 42 seconds
[serra@archlinux de.serra.condorcat]$ env JAVA_HOME=/usr/lib/jvm/default/ mvn -e --no-transfer-progress -T1C clean verify site > build.log
09:10:02 PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.
09:10:03 PIT >> INFO : Sending 75 test classes to minion
09:10:03 PIT >> INFO : Sent tests to minion
09:10:03 PIT >> INFO : MINION : 09:10:03 PIT >> INFO : Checking environment

09:10:03 PIT >> INFO : MINION : Juli 25, 2019 9:10:03 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.
Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [j
09:10:04 PIT >> INFO : MINION : qwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:04 PIT >> INFO : MINION : 09:10:04 PIT >> INFO : Found  14 tests

09:10:04 PIT >> INFO : MINION : 09:10:04 PIT >> INFO : Dependency analysis reduced number of potential tests by 0

09:10:04 PIT >> INFO : MINION : 09:10:04 PIT >> INFO : 14 tests received

09:10:04 PIT >> INFO : MINION : Juli 25, 2019 9:10:04 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:15 PIT >> INFO : MINION : Juli 25, 2019 9:10:15 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itCreates()]' as test or
09:10:26 PIT >> INFO : MINION :  test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicateChoice
09:10:26 PIT >> INFO : MINION : sInVote()]' as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicatedChoic
09:10:26 PIT >> INFO : MINION : es()]' as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicatedVotes
09:10:26 PIT >> INFO : MINION : ()]' as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsDuplicates()]' 
09:10:26 PIT >> INFO : MINION : as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsInvalidBallotRe
09:10:26 PIT >> INFO : MINION : ference()]' as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.InMemoryBallotStoreTest]/[method:itRejectsInvalidChoices(
09:10:26 PIT >> INFO : MINION : )]' as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.BallotStoreBallotsTest]/[method:testBallotStoreBal
09:10:26 PIT >> INFO : MINION : lots()]' as test or test container but could not fulfill it

09:10:26 PIT >> INFO : MINION : Juli 25, 2019 9:10:26 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:27 PIT >> INFO : MINION : Juli 25, 2019 9:10:27 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:27 PIT >> INFO : MINION : Juli 25, 2019 9:10:27 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.PersistedBallotTest]/[method:itCantBeCreatedTwice(
09:10:27 PIT >> INFO : MINION : )]' as test or test container but could not fulfill it

09:10:27 PIT >> INFO : MINION : Juli 25, 2019 9:10:27 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:27 PIT >> INFO : MINION : Juli 25, 2019 9:10:27 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.PersistedBallotTest]/[method:itCreatesEntry()]' as
09:10:27 PIT >> INFO : MINION :  test or test container but could not fulfill it

09:10:27 PIT >> INFO : MINION : Juli 25, 2019 9:10:27 VORM. net.jqwik.engine.JqwikProperties loadProperties
INFORMATION: No Jqwik properties file [jqwik.properties] found.

09:10:27 PIT >> INFO : MINION : Juli 25, 2019 9:10:27 VORM. net.jqwik.engine.discovery.HierarchicalJavaResolver resolveUniqueId
WARNUNG: Received request to resolve unique id '[engine:junit-jupiter]/[class:de.serra.condorcat.model.ballot.PersistedBallotTest]/[method:itSetsCorredHasAlread
09:10:27 PIT >> INFO : MINION : yVotedFlag()]' as test or test container but could not fulfill it

09:10:27 PIT >> INFO : Calculated coverage in 24 seconds.
09:10:28 PIT >> INFO : Created  47 mutation test units
09:10:48 PIT >> INFO : Completed in 45 seconds

Im not sure that the problem here is. Maybe it's just a configuration issue on my side or an problem with pitest.

But maybe you can shed some light on what these messages mean.

The No Jqwik properties file [jqwik.properties] found. log message is clear, but the messages regarding resolveUniqueId seem like they might be important.

Usability with failed Tests

Problem

I like Jqwik a lot. One thing that bugs when using it, is scrolling through a lot of text before finding the assertion and thus the important information when a test fails.

even if I give the test reporter two thirds of the screen, I can not see the assertion ...

Testing Problem

Running individual jqwik test fails in IDEA with the error:

2018-04-07 22:11:41 WARNING TestEngine with ID 'jqwik' failed to discover tests
org.junit.platform.commons.JUnitException: Failed to load parameter type [[byte] for method [testFoo] in class [com.example.SomeTest].
	at org.junit.platform.commons.util.ReflectionUtils.lambda$loadRequiredParameterType$13(ReflectionUtils.java:832)
	at java.util.Optional.orElseThrow(Optional.java:290)
	at org.junit.platform.commons.util.ReflectionUtils.loadRequiredParameterType(ReflectionUtils.java:831)
	at org.junit.platform.commons.util.ReflectionUtils.lambda$resolveParameterTypes$11(ReflectionUtils.java:825)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
	at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:438)
	at org.junit.platform.commons.util.ReflectionUtils.resolveParameterTypes(ReflectionUtils.java:826)
	at org.junit.platform.commons.util.ReflectionUtils.findMethod(ReflectionUtils.java:815)
	at org.junit.platform.engine.discovery.MethodSelector.lazyLoadJavaMethod(MethodSelector.java:169)
	at org.junit.platform.engine.discovery.MethodSelector.getJavaMethod(MethodSelector.java:142)
	at net.jqwik.discovery.JqwikDiscoverer.lambda$discover$4(JqwikDiscoverer.java:49)
	at java.util.ArrayList.forEach(ArrayList.java:1249)
	at net.jqwik.discovery.JqwikDiscoverer.discover(JqwikDiscoverer.java:48)
	at net.jqwik.JqwikTestEngine.discover(JqwikTestEngine.java:33)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:130)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:117)
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:82)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:48)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

2018-04-07 22:11:41 WARNING TestEngine with ID 'jqwik' failed to discover tests
org.junit.platform.commons.JUnitException: Failed to load parameter type [[byte] for method [testFoo] in class [com.groupon.ezsc.KeyCiphertextTest].
	at org.junit.platform.commons.util.ReflectionUtils.lambda$loadRequiredParameterType$13(ReflectionUtils.java:832)
	at java.util.Optional.orElseThrow(Optional.java:290)
	at org.junit.platform.commons.util.ReflectionUtils.loadRequiredParameterType(ReflectionUtils.java:831)
	at org.junit.platform.commons.util.ReflectionUtils.lambda$resolveParameterTypes$11(ReflectionUtils.java:825)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
	at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:438)
	at org.junit.platform.commons.util.ReflectionUtils.resolveParameterTypes(ReflectionUtils.java:826)
	at org.junit.platform.commons.util.ReflectionUtils.findMethod(ReflectionUtils.java:815)
	at org.junit.platform.engine.discovery.MethodSelector.lazyLoadJavaMethod(MethodSelector.java:169)
	at org.junit.platform.engine.discovery.MethodSelector.getJavaMethod(MethodSelector.java:142)
	at net.jqwik.discovery.JqwikDiscoverer.lambda$discover$4(JqwikDiscoverer.java:49)
	at java.util.ArrayList.forEach(ArrayList.java:1249)
	at net.jqwik.discovery.JqwikDiscoverer.discover(JqwikDiscoverer.java:48)
	at net.jqwik.JqwikTestEngine.discover(JqwikTestEngine.java:33)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:130)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:117)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:65)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)


Process finished with exit code 0
Empty test suite.

This occurs for any parameterized test annotated with @Property when you attempt to run a single test. Running an entire test class works fine. Running @Example non-parameterized tests works fine.

Discussion

It is useful to be able to narrow in on a single test in a class.

Poor numeric domain coverage when using default arbitraries

Testing Problem

Poor numeric domain coverage when using default arbitraries. The property evenNumbersAreEvenAndSmall is not falsified by Jqwik 0.8.4 using the default jqwik.poperties in the tests below:

public class JqwikTests {

    public static final Condition<Long> EVEN = new Condition<>(x -> x % 2 == 0, "even");
    public static final Condition<Long> SMALL = new Condition<>(x -> x < 2000, "< 2000");

    @Provide
    Arbitrary<Long> evenNumbers() {
        return Arbitraries.longs().filter(l -> l % 2 == 0);
    }

    @Property
    void evenNumbersAreEven(@ForAll("evenNumbers") long evenNumber) {
        assertThat(evenNumber)
                .is(EVEN);
    }

    @Property
    void evenNumbersAreEvenAndSmall(@ForAll("evenNumbers") long evenNumber) {
        System.out.println(evenNumber);
        assertThat(evenNumber)
                .is(EVEN)
                .is(SMALL);
    }
}

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running JqwikTests
0
-9223372036854775808
8
-468
-214
284
186
... many more (993? :-)) numbers, but all roughly in the region (-500, 500)

The other tested frameworks (JunitQuickCheck, QuickTheories, VavrTest) are able to falsify this using default settings but have unusable or no shrinking.

Manually setting the range to (Long.MIN_VALUE + 1, Long.MAX_VALUE - 1) solves this as does increasing the number of tries.

Discussion

Documenting does not need any further discussion I guess :-)

Increasing the test domain while keeping the number of tries obviously spreads out the sample data which might be wanted--or not.

empty test produces number out of range error

Version testet: 1.1.1
Java: Oracle Java 8
Kotlin: 1.3.11

Testing Problem

following kotlin test code ...

@Property
fun `what`(@ForAll @DoubleRange(min = 0.001, max = 0.2) grow: Double) {
}

produces following error:

Number <0.00> is outside allowed range 0.00010..0.2
net.jqwik.api.JqwikException: Number <0.00> is outside allowed range 0.00010..0.2
	at net.jqwik.engine.properties.shrinking.ShrinkableBigDecimal.determineTarget(ShrinkableBigDecimal.java:49)
	at net.jqwik.engine.properties.shrinking.ShrinkableBigDecimal.<init>(ShrinkableBigDecimal.java:21)
	at net.jqwik.engine.properties.arbitraries.randomized.RandomDecimalGenerators.lambda$createBaseGenerator$2(RandomDecimalGenerators.java:58)
	at net.jqwik.engine.properties.arbitraries.randomized.RandomGenerators.lambda$withEdgeCases$10(RandomGenerators.java:220)
	at net.jqwik.api.RandomGenerator.lambda$map$0(RandomGenerator.java:43)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator$RandomizedParameterGenerator.next(RandomizedShrinkablesGenerator.java:74)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator$RandomizedParameterGenerator.access$100(RandomizedShrinkablesGenerator.java:61)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.lambda$next$1(RandomizedShrinkablesGenerator.java:57)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.next(RandomizedShrinkablesGenerator.java:58)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.next(RandomizedShrinkablesGenerator.java:11)
	at net.jqwik.engine.properties.GenericProperty.checkWithoutReporting(GenericProperty.java:68)
	at net.jqwik.engine.properties.GenericProperty.check(GenericProperty.java:39)
	at net.jqwik.engine.execution.CheckedProperty.check(CheckedProperty.java:39)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executeProperty(PropertyMethodExecutor.java:121)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executeMethod(PropertyMethodExecutor.java:109)
	at net.jqwik.engine.execution.PropertyMethodExecutor.lambda$executePropertyMethod$0(PropertyMethodExecutor.java:89)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$static$0(AroundPropertyHook.java:17)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$null$1(AroundPropertyHook.java:32)
	at net.jqwik.engine.execution.lifecycle.AutoCloseableHook.aroundProperty(AutoCloseableHook.java:14)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$around$2(AroundPropertyHook.java:37)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executePropertyMethod(PropertyMethodExecutor.java:87)
	at net.jqwik.engine.execution.PropertyMethodExecutor.execute(PropertyMethodExecutor.java:46)
	at net.jqwik.engine.execution.PropertyTaskCreator.executeTestMethod(PropertyTaskCreator.java:73)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$1(PropertyTaskCreator.java:38)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:24)
	at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:81)
	at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
	at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:93)
	at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:82)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:175)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:157)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
	at java.lang.Thread.run(Thread.java:748)

following code

@Property
fun `what`(@ForAll @DoubleRange(min = 0.01, max = 0.2) grow: Double) {
}

works fine:)

Testing Problem

Right now I can't use jqwick with Spring because without @Test annotation nothing is autowired by spring. But with @Test annotation jqwik doesn't work.

My suggestion is to give ability to use properties inside normal junit tests.

MissingFormatWidthException when using Statistics

Testing Problem

The following code leads to a MissingFormatWidthException:

package de.gtrefs.pbt.chapter3

import de.gtrefs.pbt.chapter3.LibraryTest.Direction.*
import net.jqwik.api.Arbitraries.of
import net.jqwik.api.Arbitraries.oneOf
import net.jqwik.api.ForAll
import net.jqwik.api.Property
import net.jqwik.api.Provide
import net.jqwik.api.Statistics

class LibraryTest {
    @Property fun testSomeLibraryMethod(@ForAll("path") path: List<Direction>) {
        Statistics.label(path.size.toString()).collect(path)
    }

    @Provide
    fun path() = oneOf(of(LEFT, RIGHT, UP, DOWN)).list()

    enum class Direction {
        LEFT, RIGHT, UP, DOWN
    }

}

Stacktrace


java.util.MissingFormatWidthException: %02$d

	at java.base/java.util.Formatter$FormatSpecifier.checkNumeric(Formatter.java:3176)
	at java.base/java.util.Formatter$FormatSpecifier.checkInteger(Formatter.java:3136)
	at java.base/java.util.Formatter$FormatSpecifier.<init>(Formatter.java:2874)
	at java.base/java.util.Formatter.parse(Formatter.java:2713)
	at java.base/java.util.Formatter.format(Formatter.java:2655)
	at java.base/java.util.Formatter.format(Formatter.java:2609)
	at java.base/java.lang.String.format(String.java:2897)
	at net.jqwik.engine.properties.StatisticsCollectorImpl.formatEntry(StatisticsCollectorImpl.java:88)
	at net.jqwik.engine.properties.StatisticsCollectorImpl.createReportEntry(StatisticsCollectorImpl.java:80)
	at net.jqwik.engine.properties.StatisticsCollectorImpl.report(StatisticsCollectorImpl.java:33)
	at net.jqwik.engine.properties.GenericProperty.reportStatistics(GenericProperty.java:45)
	at net.jqwik.engine.properties.GenericProperty.check(GenericProperty.java:40)
	at net.jqwik.engine.execution.CheckedProperty.check(CheckedProperty.java:39)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executeProperty(PropertyMethodExecutor.java:134)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executeMethod(PropertyMethodExecutor.java:122)
	at net.jqwik.engine.execution.PropertyMethodExecutor.lambda$executePropertyMethod$1(PropertyMethodExecutor.java:102)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$static$0(AroundPropertyHook.java:17)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$null$1(AroundPropertyHook.java:32)
	at net.jqwik.engine.execution.lifecycle.AutoCloseableHook.aroundProperty(AutoCloseableHook.java:15)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$around$2(AroundPropertyHook.java:37)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executePropertyMethod(PropertyMethodExecutor.java:100)
	at net.jqwik.engine.execution.PropertyMethodExecutor.execute(PropertyMethodExecutor.java:46)
	at net.jqwik.engine.execution.PropertyTaskCreator.executeTestMethod(PropertyTaskCreator.java:79)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$0(PropertyTaskCreator.java:38)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:24)
	at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:81)
	at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
	at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:68)
	at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:57)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

Testing Problem

Consider a property like that:

@Property
<A> void aInValuePos(
	@ForAll List<@MyAnnotation A> list,
	@ForAll @MyAnnotation A anA
)

In an arbitrary provider you have access to annotation @MyAnnotation through the provided instance of TypeUsage. For anA and list to use the exact same arbitrary will require a duplication of @MyAnnotation. It would be preferrable to write:

@Property
<@MyAnnotation A> void aInValuePos(
	@ForAll List<A> list,
	@ForAll A anA
)

Discussion

It might not be a good idea to mix different kinds of annotations into the same getter.
Consider adding a new method like TypeUsage.getTypeAnnotations().

Add todo method to fail parts which need to be done

Testing Problem

The problem is actually not specific to the test engine itself. The idea is to have some utility to indicate that some part of the application still needs some work. For example:

public String readArbitraryStuffFromSomeStreamAndReturnItAsString(Stream wellBehavingStream){
  return todo();
}

In this way one is able to compile the code while still failing the parts which need refurbishment.

Discussion

The advantages I have in mind are the following:

Indicates what parts need to be done before comitting
If the local tests fail, this is a clear indicator that the part of the code is not ready to commit. This is better than having a TODO comment which does not have an effect on the build.
Code compiles and thus executes other tests
Depending on the used compiler, code will not be executed unless everything compiles. While this property might be used in TDD, it still has some burden. Especially in the case of example based testing, the happy path might be implemented but the corner cases are not yet tested.

The disadvantages I can think of are:

Mixin of test dependency into production code
In order to compile properly the todo method needs to be part of the production code. Actually, this should not be the case because production would have a hard dependency on test libraries.

setup test selection by means of tagging with gradle / intellij

Please help me setup test selection through tagging. I use intellij community 2018.2.5 and gradle 4.8. The jqwik user guide directs me to gradle/junit5 and intellij sites, but none of these solutions work for me.

I have a basic build.gradle:

buildscript {
    dependencies {
        classpath 'org.junit.platform:junit-platform-gradle-plugin:1.1.1'
    }
}

plugins {
    id 'java'
}

group 'diamond'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

apply plugin: 'org.junit.platform.gradle.plugin'

repositories {
    mavenCentral()
}

ext.junitPlatformVersion = '1.3.1'
ext.junitJupiterVersion = '5.3.1'

ext.jqwikVersion = '0.9.1'

junitPlatform {
    filters {
        includeClassNamePattern '.*Test'
        includeClassNamePattern '.*Tests'
        includeClassNamePattern '.*Properties'
    }
    // Only enable if you also want to run tests outside the junit platform runner:
    enableStandardTestTask false
}

dependencies {
    testCompile("org.junit.platform:junit-platform-launcher:${junitPlatformVersion}")
    testCompile "net.jqwik:jqwik:${jqwikVersion}"
    testCompile("org.assertj:assertj-core:3.9.1")
}

and I want to create a "skip" tag like this:

    @Example
    @Tag("skip")
    public boolean Example_jqwik_is_cool()
    {
        return true;
    }

Adding a test {} block to gradle did not work and also the menue entry shown in the intellij link is not available as of intellij community 2018.2.5, how can I make this work? Thank you!

FloatRange for narrow range leads to exception due to implicit conversion

Testing Problem

Running this test will lead to the following error after a few iterations

@Property boolean scaledFloats(@ForAll @FloatRange(min = 2.01f, max = 2.03f) float value) { return value >= 2.01f && value <= 2.03f; }

net.jqwik.api.JqwikException: Number <2.00> is outside allowed range 2.009999990463257..2.0299999713897705 at net.jqwik.engine.properties.shrinking.ShrinkableBigDecimal.determineTarget(ShrinkableBigDecimal.java:49) at net.jqwik.engine.properties.shrinking.ShrinkableBigDecimal.<init>(ShrinkableBigDecimal.java:21) at net.jqwik.engine.properties.arbitraries.randomized.RandomDecimalGenerators.lambda$createBaseGenerator$2(RandomDecimalGenerators.java:58) at net.jqwik.engine.properties.arbitraries.randomized.RandomGenerators.lambda$withEdgeCases$10(RandomGenerators.java:220) at net.jqwik.api.RandomGenerator.lambda$map$0(RandomGenerator.java:43)

Discussion

This seems to be caused by the conversion to double that occurs in BigDecimal.valueOf.

I'm happy to contribute a fix, I have a branch waiting to push :)

Split API and Implementation in two Artifacts

Testing Problem

Dependency surface could be smaller if API and implementation would be split in two.

Disadvantage: You have to specify two dependencies in Gradle/maven. Unless junit-team/junit5#1669 will be realized.

Problems

net.jqwik.api.Arbitraries has many static methods that delegate to implementation classes. Resolving that without using Java's module system (java 9+) is difficult (impossible?).

Do not use ascii as default String character range

The DefaultStringArbitrary class uses Arbitraries.chars().ascii() as default character arbitrary.

I think this is very supprising in the least and also makes the method ascii() method in DefaultStringArbitrary superfluous.

I would recommend to change the default to Arbitraries.chars().all().

At the moment it is not an easy feat to test unicode Strings.

jqwik should fail property test instead of ignoring it

Testing Problem

following kotlin test

class FooTest {

  @Property
  fun foo(name: String, @ForAll other: String) {
    assertNull(name)
  }
}

should fail, but is ignored (guess: name is not annotated but other is).

Testing Problem

(Version 0.8.14)
A @ForAll String currently generates a wide range of characters, defined by Unicode, however, it can also include the characters in the special code block for Non-characters (e.g: U+FFFF). Although technically part of the spec, these characters should never appear in a (valid) Unicode string. I suspect being able to generate an always valid Unicode string is a more common requirement than being able to generate a sometimes invalid one.

Discussion

Perhaps from a pure perspective the way the Strings are currently generated is more correct - so an alternative might be to add some sort of filter to exclude these non-characters, similar to the @CharRange annotation, or perhaps a constant which defines those usable characters which could be applied to @CharRange might also suit, otherwise that set of characters would have to be defined by the tester.

JqwikStringSupport accesses ClassUtils illegally

Description

JUnit 5.5 RC1 ships with compiled Java module descriptors hiding internal packages, such as org.junit.platform.commons.util.

Stacktrace

WARNING: TestEngine with ID 'jqwik' failed to discover tests
java.lang.IllegalAccessError:
class net.jqwik.engine.support.JqwikStringSupport (in module net.jqwik.engine)
cannot access class org.junit.platform.commons.util.ClassUtils (in module org.junit.platform.commons)
because module org.junit.platform.commons does not export org.junit.platform.commons.util to module net.jqwik.engine
        at [email protected]/net.jqwik.engine.support.JqwikStringSupport.parameterTypesToString(JqwikStringSupport.java:10)
...
        at [email protected]/org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:177)
        at [email protected]/org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:164)
...

Work-around

Pass the following option at test runtime:
--add-opens org.junit.platform.commons/org.junit.platform.commons.util=net.jqwik.engine

Checking an invariant should return a check result instead of throwing an exception

The check of an Invariant fails when an Exception is thrown:

Invariant<String> invariant = s -> throw new RuntimeException("I don't like strings");
invariant.check(); // Runtime exception indicates that the invariant does not hold

IMO, it would make more sense to return a CheckResult. In case the invariant does not hold, it can be CheckFailure and otherwise CheckSuccess. This makes it easier to work with the API and one knows how to indicate failures. For example:

Invariant<String> invariant = string -> CheckResult.failure("I don't like strings");
var result = invariant.check();
assertThat(result).isInstanceOf(CheckFailure.class);
var message = invariant.asFailure().getMessage();
assertThat(message).isEqualTo("I don't like strings");

JRE dependent "whitespace" generation

Testing Problem

Unfortunately what is considered as whitespace character (Character.isWhitespace(...)) has changed between Java 8 (OpenJDK) and Java 11 (OpenJDK):
In particular the U+180E (Mongolian Vovel Separator) counts in Java 8 as whitespace, while it does not in Java 11.

This causes problems because when generating whitespace chars with jqwik (which contains hard-coded lists of whitespace chars based on Java 8) and actually running the functionality on Java 11.

Discussion

To minimize the runtime overhead, the determination of the whitespace chars could (should) be done at class-level instead of instance-level.

Initialization takes long time

Testing Problem

The initialization of the following property takes a long time (15-20 seconds on my laptop). Reducing the number of generators passed to oneOf inside anyDateAndBirthDate reduces the time it takes.

package repro;

import net.jqwik.api.*;

import java.time.LocalDate;
import java.time.Month;
import java.time.YearMonth;

public class ReproductionTest {

  @Property
  void slow(
    @ForAll("anyDateAndBirthDate") Dates dates
  ) {
  }

  @Provide
  Arbitrary<Dates> () {
    return anyDates().flatMap(today ->
      Arbitraries.oneOf(
        datesBetween(today.minusYears(150), today.minusYears(1)),
        datesBetween(today.minusYears(200), today.minusYears(151)),
        datesBetween(today.plusYears(1), today.plusYears(5)),
        datesBetween(
          today.minusYears(18).minusDays(2),
          today.minusYears(18).plusDays(2)
        )
      ).map(birthDate -> new Dates(today, birthDate))
    );
  }

  Arbitrary<LocalDate> datesBetween(LocalDate low, LocalDate high) {
    return Arbitraries.integers()
      .between(low.getYear(), high.getYear())
      .flatMap(year ->
        Arbitraries.integers().between(
          year.equals(low.getYear()) ? low.getMonthValue() : 1,
          year.equals(high.getYear()) ? high.getMonthValue() : 12
        ).map(Month::of).flatMap(month -> {
          var yearMonth = YearMonth.of(year, month);
          return Arbitraries.integers().between(
            year.equals(low.getYear()) && month.equals(low.getMonth())
              ? low.getDayOfMonth()
              : 1,
            year.equals(high.getYear()) && month.equals(high.getMonth())
              ? high.getDayOfMonth()
              : yearMonth.lengthOfMonth()
          ).map(day -> LocalDate.of(year, month, day));
        })
      );
  }

  @Provide
  Arbitrary<LocalDate> anyDates() {
    return datesBetween(
      LocalDate.of(1900, 1, 1),
      LocalDate.of(2020, 12, 31)
    );
  }

  public class Dates {
    public final LocalDate today;
    public final LocalDate birthDate;

    public Dates(LocalDate today, LocalDate birthDate) {
      this.today = today;
      this.birthDate = birthDate;
    }

    @Override
    public String toString() {
      return "Dates{" +
        "today=" + today +
        ", birthDate=" + birthDate +
        '}';
    }
  }
}

Testing Problem

I have some classes with many fields that are created using the builder pattern. When the number of fields is less than 9, I can create arbitraries easily enough:

Combinators.combine(
            Arbitraries.of(MyEnum.class),
            Arbitraries.integers().between(0, 100000),
            Arbitraries.strings().alpha().numeric(),
           //...
            ).as((a, b, c /*...*/) ->
                new Foo.Builder().
                    setA(a).
                    setB(b).
                    setC(c).
                    //...
                    build());

There's some extra boilerplate, but it's not a big problem.

When there are many fields, I've been using this pattern:

var builder = Arbitraries.of(new Foo.Builder());
builder = Combinators.combine(builder, Arbitraries.of(MyEnum.class)).
    as(Foo.Builder::setA);
builder = Combinators.combine(builder, Arbitraries.integers().between(0, 100000)).
    as(Foo.Builder::setB);
builder = Combinators.combine(builder, Arbitraries.strings().alpha().numeric()).
    as(Foo.Builder::setC);
//...
return builder.map(Foo.Builder::build);

I've further reduced boilerplate with this wrapper class:

public class FluentCombinator<T> implements Arbitrary<T> {
    private final Arbitrary<T> delegate;

    public FluentCombinator(Arbitrary<T> delegate) {
        this.delegate = delegate;
    }

    public <U, A1> FluentCombinator<U> map(Combinators.F2<T, A1, U> mapper, Arbitrary<A1> a1) {
        return new FluentCombinator<>(Combinators.combine(delegate, a1).as(mapper));
    }

    @Override
    public RandomGenerator<T> generator(int genSize) {
        return delegate.generator(genSize);
    }
}

This allows for some nicer code:

new FluentCombinator<>(Arbitraries.of(new Foo.Builder())).
            map(Foo.Builder::setA, Arbitraries.of(MyEnum.class)).
            map(Foo.Builder::setB, Arbitraries.integers().between(0, 100000)).
            map(Foo.Builder::setC, Arbitraries.strings().alpha().numeric()).
            /*...*/
            map(Foo.Builder::build);

Discussion

The current boilerplate per setter isn't that painful until you have many dozens of fields.

Enhance documentation and accessibility with code examples explaining the rationale

Testing Problem

Currently a lot of code is written with a specific mindset about testing. For example, there are specifications, examples and property based tests. It would be nice to have some code examples which explain the rationale.

Discussion

I think in this way it would be easier to access jqwik. Currently, there are some ideas in this engine I am unfimiliar with. It would help me a lot to see the features in action and to derive own solutions from it.

scalacheck case clases fluent experience

Testing Problem

I have used (some time ago) scala language for testing whole project. It was pleasure to have case classes (simple, immutable "data" classes). When there was for example SingUpData class in project, in integration test I can easily create "copy" of given class (to ensure that api wont change) like this:

case class SingUpDataFromClient(name: String, age: Int, sex: Boolean);

Than in test I can use scalacheck generators (extra similar to Arbitraries and Combinators usage as in issue #46.

Than, If I need to test something, where I will simulate sending singup data in json, I have used code like this:

val randomUser = aSingupData()
val randomMaleUser = aSingupData().copy(sex = aSex())
val randomYoungUser = aSingupData().copy(age = aAge().filter(_ < 18))

Note, that aSex() method does generate random value allowed for sex. aAge() method generate random allowed value for age. I think that jqwik would call it Arbitrary generator.

I would like to have similar experience using lombok and its Data and Builder annotations (simulating "case class").

Sidenote to showcase some nice "advanced" features when customizing instance for test:

case class SingUpDataFromClient(name: String, age: Int, sex: Boolean) {
   SingUpDataFromClient withName(name: String) = this.copy(name = name)
   SingUpDataFromClient youngerMan() = this.copy(age = aAge().filter(_ < 16), sex = true)
}

Discussion

I have read issues #44 and #46. They suggest way I would like to go. Probably using full constructor of builder can give me builder instance, which can I easily modify (i.e. get me random, older woman data...). I would try to use something like this: @Property @ForAll @UseType Person.Builder aPersonBuilder and than just append moddificators and .build() at the end.

Is it good way to go?

generated functional interface instances with Type parameters result are sometimes impure

If you generate instances of functional interfaces whose return type is a type variable of the property function then that instance is not pure.

Example:

@Property <A> void aInArgsPos(
        @ForAll Function<A, Integer> f,
        @ForAll A a) {
    assertEquals(f.apply(a), f.apply(a)); // <- passes
}

@Property <A> void aInValuePos(
        @ForAll Function<Integer, A> f,
        @ForAll Integer i) {
    assertEquals(f.apply(i), f.apply(i)); // <- fails
}

jqwik hangs during generation of small Sets

Testing Problem

The following Set provider:

@Provide
Arbitrary<Set<SomeEnum>> smallSet() {
    var allowed = List.of(SomeEnum.A, SomeEnum.B, SomeEnum.C);
    return Arbitraries.of(allowed).set();
}

seems to hang forever when trying to generate the sets.
But, when Arbitraries.of(allowed).set().ofMaxSize(3) is used, everything is fine.

Testing Problem

In order to increase trust into your properties you sometimes want to ensure the minimal
statistical occurrence of certain cases and fail if it is not reached.

Discussion

The best and most intuitive API is probable not found yet. Some open questions:

checkCoverage might not be the best name
Statistics might not be the best entry point
checkCoverage could take an additional (or optional) boolean condition or boolean supplier.

checkCoverage could be added to a Statistics.collect call as fluent API like that

Statistics.collect(classifier)
   .checkCoverage(5, case1)
   .checkCoverage(10, case2)
   .checkCoverage(25); // the rest of cases

exclude chars from uD800 to uDEFF in DefaultCharacterArbitrary

see https://en.wikipedia.org/wiki/UTF-16#U+D800_to_U+DFFF

Testing Problem

If a String generated by jqwik is serialized as byte stream and read back (f.I. persist to database) encoding errors will lead to different string value.

Testing Problem

To circumvent the limitation of only being able to combine 4 arbitraries, I'd like to group them into tuples, but Tuple4 is not public.

Discussion

Ideally, there would also be some kind of ArbitraryBuilder, as writing them from scratch is quite cumbersome and using my workaround is also not great:

    @Provide()
    Arbitrary<ComplexObject> complexObjectGen() {
        Arbitrary<Foo> foos1 = fooGen();
        Arbitrary<Foo> foos2 = fooGen();
        Arbitrary<Foo> foos3 = fooGen();
        Arbitrary<Foo> foos4 = fooGen();
        Arbitrary<Bar> bars1 = barGen();
        Arbitrary<Bar> bars2 = barGen();
        Arbitrary<Bar> bars3 = barGen();
        Arbitrary<Bar> bars4 = barGen();
        
        Arbitrary<Tuple4<Foo, Foo, Bar, Bar>> combine1 = Combinators.combine(foos1, foos2, bars1, bars2).as((f1, f2, b1, b2) -> tuple(f1, f2, b1, b2));
        Arbitrary<Tuple4<Foo, Foo, Bar, Bar>> combine2 = Combinators.combine(foos3, foos4, bars3, bars4).as((f3, f4, b3, b4) -> tuple(f3, f4, b3, b4));
        
        return Combinators.combine(combine1, combine2).as((c1, c2) -> ComplexObject.create(c1.v1, c1.v2, c1.v3, c1.v4, c2.v1, c2.v2, c2.v3, c2.v4));
    }

Tutorial on programming your own generators

Program your own Generators and Arbitraries

Hello Johannes Link, I find myself checking the site everyday to see whether there is already a tutorial on how to program your own generators. I also wonder whether there could be synergy with a project like https://github.com/benas/random-beans ? Thanks for this exciting project!

Why the need for @ForAll

Surprisingly I could not find any issue that tracks this need, while the documentation states:

Currently jqwik cannot deal with parameters that are not annotated with ‘@forall’. However, this might change in future versions.

I think this is verbose to put @ForAll on every single parameter, and I think we all agree that it's the default need for everyone to inject random parameters.

Is it a technical limitation?

Documentation problem: Broken example

Documentation Problem

If I'm not mistaken, the User Guide is wrong in one place. The paragraph "Constraining parameterized types" provides the following example:

@Property
void aProperty(@ForAll @Size(min= 1) <@StringLength(max=10) String> listOfStrings) {
}

Discussion

Btw.: Thanks for this great piece of software & documentation :)

Allow injection of fixture objects as method parameters

Testing Problem

Very often an example (or maybe also a property) requires one or several objects in a given state.

Discussion

The usual approach in standard unit testing is to create fixture objects within the setup and put them into member variables. This has some drawbacks:

The dependency on those fixtures is harder to spot
It's easy to create fixtures that are not being needed in each tests which makes understanding
an individual test more difficult.

Exception while shrinking a BigDecimal within set of values

Hello,
I am using jqwik and have written a test that generates a BigDecimal Arbitrary; upon failure, when the data shrinks, there's an exception thrown "Number x outside the allowed range ". I understood that its happening because precision becomes 1 for the value while it runs shrinking logic. I have attached some screenshots for reference.

Can you help to fix this?
Thank you!

Add support for parallel execution of tests

Testing Problem

Let's pretend we have the following test case:

@Property(tries = Integer.MAX_VALUE)
public void testSomething(@ForAll final List<@IntRange Integer> input) {
    // ...
}

When we run this test on a computer that has 4/8, 8/16, 16/32, or 32/64 cores/threads, splitting the test space into 8, 16, 32, or 64 partitions each running in a different thread can significantly decrease the run time of the test.

Testing Problem

We want to generate a sequence of BigDecimals, however since we encountered already a problem with 0.00 we would like to add this value as a special case to all our property based tests.

From the documentation we thought that using samples is the way to go:

@Provide
Arbitrary<BigDecimal> decimals() {
    return Arbitraries.bigDecimals().withSamples(new BigDecimal("0.00"));
}

However, when using this arbitrary we only get the single sample value and no random value at all:

@Property
void shouldTestRandomBigDecimals(@ForAll("decimals") BigDecimal v) {
    System.out.println(v);
}

This will output

0.00
0.00
0.00
0.00 
0.00
etc...

What have we done wrong?

I tried to debug this, but all I found out was that the WithSamplesGenerator will be newly created for every new property, thus starting again with the first sample value.

Combinator with more than eight arbitraries

Creation of a combinator with more than eight arbitraries

In the doc it says, that I should create an issue, once I need more than eight arbitraries, so here I am :)
Unfortunately all those arbitraries have different types, so I cannot use Combinators.ListCombinator<T> as input.
An hacky way to create Objects with more than 8 fields might be to use the @interface and then pass those to each @ForAll but that would also make the tests pretty unreadable.

Is there another way to do that right now? Maybe how to create a provider with any amount of arbitraries?

Cheers,
Janos

Tests with failed setup do not fail gradle build

Testing Problem

Given test like this:

package xxx;

import net.jqwik.api.Example;

class FailingConstructorTest {

    private final String testConfig;

    FailingConstructorTest() {
        testConfig = loadConfig();
    }

    @Example
    void alwaysPass() {
        // do nothing
    }

    private String loadConfig() {
        throw new IllegalStateException();
    }
}

When running jqwik tests with gradle from command line: gradle test -i,
the error is printed out:

xxx.FailingConstructorTest STANDARD_ERROR
    May 29, 2019 4:47:59 PM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
    WARNING: TestEngine with ID 'jqwik' failed to execute tests
    net.jqwik.api.JqwikException: Cannot create instance of class 'class xxx.FailingConstructorTest'. Maybe it has no default constructor?
        at net.jqwik.engine.execution.PropertyTaskCreator.createTestInstance(PropertyTaskCreator.java:46)
        at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$0(PropertyTaskCreator.java:16)
        at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:24)
        at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:81)
        at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
        at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:93)
        at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:82)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
        at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
        at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
        at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
        at com.sun.proxy.$Proxy2.stop(Unknown Source)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:175)
        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:157)
        at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
        at java.base/java.lang.Thread.run(Thread.java:834)
    Caused by: java.lang.IllegalStateException
        at xxx.FailingConstructorTest.loadConfig(FailingConstructorTest.java:19)
        at xxx.FailingConstructorTest.<init>(FailingConstructorTest.java:10)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at org.junit.platform.commons.util.ReflectionUtils.newInstance(ReflectionUtils.java:453)
        at org.junit.platform.commons.util.ReflectionUtils.newInstance(ReflectionUtils.java:428)
        at org.junit.platform.commons.support.ReflectionSupport.newInstance(ReflectionSupport.java:180)
        at net.jqwik.engine.support.JqwikReflectionSupport.newInstanceWithDefaultConstructor(JqwikReflectionSupport.java:69)
        at net.jqwik.engine.execution.PropertyTaskCreator.createTestInstance(PropertyTaskCreator.java:40)
        ... 40 more

but the test is not reported as failed and gradle build is finished as successful. This behavior seems to occur since version 1.0.0 (tested on 1.0.0, 1.1.2 and 1.1.4).

When using jqwik version <= 0.9.3 (tested on 0.9.3, 0.9.0 and 0.8.6), the above test is reported as failed, causing gradle to fail correctly.

Question - Not a Testing Problem

What is the different between the #1 example code from README.md and this lines of code here?
From README:

class FizzBuzzTests {
	@Property
	boolean every_third_element_starts_with_Fizz(@ForAll("divisibleBy3") int i) {
		return fizzBuzz().get(i - 1).startsWith("Fizz");
	}

	@Provide
	Arbitrary<Integer> divisibleBy3() {
		return Arbitraries.integers().between(1, 100).filter(i -> i % 3 == 0);
	}

	private List<String> fizzBuzz() {
		return IntStream.range(1, 100).mapToObj((int i) -> {
			boolean divBy3 = i % 3 == 0;
			boolean divBy5 = i % 5 == 0;

			return divBy3 && divBy5 ? "FizzBuzz"
				: divBy3 ? "Fizz"
				: divBy5 ? "Buzz"
				: String.valueOf(i);
		}).collect(Collectors.toList());
	}
}

My Code:

    @Test
    public void fizzBuzz_startsWithFizzEach3thItem_shallBeTrue() {
        List<Integer> integerList = new ArrayList<>();
        IntStream.range(1, 100).filter(i -> i % 3 == 0).forEach(i -> integerList.add(i));
        for (Integer i : integerList) {
            assertTrue("fizzBuzz Index " + (i - 1) + " shall start with \"Fizz\"!",
                    fizzBuzz().get(i - 1).startsWith("Fizz"));
        }
    }

    private List<String> fizzBuzz() {
      // same code from README ...
    }

First my code here is very ugly and i will never use some code like that in my projects/at work.
Because assertions inside Loops are an bad pattern.

I don't want to start a sh*t-storm here, but i only want to discuss this with the whole java-users here.

Discussion

Old way:
Junit4/5
Hamcrest Matchers

Avoids misunderstandings.

We have enough libraries with magic behind the scenes.

Is this really need to use a new way?
The reason of my Issue is only misunderstandings about the what happen here behind the scenes.
For Newbies that i want to present code like jqwik's example, will be hard to understand.

I am ready to be convinced.

Maybe the example is not enough for me, but can good for other developers.

Same erasure when expanding @Odd annotation

Testing Problem

I followed the documentation for the implementation for @Odd but quickly ran into an issue when I wanted to add a configure method for BigInteger.

configure(Arbitrary<BigInteger>,Odd) and configure(Arbitrary<Integer>,Odd) have the same erasure

Testing Problem

Probably two issues in one, but I would like to use JUnit 5 extensions and normal lifecycle annotations like @BeforeEach, @BeforeAll etc.

I also noticed the warning:
WARNING: [class ...] has annotation [@org.junit.jupiter.api.extension.Extensions(value={@org.junit.jupiter.api.extension.ExtendWith(value={....class}), @org.junit.jupiter.api.extension.ExtendWith(value={....class})})] from JUnit which cannot be processed by jqwik

So I'm wondering what the challenges with supporting that.

Discussion

It would be a killer feature IMO because there is no other property test framework with this kind of support (to my knowledge).

Generate random beans

I guess a lot of developers using Property Based Testing are also keen on functional programming (where PBT originated), and thus have immutable dumb POJOs to store data. Building generators for these is annoying in Jqwik because I have to build a random value for each field and then invoke the constructor or setters.
This issue was already slightly discussed in #7 (with Random Beans) but unfortunately there was no follow up.

Unlike http://pholser.github.io/junit-quickcheck/site/0.8.1/usage/complex-types.html gen().fieldsOf() syntax which is dead simple for this use case, I could not find an equally simple way to build random beans/POJOs in Jqwik. Writing everything by hand seems annoying at best, an easy way to make mistakes at worst.

Kotlin interoperability

Testing Problem

In Kotlin we want to express something like this:

@Property
fun myTest(@ForAll bigDecimals : List<BigDecimal>) {...}

which is (almost) the equivalent to the following Java code:

@Property
void myTest(@ForAll List<BigDecimal> bigDecimals) {...}

What looks easy and works out of the box with Java produces a net.jqwik.api.CannotFindArbitraryException: Cannot find an Arbitrary for Parameter of type [@net.jqwik.api.ForAll(value=) List<? extends BigDecimal>] with Kotlin.
The difference is obviously the type signature which for Java is List<BigDecimal> while it is List<? extends BigDecimal> for Kotlin.
Due to this little difference, the BigDecimalArbitraryProvider decides that he is unable to provide values for the given targetType.

Discussion

We tried to get around this by creating our own provider for BigDecimals, but experienced problems in the aftermath when we had trouble to let our provider correctly work with jqwik constraint annotations (like @bigrange). Probably we are missing some knowledge here . . .

jqwik-team / jqwik Goto Github PK

jqwik's People

Contributors

Stargazers

Watchers

Forkers

jqwik's Issues

Testing Problem

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Test Execution Problem

Desired behaviour

Discussion

Testing Problem

Suggested Solution

Problem

Suggested Solution

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Testing Problem

Testing Problem

Suggested Solution

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Discussion

Testing Problem

Problems

Testing Problem

Suggested Solution

Testing Problem

Suggested Solution

Discussion

Description

Stacktrace

Work-around

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Testing Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution

Testing Problem

Suggested Solution

Discussion

Program your own Generators and Arbitraries

Documentation Problem

Suggested Solution

Discussion

Testing Problem

Suggested Solution