GithubHelp home page GithubHelp logo

lincheck's Introduction

Lincheck

Kotlin Beta JetBrains official project License: MPL 2.0

Lincheck is a practical and user-friendly framework for testing concurrent algorithms on the JVM. It provides a simple and declarative way to write concurrent tests.

With the Lincheck framework, instead of describing how to perform tests, you can specify what to test by just declaring all the data structure operations to examine. After that, Lincheck automatically generates a set of random concurrent scenarios, examines them using either stress-testing or bounded model checking, and verifies that the results of each invocation satisfy the required correctness property (linearizability by default).

Documentation and Presentations

Please see the official tutorial that showcases Lincheck features through examples.

You may also be interested in the following resources:

Using in Your Project

To use Lincheck in your project, you need to add it as a dependency. If you use Gradle, add the following lines to build.gradle.kts:

repositories {
   mavenCentral()
}

dependencies {
   // Lincheck dependency
   testImplementation("org.jetbrains.kotlinx:lincheck:2.32")
}

Example

The following Lincheck test easily finds a bug in the standard Java's ConcurrentLinkedDeque:

import org.jetbrains.kotlinx.lincheck.*
import org.jetbrains.kotlinx.lincheck.annotations.*
import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.*
import org.junit.*
import java.util.concurrent.*

class ConcurrentDequeTest {
    private val deque = ConcurrentLinkedDeque<Int>()

    @Operation
    fun addFirst(e: Int) = deque.addFirst(e)

    @Operation
    fun addLast(e: Int) = deque.addLast(e)

    @Operation
    fun pollFirst() = deque.pollFirst()

    @Operation
    fun pollLast() = deque.pollLast()

    @Operation
    fun peekFirst() = deque.peekFirst()

    @Operation
    fun peekLast() = deque.peekLast()

    // Run Lincheck in the stress testing mode
    @Test
    fun stressTest() = StressOptions().check(this::class)

    // Run Lincheck in the model checking testing mode
    @Test
    fun modelCheckingTest() = ModelCheckingOptions().check(this::class)
}

When running modelCheckingTest(), Lincheck not only detects a bug but also provides a comprehensive interleaving trace that explains it:

= Invalid execution results =
| -------------------------------------- |
|     Thread 1     |      Thread 2       |
| -------------------------------------- |
| addLast(4): void |                     |
| -------------------------------------- |
| pollFirst(): 4   | addFirst(-4): void  |
|                  | peekLast(): 4 [-,1] |
| -------------------------------------- |

---
All operations above the horizontal line | ----- | happen before those below the line
---
Values in "[..]" brackets indicate the number of completed operations
in each of the parallel threads seen at the beginning of the current operation
---

The following interleaving leads to the error:
| addLast(4): void                                                                                          |                      |
| pollFirst()                                                                                               |                      |
|   pollFirst(): 4 at ConcurrentLinkedDequeTest.pollFirst(ConcurrentLinkedDequeTest.kt:29)                  |                      |
|     first(): Node@1 at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:915)                    |                      |
|     item.READ: null at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:917)                    |                      |
|     next.READ: Node@2 at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:925)                  |                      |
|     item.READ: 4 at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:917)                       |                      |
|     prev.READ: null at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:919)                    |                      |
|     switch                                                                                                |                      |
|                                                                                                           | addFirst(-4): void   |
|                                                                                                           | peekLast(): 4        |
|     compareAndSet(Node@2,4,null): true at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:920) |                      |
|     unlink(Node@2) at ConcurrentLinkedDeque.pollFirst(ConcurrentLinkedDeque.java:921)                     |                      |
|   result: 4                                                                                               |                      |

Contributing

See Contributing Guidelines.

Acknowledgements

This is a fork of the Lin-Check framework by Devexperts.

lincheck's People

Contributors

alefedor avatar avpotapov00 avatar etolstoy avatar eupp avatar free0u avatar koshachy avatar krock21 avatar liying2010 avatar lowasser avatar mvicsokolova avatar ndkoval avatar nikpachoo avatar okue avatar rendner avatar sokolovamaria avatar tsitelov avatar zuevmaxim avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lincheck's Issues

[PROPOSAL] Customizable scenario generator

A сustomizable scenario generator must be able to create scenario according to the rules set by the user. The user should be able to influence the generation of both threads and specific operations.

Namely:

  • order of operations
  • operands of operations
  • number of operations

Usage examples:

  • binding a method to a separate thread
  • import the script
  • the ability to set arbitrary operations
  • execution complementary operations in different threads with a common operand

Thread yield transformer implementation is slightly incorrect

Version: 2.16 (and the previous ones, too)

Issue

As you can see in the implementation of ThreadYieldTransformer, in order to understand whether it should remove yield-call, it checkes owner. But it doesn't take into account that owner means the actual callee (not the concrete class which method will be called).

In Java it's possible to call static methods from base classes. So, if we have a derived thread instance and call its yield, then it will not be removed.

But, I'm not sure that it's a big problem, because I think that almost all users call exactly Thread.yield().

How to fix

We can easily check that owner extends Thread as it's done in RandomTransformer. ASAIU, it fixes the problem.

Can not use java.util.logging in my data class?

I want to use java.util.logging to print debugging information. But the Lincheck test would fail if I add java logging in my data class.

> linCheck FAILED
    java.lang.InternalError: CallerSensitive annotation expected at frame 1
        at java.base/jdk.internal.reflect.Reflection.getCallerClass(Native Method)
        at org.jetbrains.kotlinx.lincheck.tran$f*rmed.java.util.logging.Logger.getLogger(Logger.java:701)

unit test implementation for repository

Hi, I have an implementation question for a unit test.

How can I test this case, where I need to mock the repository because I'm not interested in actually saving it, but I'm interested in knowing what the concurrency behavior is when saving information.

Code:
image

Test:
image

Repository:
image

Entity:
image

How to check the time cost of a test

Hi,
Thanks to your help, I've successfully built and run the example

I have one more question now.
How could I get the time cost of a Linearizability Test?

For example, each run of a Linearizability verification test consists of the following time.

  • the starting of the program
  • time of collecting the trace.
  • time of linearizability verification
  • post-processing( I'm not sure if there is this step)

I want to get the time cost of linearizability verification.

Thanks :)

internal lincheck NPE in monitor wait deadlock

The following lincheck test deadlocks on a monitor wait, but lincheck fails to report the deadlock and rather throws an internal NPE.

import org.jetbrains.kotlinx.lincheck.annotations.Operation
import org.jetbrains.kotlinx.lincheck.check
import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.ModelCheckingOptions
import org.junit.jupiter.api.Test

class LinTest {

    private val guard = Object()

    @Operation
    fun foo() {
        synchronized(guard) {
            guard.wait()
        }
    }

    @Test
    fun test() = ModelCheckingOptions().actorsBefore(0).check(this::class)
}

Output:

java.lang.NullPointerException
	at org.jetbrains.kotlinx.lincheck.strategy.managed.MonitorTracker.releaseMonitor(ManagedStrategy.kt:867)
	at org.jetbrains.kotlinx.lincheck.strategy.managed.ManagedStrategy.beforeLockRelease$lincheck(ManagedStrategy.kt:473)
	at LinTest.foo(LinTest.kt:12)
	at org.jetbrains.kotlinx.lincheck.runner.TestThreadExecution38.run(Unknown Source)
	at org.jetbrains.kotlinx.lincheck.runner.FixedActiveThreadsExecutor$testThreadRunnable$1.run(FixedActiveThreadsExecutor.kt:173)
	at java.base/java.lang.Thread.run(Thread.java:1589)

Error message in wrong generator configuration

When I use a configuration like:
@Param(gen = IntGen.class, conf = "3,5")

I am getting an error - "Configuration should have two arguments (begin and end) separated by comma"

java.lang.IllegalStateException: Cannot create parameter gen

	at org.jetbrains.kotlinx.lincheck.CTestStructure.createGenerator(CTestStructure.java:164)
	...................
eflect.Constructor.newInstance(Constructor.java:423)
	at org.jetbrains.kotlinx.lincheck.CTestStructure.createGenerator(CTestStructure.java:162)
	... 70 more
Caused by: java.lang.IllegalArgumentException: Configuration should have two arguments (begin and end) separated by comma
	at org.jetbrains.kotlinx.lincheck.paramgen.IntGen.<init>(IntGen.java:48)

I believe it should be colon

version 2.7.1

[PROPOSAL] Smart parameter setting

The idea is that after the first run of the scenario, information about the container under test appears. This information can be useful when selecting startup options.

For example, if in some operation, there are two CAS, then you need at least three threads to check.

Therefore, there is a suggestion: to make some tool that can recommend startup parameters after the execution of a scenario.

Gradle/maven instructions

Gradle/Maven setup instructions are missing. It's not that easy to get where is jagent-impl and how to include it.

So it should be a snippet somewhere like this:

maven { url 'https://kotlin.bintray.com/kotlinx' }
maven { url 'https://dl.bintray.com/devexperts/Maven' }

treating semaphore as atomic hangs operations

Version: 2.15
The model checker incorrectly detects hung operations when treating a java.util.concurrent.Semaphore as atomic. Minimal example:

class SemaLinTest {

    private val sema = Semaphore(1, true)
    private var x = 0

    @Operation
    fun getAndInc(): Int {
        sema.acquire()
        val result = x
        x += 1
        sema.release()
        return result
    }

    @Test
    fun test() = ModelCheckingOptions()
        .addGuarantee(forClasses("java.util.concurrent.Semaphore").allMethods().treatAsAtomic())
        .check(this::class)
}

Output:

org.jetbrains.kotlinx.lincheck.LincheckAssertionError: 
= The execution has hung, see the thread dump =
Execution scenario (parallel part):
| getAndInc() | getAndInc() |

Thread-1:
	java.lang.Thread.yield(Native Method)
	SemaLinTest.getAndInc(SemaLinTest.kt:21)
	java.lang.Thread.run(Thread.java:748)
Thread-0:
	java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996)
	java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
	java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
	SemaLinTest.getAndInc(SemaLinTest.kt:18)
	java.lang.Thread.run(Thread.java:748)

Cut off events related to a detected spin lock

When a spin lock is detected in the model checking mode, hundreds of events are added to the interleaving, reducing the analysis quality and the resulting interleaving simplicity. It would be better to cut off the sequence of events related to the detected spinlock.

How to implement `extractState()` correctly for abstract set data type?

I don't fully understand what is meant by 'externally observable state of the test instance' and how to provide it.

I see that extractState() should return an object implementing equals() and hashCode() but I'm not clear on how to derive the implementations. In my case I am modelling a Set ADT with get, insert and erase operations. There is no observable state of the test instance other than the Set implementation, but the Set isn't really observable in any sense, as each of its operations are concurrent.

I look forward to your help. Thank you so much.

IllegalMonitorStateException when using synchronized(object) { object.notifyAll(); }

I'm trying to test a simple lock structure using model checking. I have found that code of the form:

synchronized (object) {
    object.notifyAll()
}

results in an IllegalMonitorStateException as the current thread is apparently not the owner of the object's monitor. I would not have expected this exception as the object's monitor is clearly held at the time.

I have attached a piece of demonstration code. The exact output from Lincheck 2.12 is:

Exception in thread "main" org.jetbrains.kotlinx.lincheck.LincheckAssertionError: 
= The execution failed with an unexpected exception =
Execution scenario (parallel part):
| increment() |

java.lang.IllegalMonitorStateException
    at java.lang.Object.notifyAll(Native Method)
    at DemoLincheckIssue$SimpleLock.unlock(DemoLincheckIssue.java:29)
    at DemoLincheckIssue.increment(DemoLincheckIssue.java:48)
    at org.jetbrains.kotlinx.lincheck.runner.TestThreadExecution48.run(Unknown Source)
    at org.jetbrains.kotlinx.lincheck.runner.FixedActiveThreadsExecutor$testThreadRunnable$1.run(FixedActiveThreadsExecutor.kt:163)
    at java.lang.Thread.run(Thread.java:748)


= The following interleaving leads to the error =
Parallel part trace:
| increment()                                                                                              |
|   lock.READ: SimpleLock@1 at DemoLincheckIssue.increment(DemoLincheckIssue.java:45)                      |
|   lock() at DemoLincheckIssue.increment(DemoLincheckIssue.java:45)                                       |
|   state.READ: 0 at DemoLincheckIssue.increment(DemoLincheckIssue.java:46)                                |
|   state.WRITE(1) at DemoLincheckIssue.increment(DemoLincheckIssue.java:46)                               |
|   state.READ: 1 at DemoLincheckIssue.increment(DemoLincheckIssue.java:47)                                |
|   lock.READ: SimpleLock@1 at DemoLincheckIssue.increment(DemoLincheckIssue.java:48)                      |
|   unlock(): threw IllegalMonitorStateException at DemoLincheckIssue.increment(DemoLincheckIssue.java:48) |
|     MONITORENTER at DemoLincheckIssue$SimpleLock.unlock(DemoLincheckIssue.java:27)                       |
|     owner.WRITE(null) at DemoLincheckIssue$SimpleLock.unlock(DemoLincheckIssue.java:28)                  |
|     MONITOREXIT at DemoLincheckIssue$SimpleLock.unlock(DemoLincheckIssue.java:27)                        |
    at org.jetbrains.kotlinx.lincheck.LinChecker.check(LinChecker.kt:51)
    at org.jetbrains.kotlinx.lincheck.LinChecker$Companion.check(LinChecker.kt:182)
    at org.jetbrains.kotlinx.lincheck.LinChecker$Companion.check$default(LinChecker.kt:181)
    at org.jetbrains.kotlinx.lincheck.LinChecker.check(LinChecker.kt)
    at DemoLincheckIssue.main(DemoLincheckIssue.java:53)

I seem to get the same output with Lincheck 2.11.

DemoLincheckIssue.txt

Hash code stub transformer implementation is incorrect

Version: 2.16 (and the previous ones, too)

Issue

As you can see in the implementation of HashCodeStubTransformer, in order to understand whether it should change a call of hashCode, it checkes owner. But it doesn't take into account that owner means the actual callee on the stack (not the concrete class which method will be called).

So, there are following problems with that:

  1. If we cast an object to Any/Object with overriden hashCode and call it, then 0 is returned. For example:
val a: Any = 1
val b = 1
println(a.hashCode()) // prints 0
println(b.hashCode()) // prints 1

I suppose, it is a huge hole. First of all, it becomes inconistent and the equals-hashCode invariant doesn't work. Also, in most collections all objects are casted to Any/Object, so all hashCode invocations return 0 and collections start working slow.

  1. If we have a class that doesn't override hashCode (and so do its parents) and owner != Object, then no transformation is done. It still calls the native implementation (which usually returns the memory address) and we have the non-determinism.

How to fix

I have the following proposal. There are two fixes (one for each problem):

  1. We check that owner is Any/Object and if it's true, we check ::class of the value on the stack. If it is actually Object/Any, then we need to do the transformation.
  2. We need another class transformer. It should check if there is a class which inherits Object directly and doesn't override the default hashCode(), it should add an implementation of hashCode which returns 0.

Java 9+ instrumentation without VM options

The current model checking implementation requires the following VM options to be added:

--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/jdk.internal.util=ALL-UNNAMED

We can overcome this restriction by transforming classes via a dynamically attached Java agent. See #249.

Subtasks to do:

  • Check on the Kotlin Coroutines build - 3d
  • Investigate whether we need beforeMethodCall[1,2,3,..] optimizations - 0.5d
  • Cleanup - 1.5d
  • Code review - 3d

Plugin [id: 'kotlinx.team.infra', version: '0.2.0-dev-55'] was not found in any of the following sources

Thanks for release the code of kotlinx-lincheck

I'm trying to build the project by ./gradlew build recently. However following errors appear.

FAILURE: Build failed with an exception.

* Where:
Build file '/data/nfs_home_data/zhenyue_orig/repos/kotlinx-lincheck/kotlinx-lincheck/build.gradle.kts' line: 12

* What went wrong:
Plugin [id: 'kotlinx.team.infra', version: '0.2.0-dev-55'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (could not resolve plugin artifact 'kotlinx.team.infra:kotlinx.team.infra.gradle.plugin:0.2.0-dev-55')
  Searched in the following repositories:
    maven(https://dl.bintray.com/kotlin/kotlinx)
    MavenRepo
    BintrayJCenter
    Gradle Central Plugin Repository

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3s

And then I try to get more detailed info by gradle --info. And then I get following errors

Initialized native services in: /data1/xiaoyang/home/.gradle/native
The client will now receive all logging from the daemon (pid: 68791). The daemon log file: /data1/xiaoyang/home/.gradle/daemon/6.6.1/daemon-68791.out.log
Starting 3rd build in daemon [uptime: 2 mins 37.363 secs, performance: 100%, non-heap usage: 17% of 268.4 MB]
Using 64 worker leases.
Starting Build
Settings evaluated using settings file '/data/nfs_home_data/zhenyue_orig/repos/kotlinx-lincheck/kotlinx-lincheck/settings.gradle.kts'.
Projects loaded. Root project using build file '/data/nfs_home_data/zhenyue_orig/repos/kotlinx-lincheck/kotlinx-lincheck/build.gradle.kts'.
Included projects: [root project 'lincheck']

> Configure project :
Evaluating root project 'lincheck' using build file '/data/nfs_home_data/zhenyue_orig/repos/kotlinx-lincheck/kotlinx-lincheck/build.gradle.kts'.
Failed to get resource: HEAD. [HTTP HTTP/1.1 403 Forbidden: https://dl.bintray.com/kotlin/kotlinx/org/jetbrains/kotlin/multiplatform/org.jetbrains.kotlin.multiplatform.gradle.plugin/1.4.0/org.jetbrains.kotlin.multiplatform.gradle.plugin-1.4.0.pom)]
Failed to get resource: GET. [HTTP HTTP/1.1 403 Forbidden: https://dl.bintray.com/kotlin/kotlinx/kotlinx/team/infra/kotlinx.team.infra.gradle.plugin/0.2.0-dev-55/kotlinx.team.infra.gradle.plugin-0.2.0-dev-55.pom)]
Resource missing. [HTTP GET: https://repo.maven.apache.org/maven2/kotlinx/team/infra/kotlinx.team.infra.gradle.plugin/0.2.0-dev-55/kotlinx.team.infra.gradle.plugin-0.2.0-dev-55.pom]
Resource missing. [HTTP GET: https://jcenter.bintray.com/kotlinx/team/infra/kotlinx.team.infra.gradle.plugin/0.2.0-dev-55/kotlinx.team.infra.gradle.plugin-0.2.0-dev-55.pom]
Resource missing. [HTTP GET: https://jcenter.bintray.com/kotlinx/team/infra/kotlinx.team.infra.gradle.plugin/0.2.0-dev-55/kotlinx.team.infra.gradle.plugin-0.2.0-dev-55.pom]

FAILURE: Build failed with an exception.

* Where:
Build file '/data/nfs_home_data/zhenyue_orig/repos/kotlinx-lincheck/kotlinx-lincheck/build.gradle.kts' line: 12

* What went wrong:
Plugin [id: 'kotlinx.team.infra', version: '0.2.0-dev-55'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (could not resolve plugin artifact 'kotlinx.team.infra:kotlinx.team.infra.gradle.plugin:0.2.0-dev-55')
  Searched in the following repositories:
    maven(https://dl.bintray.com/kotlin/kotlinx)
    MavenRepo
    BintrayJCenter
    Gradle Central Plugin Repository

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 4s

It seems that bintray does no longer host the dependency [id: 'kotlinx.team.infra', version: '0.2.0-dev-55'].

Do you have any suggestions about how to successfully build the project?

Thanks.

Add `testingTime(..)` option to configure Lincheck tests

There is almost no intuition on how to configure Lincheck tests. How to choose the number of threads and operations in each of them? How many different scenarios should be studied? Which number of invocations is sufficient to examine each of them? Users usually adjust these parameters in a way to achieve acceptable execution time. Lincheck should provide a way to configure only the total execution time, automatically tuning all the parameters most efficiently. This feature lowers the entry threshold and improves the test quality.

While this feature is non-trivial, we can start with some straightforward implementation, improving it later. The intuition is that any straightforward implementation should not be worse than the current approach, as nobody knows how to configure Lincheck tests properly.

How to run a single linearizability test?

Thanks for your previous kind reply, I've successfully run ./gradlew build

Now I want to run the test ConcurrentLinkedQueueTest.

I've tried the following commands. However, all of them failed

./gradlew test  --tests org.jetbrains.kotlinx.lincheck.test.verifier.linearizability.ConcurrentLinkedQueueTest -i
./gradlew test  -Dtest.single=org.jetbrains.kotlinx.lincheck.test.verifier.linearizability.ConcurrentLinkedQueueTest
gradle test --tests org.jetbrains.kotlinx.lincheck.test.verifier.linearizability.ConcurrentLinkedQueueTest -i
gradle test --tests ConcurrentLinkedQueueTest -i

cd build/classes/kotlin/jvm/test/
java -cp .  org.jetbrains.kotlinx.lincheck.test.verifier.linearizability.ConcurrentLinkedQueueTest
java -cp .:$PROJ_PATH/kotlinx-lincheck/build/libs/lincheck-jvm-2.14.1.jar  org.jetbrains.kotlinx.lincheck.test.verifier.linearizability.ConcurrentLinkedQueueTest

Could you give me any hint that how to run a single test?
Thanks

Multiple runtime exceptions happens when testing suspend functions

I have a stable and reproducible example of RuntimeExceptions which thrown out when suspending functions used together with kotlin-stdlib-jdk8 https://github.com/vladimir-bukhtoyarov/bucket4j/blob/5.0_try_lincheck/lincheck-tests/src/test/kotlin/io/github/bucket4j/distributed/proxy/optimizers/batch/mock/BatchingAsyncExecutorLincheckTest.kt

It looks as kotlin.UninitializedPropertyAccessException when minimizeFailedScenario=true

= Iteration 1 / 10 =
Execution scenario (parallel part):
| *testBatching(1)  | testBatching(9)   | testBatching(10)  |
| *testBatching(8)  | *testBatching(16) | testBatching(14)  |
| *testBatching(12) | *testBatching(2)  | *testBatching(20) |
| *testBatching(15) | testBatching(18)  | *testBatching(18) |
| testBatching(14)  | *testBatching(3)  | testBatching(16)  |
Execution scenario (post part):
[]

kotlin.UninitializedPropertyAccessException: lateinit property by has not been initialized

	at org.jetbrains.kotlinx.lincheck.ResumedResult.getBy(Result.kt:110)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State$getResumedOperations$1.accept(LTS.kt:182)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State$getResumedOperations$1.accept(LTS.kt:77)
	at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State.getResumedOperations(LTS.kt:181)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State.access$getResumedOperations(LTS.kt:77)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State$nextByRequest$transitionInfo$1.apply(LTS.kt:95)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State$nextByRequest$transitionInfo$1.apply(LTS.kt:77)
	at java.util.HashMap.computeIfAbsent(HashMap.java:1126)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State.nextByRequest(LTS.kt:89)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State.next(LTS.kt:86)
	at org.jetbrains.kotlinx.lincheck.verifier.linearizability.LinearizabilityContext.nextContext(LinearizabilityVerifier.kt:73)
	at org.jetbrains.kotlinx.lincheck.verifier.linearizability.LinearizabilityContext.nextContext(LinearizabilityVerifier.kt:45)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:48)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:49)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:49)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:49)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verifyResultsImpl(AbstractLTSVerifier.kt:40)
	at org.jetbrains.kotlinx.lincheck.verifier.CachedVerifier.verifyResults(CachedVerifier.java:42)
	at org.jetbrains.kotlinx.lincheck.strategy.Strategy.verifyResults(Strategy.java:57)
	at org.jetbrains.kotlinx.lincheck.strategy.stress.StressStrategy.run(StressStrategy.java:75)
	at org.jetbrains.kotlinx.lincheck.LinChecker.runScenario(LinChecker.java:168)
	at org.jetbrains.kotlinx.lincheck.LinChecker.checkImpl(LinChecker.java:103)
	at org.jetbrains.kotlinx.lincheck.LinChecker.lambda$check$0(LinChecker.java:86)
	at java.util.Collections$SingletonList.forEach(Collections.java:4822)
	at org.jetbrains.kotlinx.lincheck.LinChecker.check(LinChecker.java:84)
	at org.jetbrains.kotlinx.lincheck.LinChecker.check(LinChecker.java:74)
	at io.github.bucket4j.distributed.proxy.optimizers.batch.mock.BatchingAsyncExecutorLincheckTest.runTest(BatchingAsyncExecutorLincheckTest.kt:40)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

It looks as java.lang.IllegalStateException when minimizeFailedScenario=false Sometimes as

java.lang.IllegalStateException: Cancelled operations should not be processed as the resumed ones

	at org.jetbrains.kotlinx.lincheck.verifier.LTS.invoke(LTS.kt:213)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS.access$invoke(LTS.kt:59)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State$nextByCancellation$1.apply(LTS.kt:118)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State$nextByCancellation$1.apply(LTS.kt:77)
	at java.util.HashMap.computeIfAbsent(HashMap.java:1126)
	at org.jetbrains.kotlinx.lincheck.verifier.LTS$State.nextByCancellation(LTS.kt:114)
	at org.jetbrains.kotlinx.lincheck.verifier.linearizability.LinearizabilityContext.nextContext(LinearizabilityVerifier.kt:68)
	at org.jetbrains.kotlinx.lincheck.verifier.linearizability.LinearizabilityContext.nextContext(LinearizabilityVerifier.kt:45)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:48)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:49)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verify(AbstractLTSVerifier.kt:49)
	at org.jetbrains.kotlinx.lincheck.verifier.AbstractLTSVerifier.verifyResultsImpl(AbstractLTSVerifier.kt:40)
	at org.jetbrains.kotlinx.lincheck.verifier.CachedVerifier.verifyResults(CachedVerifier.java:42)
	at org.jetbrains.kotlinx.lincheck.strategy.Strategy.verifyResults(Strategy.java:57)
	at org.jetbrains.kotlinx.lincheck.strategy.stress.StressStrategy.run(StressStrategy.java:75)
	at org.jetbrains.kotlinx.lincheck.LinChecker.runScenario(LinChecker.java:168)
	at org.jetbrains.kotlinx.lincheck.LinChecker.checkImpl(LinChecker.java:103)
	at org.jetbrains.kotlinx.lincheck.LinChecker.lambda$check$0(LinChecker.java:86)
	at java.util.Collections$SingletonList.forEach(Collections.java:4822)
	at org.jetbrains.kotlinx.lincheck.LinChecker.check(LinChecker.java:84)
	at org.jetbrains.kotlinx.lincheck.LinChecker.check(LinChecker.java:74)
	at io.github.bucket4j.distributed.proxy.optimizers.batch.mock.BatchingAsyncExecutorLincheckTest.runTest(BatchingAsyncExecutorLincheckTest.kt:40)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

Could somebody point out what I am doing in the wrong way?

NoSuchFieldException for com.fasterxml.jackson.databind.ObjectMapper

I have a class that I test containing com.fasterxml.jackson.databind.ObjectMapper to store json in the file.
The test suite looks like this:

@StressCTest
@ModelCheckingCTest
class ATest : VerifierState() {
    private val a = A(Files.createTempFile("storage", ".json"))
...
}

class A(...) {
        private val objectMapper = ObjectMapper()
}

When running tests, I get the following error:

java.lang.ExceptionInInitializerError
	<here it points to the ObjectMapper line>
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
...
Caused by: java.lang.IllegalStateException: Cannot transform class com.fasterxml.jackson.databind.ser.BeanPropertyWriter
	at org.jetbrains.kotlinx.lincheck.TransformationClassLoader.loadClass(TransformationClassLoader.java:137)
	at com.fasterxml.jackson.databind.ObjectMapper.<clinit>(ObjectMapper.java:350)
	... 43 more
Caused by: java.lang.IllegalStateException: Cannot transform class com.fasterxml.jackson.databind.ser.PropertyWriter
	at org.jetbrains.kotlinx.lincheck.TransformationClassLoader.loadClass(TransformationClassLoader.java:137)
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
...
Caused by: java.lang.NoSuchFieldException
	at org.jetbrains.kotlinx.lincheck.strategy.managed.ManagedStrategyTransformerKt.findField(ManagedStrategyTransformer.kt:1422)
	at org.jetbrains.kotlinx.lincheck.strategy.managed.ManagedStrategyTransformerKt.isFinalField(ManagedStrategyTransformer.kt:1407)
	... 68 more

[PROPOSAL] Support for `suspend` operations cancellation

Most of synchronization and communication primitives in Kotlin Coroutines support cancellation on their suspend requests. For example, both receive and send operations on rendezvous channels suspend if there is no opposite request waiting on this channel. At the same time, these requests can be canceled while waiting for opposite ones; these canceled requests should be ignored by further operations. Moreover, this cancellation mechanism should be linearizable for rendezvous channels, as well as for most of the other primitives.

This proposal aims at adding cancellation to test scenarios. First of all, this should be supported by scenario generators, so that they can add request cancellations in addition to other operations. Despite the fact that in real-world requests and cancellations can be invoked from different threads, here I support adding cancellations right after the corresponding requests, with some probability (e.g., 50%). See the following example of such a scenario; this receive operation can be canceled in case of suspension, and the result should be CANCELLED if the cancellation succeeds.

send(4): Unit | receive(): Int or CANCELLED

Since not all requests can be canceled, this proposal suggests adding a new cancellable parameter to the Operation annotation. This parameter is true by default and affects only suspend functions. Thus, it is possible to tell scenario generators that some operations cannot be canceled.

Currently, all suspend functions are invoked via suspendCoroutineUninterceptedOrReturn, which returns COROUTINE_SUSPENDED if this function has been suspended. In this case, the corresponding lincheck thread spins until either the request is resumed (interceptors are used to detect this) or the overall execution is completed. In case of cancellation, lincheck should cancel the request instead of waiting in a spin loop.

The last but not the least problem to be solved is intercepting continuations associated with cancellable request invocations so that they can be canceled by lincheck (see CancellableContinuation.cancel(..) function). Usually, cancellable operations use suspendAtomicCancellableCoroutine function, which creates a CancellableContinuation instance and invokes the getResult method onto it. This proposal suggests storing the continuation instance to some thread-local storage (e.g. (Thread.currentThread() as LinCheckThread).cont) if getResult invocation returns false. In order to implement this, byte-code transformation paired with custom class loaders can be used.

Improve scenario minimization for concurrent containers

Consider, we have a channel with the following incorrect execution found:

= Invalid execution results: =
Parallel part:
| receive(): S | send(5):           S + void | send(2):           void  |
| send(4):   - | isClosedForSend(): false    | isClosedForSend(): false |
| offer(1):  - | isClosedForSend(): false    | receive():         2     |

Here, we can easily remove send(2) and receive(): 2 from the last thread. At the same time, removing these operations one by one could not work since the removals change the original scenario. However, removing such pairs of operations is more likely to be successful.

I would suggest trying to choose two operations to be removed. Thus, the minimization algorithm complexity increases by K (K is the total number of operations), but it should be fine in practice.

parameter generator 100% correlated with operation generator

Version: 2.15
I was introducing defects into my code on the expectation that lincheck would detect them and ran into a false negative. Here is a minimal example.

@Param(name = "key", gen = IntGen::class, conf = "0:1")
class RandomLinTest {

    private val array = IntArray(size = 2)

    @Operation
    fun get(@Param(name = "key") key: Int) = array[key]

    @Operation
    fun inc(@Param(name = "key") key: Int) {
        array[key] += 1
    }

    @Test
    fun test() = ModelCheckingOptions().logLevel(LoggingLevel.INFO).check(this::class)
}

Although these operations are clearly not linearizable, lincheck doesn't detect it. Inspecting one of the generated test cases reveals why:

= Iteration 1 / 100 =
Execution scenario (init part):
[inc(1), inc(1), get(0), inc(1), inc(1)]
Execution scenario (parallel part):
| get(0) | inc(1) |
| get(0) | inc(1) |
| inc(1) | get(0) |
| get(0) | get(0) |
| inc(1) | inc(1) |
Execution scenario (post part):
[inc(1), inc(1), get(0), inc(1), get(0)]

Every get() is for key 0 and every inc() is for key 1. This is because operation generation and int generation use separate Random instances with the same seed.

Modular testing: adding guarantees on methods of java.util.concurrent.* classes fails

I'm trying to add atomic guarantees for all methods of java.util.concurrent.ConcurrentHashMap like this:

@Test
    fun runModularTesting() = ModelCheckingOptions()
        ...
        .addGuarantee(forClasses("java.util.concurrent.ConcurrentHashMap").allMethods().treatAsAtomic())
        .check(this::class.java)

The test fails with the following error:

java.lang.IllegalStateException: The result should either be a type always loaded by the system class loader (e.g., Int, String, List<T>) or implement Serializable interface; the actual class is null.

Here is the test to reproduce the error (runModularTesting ).

memory leak resulting in hung threads

Version: 2.15
I'm model checking a fairly complex piece of code and found that several iterations complete successfully but eventually lincheck detects hung threads.

Increasing heap size allows more iterations to succeed. This made me suspect a memory leak across iterations. I analyzed some heap dumps and found that most memory was consumed by ModelCheckingStrategy instances which contain huge (100s of MB) trees of potential context switch choices. It seems that each iteration uses a fresh ModelCheckingStrategy instance but something is holding on to references to the instances from completed iterations so that they are not GC'd.

E.g. when I take a snapshot during iteration 3, I see 3 instances of ModelCheckingStrategy in the dump despite there having been full GCs.

Do not ask for equals/hashCode on the sequential specification until necessary

Usually, the verification phase is super-fast, even with default equals and hashCode implementations. However, it might be critical to provide custom implementations when testing scenarios are large or when the data structure operations are expensive.

Let's not require equals/hashCode to be implemented if the verification phase takes excepted time -- Lincheck should neither throw an exception nor print a warning. However, when the verification process takes an unexpectedly long time, it would be helpful to print a warning with instructions on how to implement equals and hashCode.

[Proposal] Cleanup functions or specific Post parallel operation

Currently, I can not find a way to execute either a cleanup function or a specific operation after the test.

In custom scenario mode, it is possible to specify the specific Post operation, but there should be a way to specify an after-test cleanup method for randomised scenarios.

In certain cases, this is needed. An example would be when a data structure is dealing with closeable resources or is replicated to the disk, and those resources need to be closed after the test. Otherwise, they will cause all kinds of out-of-memory errors. This would be the opposite operation of the init part of the test.

If a dedicated cleanup function/annotation is too much or does not align with the rest, an operation that can be pinned only for post parallel part would suffice, since it could be combined with runOnce and be executed last or among last.

If I am missing something and this is already possible without replacing RandomExecutionGenerator please do let me know.

Start with the last failed scenario

When a Lincheck test fails, users usually fix the bug and re-run the test. The same scenario will likely fail if the bug has not been resolved. To detect that faster, Lincheck could save the number of failed scenario somewhere, starting next time with it.

[PROPOSAL] Binding a method to a separate thread

Scanning operations are common in concurrent data structures. Scanning operations are operations that, in the course of their execution, go through all the elements. For example, for a Set, this would be "isEpmty".
For testing, the most interesting option seems to be when such operations will be performed in a separate thread. This will increase the number of cases when several operations are performed on one element at once.

Now:

Execution scenario (init part):
[add(6), remove(1), remove(4), isEmpty(), add(0)]
Execution scenario (parallel part):
| remove(-10) | isEmpty()   | remove(4)  | isEmpty() |
| add(-9)     | isEmpty()   | add(-7)    | remove(7) |
| isEmpty()   | remove(-8)  | remove(6)  | isEmpty()|

Proposal:

Execution scenario (init part):
[add(6), remove(1), remove(4), isEmpty(), add(0)]
Execution scenario (parallel part):
| isEmpty() | remove(-10)| remove(4)| add(8)    |
| isEmpty() | isEmpty()  | add(8)   | add(1)    |
| isEmpty() | remove(-8) | remove(6)| remove(-4)|

How to interpret green test results?

Hello, I tried the library and wrote simple test to check how it discloses incorrectly written concurrency code in one of my classes.

So here's the code of my classes:

  1. Model:
package com.devexperts.dxlab.lincheck.tests.ict;

import java.util.Objects;

public class EngageTaskData {

    public EngageTaskData(String channelId, String sourceId, String channelName, String channelLabel, String sourceName) {
        this.channelId = channelId;
        this.sourceId = sourceId;
        this.channelName = channelName;
        this.channelLabel = channelLabel;
        this.sourceName = sourceName;
    }

    private String channelId;

    private String sourceId;

    private final String channelName;

    private final String channelLabel;

    private final String sourceName;

    public void setChannelId(String channelId) {
        this.channelId = channelId;
    }

    public void setSourceId(String sourceId) {
        this.sourceId = sourceId;
    }

    public String getChannelId() {
        return channelId;
    }

    public String getSourceId() {
        return sourceId;
    }

    public String getChannelName() {
        return channelName;
    }

    public String getChannelLabel() {
        return channelLabel;
    }

    public String getSourceName() {
        return sourceName;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        EngageTaskData that = (EngageTaskData) o;
        return Objects.equals(channelId, that.channelId) &&
                Objects.equals(sourceId, that.sourceId) &&
                Objects.equals(channelName, that.channelName) &&
                Objects.equals(channelLabel, that.channelLabel) &&
                Objects.equals(sourceName, that.sourceName);
    }

    @Override
    public int hashCode() {
        return Objects.hash(channelId, sourceId, channelName, channelLabel, sourceName);
    }
}
  1. Class to be tested on thread safety
package com.devexperts.dxlab.lincheck.tests.ict;

import org.apache.commons.lang3.tuple.Pair;

import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.BiFunction;

public class InMemoryEngageTaskDataStorage {

    private static final String EG_DATA_PREFIX = "engage.";
    private static final String EG_CATEGORIES_PREFIX = EG_DATA_PREFIX + "categories.";

    //private final Map<String, Pair<EngageTaskData, Long>> egChannels = new ConcurrentHashMap<>();
    private final Map<String, Pair<EngageTaskData, Long>> egChannels = new HashMap<>();

    private long engDataTtlInMillis;

    public InMemoryEngageTaskDataStorage( long expirationTimeInSeconds) {
        this.engDataTtlInMillis = expirationTimeInSeconds * 1000L;
    }

    public void storeEngageData(String egDomainId, Collection<EngageTaskData> data) {
        data.forEach(item -> {
            String key = egDomainId + "." + item.getChannelId() + "." + item.getSourceId();
            getOrCalculate(EG_DATA_PREFIX, key, item.getChannelId(), item.getSourceId(), (c, s) -> item, egChannels, engDataTtlInMillis, true);
        });
    }

    public EngageTaskData getEngageData(String channelId, String sourceId, String egDomainId) {
        String key = egDomainId + "." + channelId + "." + sourceId;
        Pair<EngageTaskData, Long> pair = egChannels.get(key);
        return pair == null ? null : pair.getKey();
    }

    public EngageTaskData getOrComputeEngageData(String channelId, String sourceId, String egDomainId, BiFunction<String, String, EngageTaskData> computer) {
        /*String key = egDomainId + "." + channelId + "." + sourceId;
        return getOrCalculate(EG_DATA_PREFIX, key, channelId, sourceId, computer, egChannels, engDataTtlInMillis, false);*/
        return getOrComputeEngageDataV2(channelId, sourceId, egDomainId, computer);
    }

    public EngageTaskData getOrComputeEngageDataV2(String channelId, String sourceId, String egDomainId, BiFunction<String, String, EngageTaskData> computer) {
        String key = egDomainId + "." + channelId + "." + sourceId;
        Pair<EngageTaskData, Long> data = egChannels.get(key);
        if (data == null || dataIsExpired(data.getRight(), engDataTtlInMillis)) {
            data = Pair.of(computer.apply(channelId, sourceId), System.currentTimeMillis());
            egChannels.put(key, data);
        }
        return data.getKey();
        //return getOrCalculate(EG_DATA_PREFIX, key, channelId, sourceId, computer, egChannels, engDataTtlInMillis, false);
    }

    static <T, A, B> T getOrCalculate(String prefix, String key, A arg1, B arg2, BiFunction<A, B, T> computer, Map<String, Pair<T, Long>> storage,
                                      long dataTtlInMillis, boolean rewrite) {
        String compoundKey = prefix + key;
        Pair<T, Long> pair = storage.compute(compoundKey, (k, v) -> {
            if (rewrite || v == null || dataIsExpired(v.getRight(), dataTtlInMillis)) {
                return Pair.of(computer.apply(arg1, arg2), System.currentTimeMillis());
            }
            return v;
        });
        return pair.getLeft();
    }

    private static boolean dataIsExpired(Long right, long ttl) {
        return (System.currentTimeMillis() - right) >= ttl;
    }

    public void clearData() {
        egChannels.clear();
    }
}
  1. And here's the Lincheck-powered test (Java, not Kotlin, yes):
package com.devexperts.dxlab.lincheck.tests.ict;

import com.devexperts.dxlab.lincheck.LinChecker;
import com.devexperts.dxlab.lincheck.annotations.Operation;
import com.devexperts.dxlab.lincheck.annotations.Param;
import com.devexperts.dxlab.lincheck.paramgen.StringGen;
import com.devexperts.dxlab.lincheck.strategy.stress.StressCTest;
import org.junit.Test;

@StressCTest(threads = 8)
@Param(name = "channelId", gen = StringGen.class)
@Param(name = "sourceId", gen = StringGen.class)
@Param(name = "egDomainId", gen = StringGen.class)
@Param(name = "sourceName", gen = StringGen.class)
public class InMemoryEngageTaskDataStorageTest {

    private InMemoryEngageTaskDataStorage storage = new InMemoryEngageTaskDataStorage(100500L);

    @Operation(params = {"channelId", "sourceId", "egDomainId", "sourceName"})
    public EngageTaskData getOrComputeEngageData(String channelId, String sourceId, String egDomainId, String sourceName) {
        return storage.getOrComputeEngageData(channelId, sourceId, egDomainId, (x, y) -> new EngageTaskData(x, y, x+ "chan", y +"chanLabel", sourceName));
    }

    @Operation
    public void clearData() {
        storage.clearData();
    }

    @Test
    public void test() {
        LinChecker.check(InMemoryEngageTaskDataStorageTest.class);
    }
}

As the result, test is stable and green (sic!). No concurrent issues found. But looking at the method:
InMemoryEngageTaskDataStorage#getOrComputeEngageData
one surely can say there are dozens of ways that something could go wrong (no synschronizations at all, collection is HashMap etc.) which test does not indicate at all..

How that could be? Should I increase iterations count to some very large number? Did I miss something in test configuration/annotating?

Many thanks in advance!

log4j2 incompatibility

lincheck version: 2.15
log4j version: 2.17.2

In my system under test I have

private val log: Logger = LogManager.getLogger("SystemUnderTestKt")

When I run a lincheck test I get

java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.util.PropertiesUtil
	at org.apache.logging.log4j.status.StatusLogger.<clinit>(StatusLogger.java:78)
	at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:61)
	at SystemUnderTestKt.<clinit>(SystemUnderTest.kt:17)
	at LinTest.<init>(LinTest.kt:111)

Ignoring log4j classes with .addGuarantee(forClasses(LogManager::class, Logger::class).allMethods().ignore()) doesn't help.

Non-lincheck unit tests for the system under test work fine.

Cut exception stack straces thrown in validation functions

When a validation function (annotated with @Validate) throws an exception, the full stack trace, including the Lincheck internals, is provided. It takes dozens of lines in the console, being useless at the same time. The suggestion is to cut the stack trace at the point of the validation function call.

[PROPOSAL] Binding of tested operations to data

When testing concurrent data structures, it seems useful to provide information to Linkchek about conjugate operations. Conjugate operations are operations that must be performed on the same data in order to be interesting in terms of how the data structure works. For example, for Set, add and delete operations will be conjugate.

Linkchek now needs to do a lot of unnecessary iterations to find useful cases. Any conjugate operations with different operands are the same option. Only the order of execution of operations will differ.

Now:

Execution scenario (init part):
[add(6), remove(1), remove(4), isEmpty(), add(0)]
Execution scenario (parallel part):
| remove(-10) | remove(10)  | remove(4)  | isEmpty() |
| add(-9)     | isEmpty()   | add(-7)    | remove(7) |
| remove(1)   | remove(-8)  | remove(6)  | remove(-4)|

Proposal:

Execution scenario (init part):
[add(6), remove(1), remove(4), isEmpty(), add(0)]
Execution scenario (parallel part):
| add(-10)   | remove(-10)| remove(4)| isEmpty() |
| add(4)     | isEmpty()  | add(8)   | add(1)    |
| remove(-2) | remove(-8) | remove(6)| remove(-4)|

A similar result is given by narrowing the range of generated values ​​for the operands:

Execution scenario (init part):
[add(1), isEmpty(), isEmpty(), remove(2), remove(2)]
Execution scenario (parallel part):
| remove(3) | remove(3) | add(3)    | add(3)    |
| remove(1) | remove(1) | remove(3) | remove(3) |
| isEmpty() | remove(3) | remove(3) | isEmpty() |

However, this method is not suitable for some data structures, for example, Hashmap, so it is important to add the ability to specify conjugate operations.

Wrong alignment in incorrect results representation

Currently a wrong alignment is added to init and post parts (additional meaningless spaces).

Example of wrong representation:

= Invalid execution results: =
Init part:
[getAmount(4):     0, transfer(3,1,-9): void, getAmount(2):     0, transfer(3,2,-7): void, setAmount(2,7):   void]
Parallel part:
| setAmount(1,1):   void | setAmount(4,-8): void | setAmount(4,6):  void |
| setAmount(4,-4):  void | setAmount(3,9):  void | setAmount(2,-1): void |
| transfer(4,3,10): void | setAmount(4,4):  void | getAmount(3):    12   |
| transfer(3,4,7):  void | setAmount(1,-8): void | getAmount(4):    -7   |
| setAmount(2,4):   void | setAmount(3,1):  void | getAmount(3):    12   |
Post part:
[transfer(4,4,7):  void, getAmount(1):     -8, getAmount(3):     12, getAmount(1):     -8, transfer(1,4,-7): void]

There are odd spaces in third and last lines.

coroutine finishing early

Version: 2.15
I'm model checking some code that uses Mutex. I ran into a case where lincheck incorrectly considers a coroutine that should still be running to have finished. Here is a minimal example:

class CoroutineLinTest {

    private var x = AtomicInteger()
    private val mutex = Mutex()

    @Operation(cancellableOnSuspension = false, allowExtraSuspension = true)
    suspend fun getAndInc(): Int {
        while (true) {
            val snapshot = x.get()
            mutex.withLock {
                if (snapshot == x.get()) {
                    return x.getAndIncrement()
                }
            }
        }
    }

    @Test
    fun test() = ModelCheckingOptions().check(this::class)
}

Here is the assertion error: coroutine-assertion-error.txt

In contrast, the equivalent code with thread synchronization model checks as expected (no counterexamples found):

class ThreadLinTest {

    private var x = AtomicInteger()
    private val mutex = Object()

    @Operation
    fun getAndInc(): Int {
        while (true) {
            val snapshot = x.get()
            synchronized(mutex) {
                if (snapshot == x.get()) {
                    return x.getAndIncrement()
                }
            }
        }
    }

    @Test
    fun test() = ModelCheckingOptions().check(this::class)
}

Incorrect warning about state equivalency relation

For tests in model checking mode, with scenario minimization turned on I get the warning about setting state equivalency repeated for every iteration of scenario minimization, I guess (with scenario minimization turned off it's ok).

Also there are several tips in the message itself, marked them in bold:

To verify outcome results faster, it is highly recommended to specify the state equivalence relation on yoursequential specification. However, on class org.jetbrains.kotlinx.lincheck.test.guide.LongAdderTest it is is not defined or is implemented incorrectly. Please, specify the equivalence relation via implementing equals() and hashCode() functions on class org.jetbrains.kotlinx.lincheck.test.guide.LongAdderTest. The most convenient way is to extend a special VerifierState class and override the extractState() function, whichextracts and returns the logical state, which is used for further equals() and hashCode() calls.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.