Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NVM testing #49

Open
wants to merge 190 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
190 commits
Select commit Hold shift + click to select a range
be2d094
NRL stress testing
zuevmaxim Nov 25, 2020
a5700d8
Some optimization
zuevmaxim Dec 14, 2020
2cda77f
Implement primitives
zuevmaxim Dec 14, 2020
98bf051
Fix
zuevmaxim Dec 14, 2020
d14e554
Use one crash exception
zuevmaxim Jan 10, 2021
af678e2
Execution result equals hashCode
zuevmaxim Jan 10, 2021
56376fb
Update primitives
zuevmaxim Jan 12, 2021
2f838bf
Implement system crash
zuevmaxim Jan 14, 2021
959c58e
Implement small set for cache
zuevmaxim Jan 16, 2021
e54fab1
Test with 3 threads
zuevmaxim Jan 16, 2021
0a22fea
Extract recoverable runner
zuevmaxim Jan 17, 2021
d5c39ba
Implement durable model run
zuevmaxim Jan 20, 2021
21508fb
Durable verification, WIP
zuevmaxim Jan 22, 2021
2d087fc
Do not save crashes
zuevmaxim Jan 23, 2021
7290fce
Context container
zuevmaxim Jan 23, 2021
5c27d3d
Reset cache
zuevmaxim Jan 24, 2021
ccdfcdb
No crashes recover model
zuevmaxim Jan 24, 2021
495c344
Busy waiting barrier
zuevmaxim Jan 24, 2021
6c8b37f
Fix increment clock on crash
zuevmaxim Jan 25, 2021
1a4efb4
Implement system crash
zuevmaxim Feb 1, 2021
c277bca
Durable recover
zuevmaxim Feb 1, 2021
f00f8c1
Await recover
zuevmaxim Feb 1, 2021
e1dcb0c
Use clocks only on crash
zuevmaxim Feb 1, 2021
34870a7
Update queue test
zuevmaxim Feb 1, 2021
0cb603a
Fix barrier
zuevmaxim Feb 1, 2021
596c060
Fix system crash for NRL
zuevmaxim Feb 2, 2021
f502f2e
Add tests
zuevmaxim Feb 2, 2021
22cb87f
Merge remote-tracking branch 'origin/develop' into test-nlr
zuevmaxim Feb 3, 2021
d0e6d26
Probability model: take actual number of 'possiblyCrash' calls into a…
zuevmaxim Feb 3, 2021
447887c
Remove durable automatic recover
zuevmaxim Feb 7, 2021
32bbf8e
Rename node
zuevmaxim Feb 9, 2021
ea6d813
Update test
zuevmaxim Feb 13, 2021
7dd37d1
Add link free test
zuevmaxim Feb 16, 2021
eceb400
Revert durable recover for all threads
zuevmaxim Feb 23, 2021
b1dbab8
Merge remote-tracking branch 'origin/develop' into test-nlr
zuevmaxim Feb 23, 2021
e49ffb0
Revert durable recover for single thread
zuevmaxim Feb 23, 2021
e93a115
Move test
zuevmaxim Feb 23, 2021
f2b78d0
Crash only before api calls
zuevmaxim Feb 23, 2021
bf92ec1
Crash on writes
zuevmaxim Feb 23, 2021
5086aa5
Introduce proxy crash error
zuevmaxim Feb 23, 2021
5b24ada
Pass CrashError through catch Throwable
zuevmaxim Feb 23, 2021
05c370a
Remove log
zuevmaxim Feb 23, 2021
79d90ca
Add uniform distribution test
zuevmaxim Mar 14, 2021
fe90e1e
Fix test
zuevmaxim Mar 21, 2021
a698a85
Fix test with recovery
zuevmaxim Mar 24, 2021
014218c
Add failing counter tests
zuevmaxim Mar 24, 2021
9df0ee6
Update persistent test
zuevmaxim Mar 24, 2021
274cb6c
Add RWO failing tests
zuevmaxim Mar 24, 2021
f1f64be
Update tests
zuevmaxim Mar 24, 2021
69d6916
Add TAS failing tests
zuevmaxim Mar 24, 2021
3529e99
Some small refactorings
zuevmaxim Mar 25, 2021
4c85ef8
Add durable queue failing tests
zuevmaxim Mar 25, 2021
4284345
Disable failing tests
zuevmaxim Mar 25, 2021
cffd14d
Add a workaround for TAS tests
zuevmaxim Mar 25, 2021
bff8892
Turn on counter test
zuevmaxim Mar 25, 2021
25a79dd
Fix failing tests
zuevmaxim Mar 25, 2021
0172605
Fix set tests
zuevmaxim Mar 25, 2021
64307da
Add uniform distribution test
zuevmaxim Mar 26, 2021
82dc16d
Use binary search
zuevmaxim Mar 26, 2021
60869ff
Reload static context per scenario
zuevmaxim Mar 28, 2021
8e48c95
Add durable set failing tests
zuevmaxim Mar 28, 2021
bb25dea
Update durable set test
zuevmaxim Mar 28, 2021
1cfa600
Add MCAS tests
zuevmaxim Mar 28, 2021
fd35947
Fix: add a crash in before return
zuevmaxim Mar 28, 2021
5b37663
Implement detectable execution model
zuevmaxim Mar 28, 2021
f9f44f6
Fix counter test
zuevmaxim Mar 28, 2021
4a8e5cc
Fix recover for detectable execution
zuevmaxim Mar 30, 2021
d5cbfb7
Fix recover for durable model
zuevmaxim Apr 4, 2021
a3aeb49
Add one actor probability model
zuevmaxim Apr 18, 2021
ef9bfe9
Enable model checking NVM options
zuevmaxim Apr 18, 2021
54b8938
Refactor tests run
zuevmaxim Apr 18, 2021
a97cc8b
Add log queue test
zuevmaxim Apr 18, 2021
ce28a0c
Prepare for model checking implementation
zuevmaxim Apr 18, 2021
e954567
Disable log queue test
zuevmaxim Apr 27, 2021
8231053
Model checking implementation for NVM (#79)
zuevmaxim May 12, 2021
7d952f5
Nvm probability (#85)
zuevmaxim May 14, 2021
387cfe3
Fix set test
zuevmaxim May 14, 2021
55b03f5
Fix test
zuevmaxim May 14, 2021
bc0b81c
Add log queue test
zuevmaxim May 14, 2021
5107665
Fix test
zuevmaxim May 14, 2021
965f69f
Fix log test
zuevmaxim May 15, 2021
e8b7ab3
Single thread crash in MC (#88)
zuevmaxim May 19, 2021
0b6c447
Update ReadWriteObjectTest.kt
ndkoval May 25, 2021
d615b8d
Limit crashes number in MC
zuevmaxim May 24, 2021
199ac54
Rewrite MCAS test
zuevmaxim May 24, 2021
a590665
Merge remote-tracking branch 'origin/develop' into test-nlr
zuevmaxim Jun 1, 2021
68e45b1
Fix merge
zuevmaxim Jun 1, 2021
4199c7f
Merge remote-tracking branch 'origin/develop' into test-nlr
zuevmaxim Jun 29, 2021
26867b0
Fix merge
zuevmaxim Jun 29, 2021
1d54853
Refactor durable verifier
zuevmaxim Jun 29, 2021
793e098
Decrease test iterations
zuevmaxim Jun 29, 2021
39f2a49
Fix
zuevmaxim Jun 29, 2021
96c08f2
Fix random flush: it can happen after any write operation
zuevmaxim Jul 5, 2021
abc4ab6
Add repository
zuevmaxim Jul 5, 2021
6f60e22
Add repository
zuevmaxim Jul 5, 2021
136e51c
Merge remote-tracking branch 'origin/develop' into test-nlr
zuevmaxim Jul 25, 2021
936a4c6
Add links
zuevmaxim Jul 25, 2021
42c17c8
Extract model checking for crashes (#103)
zuevmaxim Jul 26, 2021
65284f9
Add random choosing node
zuevmaxim Jul 5, 2021
8f4e923
Fasten simple tests
zuevmaxim Jul 26, 2021
bfce59c
Fix CrashFree mc
zuevmaxim Jul 27, 2021
36c4cea
Fix force switch on crash
zuevmaxim Jul 27, 2021
194d035
Choose random seed only in root
zuevmaxim Jul 27, 2021
567a953
Set iteration seed for custom scenarios
zuevmaxim Aug 1, 2021
591aa0b
Add relaxed queue test
zuevmaxim Aug 1, 2021
3c43d1c
Set random seeds to 16
zuevmaxim Aug 1, 2021
7c656e3
Fix test
zuevmaxim Aug 2, 2021
00a40b3
Smaller MSQueue test
zuevmaxim Aug 2, 2021
bed2644
Larger smart set
zuevmaxim Aug 2, 2021
1c79c0f
Fix relaxed queue test
zuevmaxim Aug 2, 2021
ec07109
Revert "Choose random seed only in root"
zuevmaxim Aug 2, 2021
3797e6d
Add stack test
zuevmaxim Aug 2, 2021
26822eb
Durable Stack fixed + flushInternal-s should be synchronized
ndkoval Aug 2, 2021
f678455
Fix stack test
zuevmaxim Aug 2, 2021
4fd4339
MCAS test
zuevmaxim Aug 2, 2021
90a3e2c
Fix set to NVM method
zuevmaxim Aug 2, 2021
d768daf
Fix setToNVM mc tests
zuevmaxim Aug 3, 2021
c8a4438
Add durable stack failing tests
zuevmaxim Aug 3, 2021
99d825a
Fix ms queue iterations
zuevmaxim Aug 3, 2021
40142f2
Add relaxed queue custom scenarios
zuevmaxim Aug 3, 2021
49d68b8
Revert atomic nvm api
zuevmaxim Aug 3, 2021
8ae6cc1
Fix mc crash count
zuevmaxim Aug 4, 2021
f144671
Remove bintray
zuevmaxim Aug 4, 2021
73a09b2
MC immediate system crash
zuevmaxim Aug 4, 2021
3ef583f
Refactor recover_all transformer
zuevmaxim Aug 4, 2021
7da2c12
Revert crashes in ignored section
zuevmaxim Aug 4, 2021
88f5d25
Fix immediate switch in system crash
zuevmaxim Aug 4, 2021
c994c2b
Add assertions
zuevmaxim Aug 4, 2021
ed30f0a
Fix mc random flush non-determinism
zuevmaxim Aug 4, 2021
2df37b7
Small random
zuevmaxim Aug 5, 2021
ee8bb9b
Add single word cas test
zuevmaxim Aug 5, 2021
929a923
Fix single cas test
zuevmaxim Aug 5, 2021
380d516
Add queue lock test
zuevmaxim Aug 5, 2021
6a53cb0
Replace lok free test with mc
zuevmaxim Aug 5, 2021
3d12ec4
Fix one thread crash
zuevmaxim Aug 6, 2021
27c6dac
Add RME test
zuevmaxim Aug 6, 2021
7e088fc
Rename package
zuevmaxim Aug 9, 2021
4a0fa4d
Save crashed actors into CrashResult
zuevmaxim Aug 9, 2021
8db5bd8
Prepare for buffered durable linearizability
zuevmaxim Aug 10, 2021
ce87b81
Fix HB in system crash
zuevmaxim Aug 10, 2021
e65c90a
Fix crash print
zuevmaxim Aug 10, 2021
6ac8d53
Implement buffered durable linearizability
zuevmaxim Aug 10, 2021
23384c7
Fix tests
zuevmaxim Aug 11, 2021
b48ba31
Fix verification, add tests
zuevmaxim Aug 11, 2021
91b7b71
Fix set test
zuevmaxim Aug 12, 2021
35ee8cf
Revert mc immediate crash: can crash only inside actor
zuevmaxim Aug 13, 2021
364a71a
Simplify buffered durable lin verifier
zuevmaxim Aug 13, 2021
4d52285
Fix choose random seed check
zuevmaxim Aug 13, 2021
93d403d
Fix CAS test
zuevmaxim Aug 13, 2021
ac82b7b
Fix verifier
zuevmaxim Aug 14, 2021
d756024
Fix mc: do not create crash points during system crash
zuevmaxim Aug 22, 2021
bf605f8
Fix mc: random must be changed only after a system crash
zuevmaxim Sep 1, 2021
13ca195
Revert immutable list
zuevmaxim Sep 9, 2021
94d456c
Set iterations for crash minimization
zuevmaxim Sep 10, 2021
8a08dd7
Clean up
zuevmaxim Sep 10, 2021
4e449f9
Clean up
zuevmaxim Sep 10, 2021
c676f65
Rename
zuevmaxim Sep 17, 2021
6887af2
Clean up
zuevmaxim Sep 17, 2021
dbb2706
Rename
zuevmaxim Sep 17, 2021
b82f268
Clean up
zuevmaxim Sep 17, 2021
b43c24b
Clean up
zuevmaxim Sep 17, 2021
78e5ed7
Add comments
zuevmaxim Sep 17, 2021
f171b70
Add comments
zuevmaxim Sep 19, 2021
232edd9
Add comments and clean up
zuevmaxim Sep 19, 2021
1a485d5
NRL Operation must be Recoverable
zuevmaxim Sep 22, 2021
8c5a49c
Make ExecutionScenario immutable
zuevmaxim Sep 19, 2021
35bfc4f
Get rid of global objects!!
zuevmaxim Oct 10, 2021
c10957e
Fix test
zuevmaxim Oct 10, 2021
da3cd7c
Clean up
zuevmaxim Oct 10, 2021
84b813d
Add comments
zuevmaxim Oct 10, 2021
e12ec79
Clean up
zuevmaxim Oct 10, 2021
98c2b56
Add comments
zuevmaxim Oct 11, 2021
cefdae2
Add comments
zuevmaxim Oct 11, 2021
cf12fd1
Clean up
zuevmaxim Oct 12, 2021
2d72d92
Add comments
zuevmaxim Oct 12, 2021
4aa915a
Add comments
zuevmaxim Oct 12, 2021
8fb4320
Add comments
zuevmaxim Oct 12, 2021
3ced377
Use inline class instead of ContextContainer
zuevmaxim Oct 12, 2021
f8a01cf
Move NVM tests to one package
zuevmaxim Oct 12, 2021
f93a119
Clean up
zuevmaxim Oct 12, 2021
6db33d9
Add test
zuevmaxim Oct 12, 2021
4273c0c
Clean up
zuevmaxim Oct 12, 2021
46eb485
Clean up
zuevmaxim Oct 12, 2021
61f7f2c
Make parallel tests available
zuevmaxim Oct 12, 2021
7c32f72
Primitives: remove default state
zuevmaxim Oct 13, 2021
39f7769
Reload nonVolatile methods
zuevmaxim Oct 13, 2021
1f42292
Clean up
zuevmaxim Oct 13, 2021
ac5f8ca
Clean up
zuevmaxim Oct 13, 2021
028179a
Merge remote-tracking branch 'origin/develop' into test-nlr
zuevmaxim Feb 21, 2022
ead2f97
Small ParallelTestsTest
zuevmaxim Feb 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ kotlin {
dependencies {
implementation("junit:junit:$junitVersion")
implementation("org.jctools:jctools-core:$jctoolsVersion")
implementation("org.apache.commons:commons-math3:3.5")
}
}
}
Expand All @@ -89,6 +90,7 @@ tasks {
}
withType<Test> {
maxParallelForks = 1
maxHeapSize = "4g"
jvmArgs("--add-opens", "java.base/jdk.internal.misc=ALL-UNNAMED",
"--add-exports", "java.base/jdk.internal.util=ALL-UNNAMED")
}
Expand Down
9 changes: 8 additions & 1 deletion src/jvm/main/org/jetbrains/kotlinx/lincheck/Actor.kt
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ data class Actor @JvmOverloads constructor(
val promptCancellation: Boolean = false,
// we have to specify `isSuspendable` property explicitly for transformed classes since
// `isSuspendable` implementation produces a circular dependency and, therefore, fails.
val isSuspendable: Boolean = method.isSuspendable()
val isSuspendable: Boolean = method.isSuspendable(),
// save the indices of ThreadId params to replace them later in copyWithThreadId
val threadIdArgsIndices: List<Int> = emptyList()
) {
init {
if (promptCancellation) require(cancelOnSuspension) {
Expand All @@ -57,6 +59,11 @@ data class Actor @JvmOverloads constructor(
(if (cancelOnSuspension) "cancel" else "")

val handlesExceptions = handledExceptions.isNotEmpty()

/** Returns the copy of this actor with all ThreadId params are set to [threadId]. */
fun copyWithThreadId(threadId: Int): Actor = copy(arguments = List(arguments.size) { i ->
if (i in threadIdArgsIndices) threadId else arguments[i]
})
}

fun Method.isSuspendable(): Boolean = kotlinFunction?.isSuspend ?: false
15 changes: 11 additions & 4 deletions src/jvm/main/org/jetbrains/kotlinx/lincheck/CTestConfiguration.kt
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ package org.jetbrains.kotlinx.lincheck

import org.jetbrains.kotlinx.lincheck.CTestConfiguration.Companion.DEFAULT_TIMEOUT_MS
import org.jetbrains.kotlinx.lincheck.execution.*
import org.jetbrains.kotlinx.lincheck.nvm.NoRecoverModel
import org.jetbrains.kotlinx.lincheck.nvm.RecoverabilityModel
import org.jetbrains.kotlinx.lincheck.nvm.StrategyRecoveryOptions
import org.jetbrains.kotlinx.lincheck.strategy.*
import org.jetbrains.kotlinx.lincheck.strategy.managed.*
import org.jetbrains.kotlinx.lincheck.strategy.managed.ManagedCTestConfiguration.Companion.DEFAULT_ELIMINATE_LOCAL_OBJECTS
Expand All @@ -44,13 +47,16 @@ abstract class CTestConfiguration(
val actorsBefore: Int,
val actorsAfter: Int,
val generatorClass: Class<out ExecutionGenerator>,
val verifierClass: Class<out Verifier>,
_verifierClass: Class<out Verifier>,
val requireStateEquivalenceImplCheck: Boolean,
val minimizeFailedScenario: Boolean,
val sequentialSpecification: Class<*>,
val timeoutMs: Long,
val customScenarios: List<ExecutionScenario>
val customScenarios: List<ExecutionScenario>,
val recoverabilityModel: RecoverabilityModel
) {
val verifierClass = if (recoverabilityModel is NoRecoverModel) _verifierClass else recoverabilityModel.verifierClass

abstract fun createStrategy(testClass: Class<*>, scenario: ExecutionScenario, validationFunctions: List<Method>,
stateRepresentationMethod: Method?, verifier: Verifier): Strategy
companion object {
Expand All @@ -74,7 +80,7 @@ internal fun createFromTestClassAnnotations(testClass: Class<*>): List<CTestConf
ann.generator.java, ann.verifier.java, ann.invocationsPerIteration,
ann.requireStateEquivalenceImplCheck, ann.minimizeFailedScenario,
chooseSequentialSpecification(ann.sequentialSpecification.java, testClass),
DEFAULT_TIMEOUT_MS, emptyList()
DEFAULT_TIMEOUT_MS, emptyList(), ann.recover.createModel(StrategyRecoveryOptions.STRESS)
)
}
val modelCheckingConfigurations: List<CTestConfiguration> = testClass.getAnnotationsByType(ModelCheckingCTest::class.java)
Expand All @@ -84,7 +90,8 @@ internal fun createFromTestClassAnnotations(testClass: Class<*>): List<CTestConf
ann.generator.java, ann.verifier.java, ann.checkObstructionFreedom, ann.hangingDetectionThreshold,
ann.invocationsPerIteration, ManagedCTestConfiguration.DEFAULT_GUARANTEES, ann.requireStateEquivalenceImplCheck,
ann.minimizeFailedScenario, chooseSequentialSpecification(ann.sequentialSpecification.java, testClass),
DEFAULT_TIMEOUT_MS, DEFAULT_ELIMINATE_LOCAL_OBJECTS, DEFAULT_VERBOSE_TRACE, emptyList()
DEFAULT_TIMEOUT_MS, DEFAULT_ELIMINATE_LOCAL_OBJECTS, DEFAULT_VERBOSE_TRACE, emptyList(),
ann.recover.createModel(StrategyRecoveryOptions.MANAGED)
)
}
return stressConfigurations + modelCheckingConfigurations
Expand Down
74 changes: 56 additions & 18 deletions src/jvm/main/org/jetbrains/kotlinx/lincheck/LinChecker.kt
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ package org.jetbrains.kotlinx.lincheck
import org.jetbrains.kotlinx.lincheck.annotations.*
import org.jetbrains.kotlinx.lincheck.execution.*
import org.jetbrains.kotlinx.lincheck.strategy.*
import org.jetbrains.kotlinx.lincheck.strategy.stress.StressCTestConfiguration
import org.jetbrains.kotlinx.lincheck.verifier.*
import kotlin.reflect.*

Expand Down Expand Up @@ -66,13 +67,15 @@ class LinChecker (private val testClass: Class<*>, options: Options<*, *>?) {
val exGen = createExecutionGenerator()
val verifier = createVerifier(checkStateEquivalence = true)
for (i in customScenarios.indices) {
iterationId.set(-(i + 1))
val scenario = customScenarios[i]
scenario.validate()
reporter.logIteration(i + 1, customScenarios.size, scenario)
val failure = scenario.run(this, verifier)
if (failure != null) return failure
}
repeat(iterations) { i ->
iterationId.set(i)
val scenario = exGen.nextExecution()
scenario.validate()
reporter.logIteration(i + 1 + customScenarios.size, iterations, scenario)
Expand All @@ -97,12 +100,12 @@ class LinChecker (private val testClass: Class<*>, options: Options<*, *>?) {
reporter.logScenarioMinimization(scenario)
var minimizedFailure = this
while (true) {
minimizedFailure = minimizedFailure.scenario.tryMinimize(testCfg) ?: break
minimizedFailure = minimizedFailure.scenario.tryMinimize(testCfg, minimizedFailure) ?: break
}
return minimizedFailure
}

private fun ExecutionScenario.tryMinimize(testCfg: CTestConfiguration): LincheckFailure? {
private fun ExecutionScenario.tryMinimize(testCfg: CTestConfiguration, currentFailure: LincheckFailure): LincheckFailure? {
// Reversed indices to avoid conflicts with in-loop removals
for (i in parallelExecution.indices.reversed()) {
for (j in parallelExecution[i].indices.reversed()) {
Expand All @@ -118,23 +121,55 @@ class LinChecker (private val testClass: Class<*>, options: Options<*, *>?) {
val failure = tryMinimize(threads + 1, j, testCfg)
if (failure != null) return failure
}
if (testCfg is StressCTestConfiguration
&& testCfg.recoverabilityModel.crashes
&& currentFailure is IncorrectResultsFailure)
return minimizeCrashes(testCfg, currentFailure)
return null
}

private fun ExecutionScenario.tryMinimize(threadId: Int, position: Int, testCfg: CTestConfiguration): LincheckFailure? {
val newScenario = this.copy()
val actors = newScenario[threadId] as MutableList<Actor>
actors.removeAt(position)
if (actors.isEmpty() && threadId != 0 && threadId != newScenario.threads + 1) {
// Also remove the empty thread
newScenario.parallelExecution.removeAt(threadId - 1)
}
val newScenario = this.copyWithRemovedActor(threadId, position)
return if (newScenario.isValid) {
val verifier = testCfg.createVerifier(checkStateEquivalence = false)
newScenario.run(testCfg, verifier)
newScenario.runTryMinimize(testCfg)
} else null
}

private fun ExecutionScenario.runTryMinimize(testCfg: CTestConfiguration): LincheckFailure? {
alefedor marked this conversation as resolved.
Show resolved Hide resolved
val verifier = testCfg.createVerifier(checkStateEquivalence = false)
return run(testCfg, verifier)
}

private fun IncorrectResultsFailure.crashesNumber() = results.crashes.sumBy { it.size }

/**
* Run the scenario at most [testCfg.iterations] times with maximum crashes allowed decreased to minimize the number of crashes.
*/
private fun ExecutionScenario.minimizeCrashes(testCfg: CTestConfiguration, failure: IncorrectResultsFailure): LincheckFailure? {
var scenario = this
var crashes = failure.crashesNumber()
var result: LincheckFailure? = null
var iterations = testCfg.iterations
useProxyCrash.set(false)
while (iterations > 0) {
crashesMinimization.set(crashes - 1)
val newFailure = scenario.runTryMinimize(testCfg)
if (newFailure != null
&& newFailure is IncorrectResultsFailure
&& newFailure.crashesNumber() < crashes
) {
result = newFailure
scenario = newFailure.scenario
crashes = newFailure.crashesNumber()
continue
}
iterations--
}
crashesMinimization.remove()
useProxyCrash.remove()
return result
}

private fun ExecutionScenario.run(testCfg: CTestConfiguration, verifier: Verifier): LincheckFailure? =
testCfg.createStrategy(
testClass = testClass,
Expand All @@ -144,12 +179,6 @@ class LinChecker (private val testClass: Class<*>, options: Options<*, *>?) {
verifier = verifier
).run()

private fun ExecutionScenario.copy() = ExecutionScenario(
ArrayList(initExecution),
parallelExecution.map { ArrayList(it) },
ArrayList(postExecution)
)

private val ExecutionScenario.isValid: Boolean
get() = !isParallelPartEmpty &&
(!hasSuspendableActors() || (!hasSuspendableActorsInInitPart && !hasPostPartAndSuspendableActors))
Expand Down Expand Up @@ -209,6 +238,15 @@ class LinChecker (private val testClass: Class<*>, options: Options<*, *>?) {
fun check(testClass: Class<*>, options: Options<*, *>? = null) {
LinChecker(testClass, options).check()
}

/** A test-local setting of the maximum number of crashes allowed. */
internal val crashesMinimization = ThreadLocal.withInitial<Int?> { null }

/** A test-local setting whether to use crash proxy. Proxy is not used during the minimization only. */
internal val useProxyCrash = ThreadLocal.withInitial { true }

/** A test-local storage for the current iteration number. */
internal val iterationId = ThreadLocal.withInitial { 0 }
}
}

Expand All @@ -231,4 +269,4 @@ fun <O : Options<O, *>> O.check(testClass: Class<*>) = LinChecker.check(testClas
*/
fun <O : Options<O, *>> O.check(testClass: KClass<*>) = this.check(testClass.java)

internal fun <O : Options<O, *>> O.checkImpl(testClass: Class<*>) = LinChecker(testClass, this).checkImpl()
internal fun <O : Options<O, *>> O.checkImpl(testClass: Class<*>) = LinChecker(testClass, this).checkImpl()
13 changes: 13 additions & 0 deletions src/jvm/main/org/jetbrains/kotlinx/lincheck/Options.kt
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ package org.jetbrains.kotlinx.lincheck

import org.jetbrains.kotlinx.lincheck.annotations.Operation
import org.jetbrains.kotlinx.lincheck.execution.*
import org.jetbrains.kotlinx.lincheck.nvm.Recover
import org.jetbrains.kotlinx.lincheck.verifier.*

/**
Expand All @@ -42,6 +43,7 @@ abstract class Options<OPT : Options<OPT, CTEST>, CTEST : CTestConfiguration> {
protected var sequentialSpecification: Class<*>? = null
protected var timeoutMs: Long = CTestConfiguration.DEFAULT_TIMEOUT_MS
protected var customScenarios: MutableList<ExecutionScenario> = mutableListOf()
protected var recover: Recover = Recover.NO_RECOVER

/**
* Number of different test scenarios to be executed
Expand Down Expand Up @@ -158,6 +160,17 @@ abstract class Options<OPT : Options<OPT, CTEST>, CTEST : CTestConfiguration> {
customScenarios.add(scenario)
}

/**
* Set the recovery model. [Recover.NO_RECOVER] is used by default.
* With any other recovery option enables non-volatile memory testing
* in the specified model.
*
* @see Recover
*/
fun recover(recoverModel: Recover): OPT = applyAndCast {
zuevmaxim marked this conversation as resolved.
Show resolved Hide resolved
recover = recoverModel
}

/**
* Examine the specified custom scenario additionally to the generated ones.
*/
Expand Down
35 changes: 27 additions & 8 deletions src/jvm/main/org/jetbrains/kotlinx/lincheck/Reporter.kt
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ package org.jetbrains.kotlinx.lincheck

import org.jetbrains.kotlinx.lincheck.LoggingLevel.*
import org.jetbrains.kotlinx.lincheck.execution.*
import org.jetbrains.kotlinx.lincheck.nvm.CrashError
import org.jetbrains.kotlinx.lincheck.runner.*
import org.jetbrains.kotlinx.lincheck.strategy.*
import org.jetbrains.kotlinx.lincheck.strategy.managed.*
Expand Down Expand Up @@ -186,17 +187,21 @@ private fun StringBuilder.appendDeadlockWithDumpFailure(failure: DeadlockWithDum
for ((t, stackTrace) in failure.threadDump) {
val threadNumber = if (t is FixedActiveThreadsExecutor.TestThread) t.iThread.toString() else "?"
appendLine("Thread-$threadNumber:")
stackTrace.map {
StackTraceElement(it.className.removePrefix(TransformationClassLoader.REMAPPED_PACKAGE_CANONICAL_NAME), it.methodName, it.fileName, it.lineNumber)
}.map { it.toString() }.filter { line ->
"org.jetbrains.kotlinx.lincheck.strategy" !in line
&& "org.jetbrains.kotlinx.lincheck.runner" !in line
&& "org.jetbrains.kotlinx.lincheck.UtilsKt" !in line
}.forEach { appendLine("\t$it") }
appendStackTrace(stackTrace)
}
return this
}

private fun StringBuilder.appendStackTrace(stackTrace: Array<StackTraceElement>) {
stackTrace.map {
StackTraceElement(it.className.removePrefix(TransformationClassLoader.REMAPPED_PACKAGE_CANONICAL_NAME), it.methodName, it.fileName, it.lineNumber)
}.map { it.toString() }.filter { line ->
"org.jetbrains.kotlinx.lincheck.strategy" !in line
&& "org.jetbrains.kotlinx.lincheck.runner" !in line
&& "org.jetbrains.kotlinx.lincheck.UtilsKt" !in line
}.forEach { appendLine("\t$it") }
}

private fun StringBuilder.appendIncorrectResultsFailure(failure: IncorrectResultsFailure): StringBuilder {
appendln("= Invalid execution results =")
if (failure.scenario.initExecution.isNotEmpty()) {
Expand Down Expand Up @@ -224,6 +229,16 @@ private fun StringBuilder.appendIncorrectResultsFailure(failure: IncorrectResult
if (failure.results.parallelResultsWithClock.flatten().any { !it.clockOnStart.empty })
appendln("\n---\nvalues in \"[..]\" brackets indicate the number of completed operations \n" +
"in each of the parallel threads seen at the beginning of the current operation\n---")
failure.results.crashes.forEachIndexed { threadId, threadCrashes ->
if (threadCrashes.isNotEmpty()) {
appendLine("\nCrashes in thread $threadId:")
threadCrashes.forEach { crash ->
val actor = if (crash.actorIndex == -1) "constructor" else "actor ${1 + crash.actorIndex}"
appendLine("Crashed inside $actor:")
appendCrash(crash)
}
}
}
return this
}

Expand Down Expand Up @@ -251,4 +266,8 @@ internal fun StringBuilder.appendStateEquivalenceViolationMessage(sequentialSpec
"At the current moment, `${sequentialSpecification.simpleName}` does not specify it, or the equivalence relation implementation is incorrect.\n" +
"To fix this, please implement `equals()` and `hashCode()` functions on `${sequentialSpecification.simpleName}`; the simplest way is to extend `VerifierState`\n" +
"and override the `extractState()` function, which is called at once and the result of which is used for further `equals()` and `hashCode()` invocations.")
}
}

private fun StringBuilder.appendCrash(crash: CrashError) {
appendStackTrace(crash.crashStackTrace)
}
22 changes: 21 additions & 1 deletion src/jvm/main/org/jetbrains/kotlinx/lincheck/Result.kt
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package org.jetbrains.kotlinx.lincheck

import org.jetbrains.kotlinx.lincheck.nvm.NVMStateHolder
import java.io.Serializable
import kotlin.coroutines.*

Expand Down Expand Up @@ -111,6 +112,7 @@ data class ExceptionResult private constructor(val tClazz: Class<out Throwable>,
fun create(tClazz: Class<out Throwable>, wasSuspended: Boolean = false) = ExceptionResult(tClazz.normalize(), wasSuspended)
}
}

// for byte-code generation
@JvmSynthetic
fun createExceptionResult(tClazz: Class<out Throwable>) = ExceptionResult.create(tClazz, false)
Expand Down Expand Up @@ -138,4 +140,22 @@ internal data class ResumedResult(val contWithSuspensionPointRes: Pair<Continuat

lateinit var resumedActor: Actor
lateinit var by: Actor
}
}


class CrashResult : Result() {
lateinit var crashedActors: IntArray
override val wasSuspended get() = false
override fun toString() = "CRASH${crashedActors.joinToString(",", "(", ")") { s -> if (s == -1) "-" else s.toString() }}"

override fun hashCode() = crashedActors.contentHashCode()
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (other !is CrashResult) return false
return crashedActors.contentEquals(other.crashedActors)
}
}

// for byte-code generation
@JvmSynthetic
fun createCrashResult() = CrashResult()
Loading