See also ✨
elm-microthesis
✨ (and
its various branches/tags) which is more Elm-idiomatic implementation of the
same ideas. The repo you are currently viewing is a more direct port of the
Python Minithesis idea. Read below.
elm-minithesis
is a property-based testing library based on Minithesis, which is the minimal implementation of the core idea of Hypothesis.
Read more in the About section.
import Minithesis.Fuzz as Fuzz exposing (Fuzzer)
import Minithesis exposing (Test, TestResult)
ints : Fuzzer (List Int)
ints =
Fuzz.list (Fuzz.int 0 10000)
findsSmallList : Test (List Int)
findsSmallList =
Minithesis.test "list always sums under 1000 lol" ints <|
\fuzzedList ->
List.sum fuzzedList <= 1000
{-| Will fail and shrink to the minimal example:
`( "list always sums under 1000 lol", FailsWith [ 1001 ] )`
-}
result : Int -> TestResult (List Int)
result seed =
Minithesis.run seed findsSmallList
{-| Running these tests inside elm-test can be done via functions inside
the Test.Minithesis module
-}
test : Test.Test
test =
Test.Minithesis.mFuzz findsSmallList
Try Fuzz.example
and Fuzz.exampleWithSeed
in the REPL for quick
sanity checks of your fuzzers!
import Minithesis.Fuzz as F
F.string |> F.exampleWithSeed 0
--> gives 10 examples
["x","I","","6a=U",";W?","uDc",":ei_^~","=Y","-NAT\\QJ","{92H2DI}-(KOc"]
F.string |> F.exampleWithSeed 1
--> different seed -> different examples
["KI","<j XT","","'xpvdQ1ONkM/","tdVd_v","I3=:e0i3","","P)y8$e@^y}1s",",]uz\\","8"]
Use the showShrinkHistory
field of Minithesis.runWith
to get additional
information about how shrinking of your data went. All FailsWith
results
become FailsWithShrinks
containing additional info. This gives a bit of
visibility into what happens in the black box that Minithesis shrinking is.
import Minithesis as M
import Minithesis.Fuzz as F
M.runWith
{ maxExamples = 100
, showShrinkHistory = True
}
1
(M.test "list always sums under 1000 lol"
(F.list (F.int 0 10000))
(\list -> List.sum list <= 1000)
)
-->
( "list always sums under 1000 lol"
, FailsWithShrinks
{ finalRun = [1,1001,0]
, finalValue = [1001]
, history =
[ value = [166,5536,4725,8499,7844,1727], { run = [1,166,1,5536,1,4725,1,8499,1,7844,1,1727,0], shrinkerUsed = "Initial" }
, value = [166,5536], { run = [1,166,1,5536,0], shrinkerUsed = "DeleteChunkAndMaybeDecrementPrevious { size = 8, startIndex = 5 }" }
, value = [5536], { run = [1,5536,0], shrinkerUsed = "DeleteChunkAndMaybeDecrementPrevious { size = 2, startIndex = 1 }" }
, value = [1001], { run = [1,1001,0], shrinkerUsed = "MinimizeChoiceWithBinarySearch { index = 1 }" }
]
}
)
Paired with some knowledge about which shrinking strategies there are and what they do, you can sometimes tweak your fuzzers to optimize how they interact with the shrinking process, allowing them to be shrunk better.
(Related: Hypothesis docs: "Strategies that shrink")
elm-minithesis
is a property-based testing library based on Minithesis, which is the minimal implementation of the core idea of Hypothesis.
Hypothesis itself is a Python testing library for property-based testing. What
sets it apart is its underlying implementation: instead of working on the
generated values themselves (defining shrinkers on these values, eg. saying that
a Bool will shrink from True
to [False]
and from False
to []
), it
remembers the underlying random "dice rolls" that were used to generate the
values, and it shrinks those.
-- "type-based shrinking"
-- (QuickCheck and most other property-based testing libraries, including elm-test)
shrink : a -> LazyList a -- has to be defined for each fuzzed type,
-- doesn't shrink by default
-- "integrated shrinking"
-- (Hypothesis, jqwik, elm-minithesis :) )
shrink : List Int -> LazyList (List Int) -- shrinks all fuzzers automatically,
-- can't be configured
A very cool consequence of the above is is that it mostly sidesteps the issue
most other property-based testing libraries have: andThen
(monadic bind). More
specifically, QuickCheck-like libraries either don't expose andThen
at all (as
is the case with Elm), or struggle with making the shrunk values satisfy the
same invariants the andThen
-generated values do. Hypothesis instead shrinks
the underlying "dice roll" history and generates a new value from that, so
its values satisfy the invariants out of the box even if using andThen
!
(Source: Hypothesis blog: "Integrated vs type based shrinking")
There is a small piece of internet dedicated to elm-minithesis
: the
#elm-minithesis
channel on Incremental Elm
Discord. Come join and hear about updates first!
The original discussion around elm-minithesis
happened on the Elm
Discourse.