johnwalker / test.check

QuickCheck for Clojure

test.check

test.check is a Clojure property-based testing tool inspired by QuickCheck. The core idea of test.check is that instead of enumerating expected input and output for unit tests, you write properties about your function that should hold true for all inputs. This lets you write concise, powerful tests.

test.check used to be called simple-check.

Releases and Dependency Information

Leiningen

[org.clojure/test.check "0.5.9"]

Maven

<dependency>
  <groupId>org.clojure</groupId>
  <artifactId>test.check</artifactId>
  <version>0.5.9</version>
</dependency>

If you'd like to try a SNAPSHOT version, add the sonatype repository to your project.

Version numbers

test.check version numbers start where simple-check left off: 0.5.7.

Documentation

• Generator writing guide
• Examples (some of these may refer to simple-check):
  • core.matrix
  • byte-streams
  • byte-transforms
  • collection-check
• Blog posts and videos (some of these may refer to simple-check):
  • Powerful Testing with test.check - Clojure/West -- Slides
  • Check your work - 8th Light
  • Writing simple-check - Reid Draper
  • Generative testing in Clojure - Youtube
  • Using simple-check with Expectations - Curtis Gagliardi

Migrating from simple-check

In order to migrate from simple-check to test.check, you'll need to do two things:

• Update project.clj

  In your project.clj replace [reiddraper/simple-check "0.5.6"] with [org.clojure/test.check "0.5.9"] (note: your version numbers may be different).

• Update namespace declarations

  Update your namespaces: simple-check.core becomes clojure.test.check (note the dropping of 'core'). Everything else you can simply replace simple-check with clojure.test.check. Let's make it easy:

  find test -name '*.clj' -print0 | xargs -0 sed -i.bak \
  -e 's/simple-check.core/clojure.test.check/' \
  -e 's/simple-check/clojure.test.check/'

  Review the updates.

Examples

Let's say we're testing a sort function. We want to check that that our sort function is idempotent, that is, applying sort twice should be equivalent to applying it once: (= (sort a) (sort (sort a))). Let's write a quick test to make sure this is the case:

(require '[clojure.test.check :as tc])
(require '[clojure.test.check.generators :as gen])
(require '[clojure.test.check.properties :as prop])

(def sort-idempotent-prop
  (prop/for-all [v (gen/vector gen/int)]
    (= (sort v) (sort (sort v)))))

(tc/quick-check 100 sort-idempotent-prop)
;; => {:result true, :num-tests 100, :seed 1382488326530}

In prose, this test reads: for all vectors of integers, v, sorting v is equal to sorting v twice.

What happens if our test fails? test.check will try and find 'smaller' input that still fails. This process is called shrinking. Let's see it in action:

(def prop-sorted-first-less-than-last
  (prop/for-all [v (gen/not-empty (gen/vector gen/int))]
    (let [s (sort v)]
      (< (first s) (last s)))))

(tc/quick-check 100 prop-sorted-first-less-than-last)
;; => {:result false, :failing-size 0, :num-tests 1, :fail [[3]],
       :shrunk {:total-nodes-visited 5, :depth 2, :result false,
                :smallest [[0]]}}

This test claims that the first element of a sorted vector should be less-than the last. Of course, this isn't true: the test fails with input [3], which gets shrunk down to [0], as seen in the output above. As your test functions require more sophisticated input, shrinking becomes critical to being able to understand exactly why a random test failed. To see how powerful shrinking is, let's come up with a contrived example: a function that fails if its passed a sequence that contains the number 42:

(def prop-no-42
  (prop/for-all [v (gen/vector gen/int)]
    (not (some #{42} v))))

(tc/quick-check 100 prop-no-42)
;; => {:result false,
       :failing-size 45,
       :num-tests 46,
       :fail [[10 1 28 40 11 -33 42 -42 39 -13 13 -44 -36 11 27 -42 4 21 -39]],
       :shrunk {:total-nodes-visited 38,
                :depth 18,
                :result false,
                :smallest [[42]]}}

We see that the test failed on a rather large vector, as seen in the :fail key. But then test.check was able to shrink the input down to [42], as seen in the keys [:shrunk :smallest].

To learn more, check out the documentation links.

clojure.test Integration

The macro clojure.test.check.clojure-test/defspec allows you to succinctly write properties that run under the clojure.test runner, for example:

(defspec first-element-is-min-after-sorting ;; the name of the test
         100 ;; the number of iterations for test.check to test
         (prop/for-all [v (such-that not-empty (gen/vector gen/int))]
           (= (apply min v)
              (first (sorted v)))))

Release Notes

Release notes for each version are available in CHANGELOG.markdown. Remember that prior to version 0.5.7, test.check was called simple-check.

See also...

Other implementations

• QC for Haskell
• The significantly more advanced QC for Erlang

Papers

• QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs

Contributing

We can not accept pull requests. Please see CONTRIBUTING.md for details.

License

Copyright © 2014 Rich Hickey, Reid Draper and contributors

Distributed under the Eclipse Public License, the same as Clojure.