Plain data Schemas for Clojure/Script.
STATUS: Pre-alpha, in design and prototyping phase.
- Schemas as plain data
- Schema-driven Validation
- Schema-driven Value Transformation
- Generating values from Schemas
- Inferring Schemas from sample values
- Tools for programming with Schemas
- First class error-messages and spell checking
- Schema Transformations to JSON Schema and Swagger2
- Multi-schemas, Recursive Schemas and Default values
- Persisting schemas and the alternative Map-syntax
- Immutable, Mutable, Dynamic and Local Schema Registries
- Fast
Presentations:
- ClojureD 2020: Malli: Inside Data-driven Schemas, slides here
- CEST 2.6.2020: Data-driven Rapid Application Development with Malli
Try the online demo. Libraries using malli:
- Aave, a code checking tool for Clojure.
Definining and validating Schemas:
(require '[malli.core :as m])
(m/validate int? "1")
; => false
(m/validate int? 1)
; => true
(m/validate [:and int? [:> 6]] 7)
; => true
(def valid?
(m/validator
[:map
[:x boolean?]
[:y {:optional true} int?]
[:z string?]]))
(valid? {:x true, :z "kikka"})
; => true
Schemas can have properties:
(def Age
[:and
{:title "Age"
:description "It's an age"
:json-schema/example 20}
int? [:> 18]])
(m/properties Age)
; => {:title "Age"
; :description "It's an age"
; :json-schema/example 20}
Maps are open by default:
(m/validate
[:map [:x int?]]
{:x 1, :extra "key"})
; => true
Maps can be closed with :closed
property:
(m/validate
[:map {:closed true} [:x int?]]
{:x 1, :extra "key"})
; => false
Maps keys are not limited to keywords:
(m/validate
[:map
["status" [:enum "ok"]]
[1 any?]
[nil any?]
[::a string?]
[[1 2 3] number?]]
{"status" "ok"
1 'number
nil :yay
::a "properly awesome"
[1 2 3] 1})
; => true
Using a predicate:
(m/validate string? "kikka")
Using :string
Schema:
(m/validate :string "kikka")
; => true
(m/validate [:string {:min 1, :max 4}] "")
; => false
:fn
allows any predicat function to be used:
(def my-schema
[:and
[:map
[:x int?]
[:y int?]]
[:fn (fn [{:keys [x y]}] (> x y))]])
(m/validate my-schema {:x 1, :y 0})
; => true
(m/validate my-schema {:x 1, :y 2})
; => false
Serializable function schemas using sci:
(def my-schema
[:and
[:map
[:x int?]
[:y int?]]
[:fn '(fn [{:keys [x y]}] (> x y))]])
(m/validate my-schema {:x 1, :y 0})
; => true
(m/validate my-schema {:x 1, :y 2})
; => false
Detailed errors with m/explain
:
(def Address
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:map
[:street string?]
[:city string?]
[:zip int?]
[:lonlat [:tuple double? double?]]]]])
(m/explain
Address
{:id "Lillan"
:tags #{:artesan :coffee :hotel}
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322, 23.7854658]}})
; => nil
(m/explain
Address
{:id "Lillan"
:tags #{:artesan "coffee" :garden}
:address {:street "Ahlmanintie 29"
:zip 33100
:lonlat [61.4858322, nil]}})
;{:schema [:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address [:map
; [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]]]],
; :value {:id "Lillan",
; :tags #{:artesan :garden "coffee"},
; :address {:street "Ahlmanintie 29"
; :zip 33100
; :lonlat [61.4858322 nil]}},
; :errors (#Error{:path [2 1 1], :in [:tags 0], :schema keyword?, :value "coffee"}
; #Error{:path [3 1],
; :in [:address],
; :schema [:map
; [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]],
; :type :malli.core/missing-key,
; :malli.core/key :city}
; #Error{:path [3 1 4 1 2], :in [:address :lonlat 1], :schema double?, :value nil})}
Explain results can be humanized with malli.error/humanize
:
(require '[malli.error :as me])
(-> Address
(m/explain
{:id "Lillan"
:tags #{:artesan "coffee" :garden}
:address {:street "Ahlmanintie 29"
:zip 33100
:lonlat [61.4858322, nil]}})
(me/humanize))
;{:tags #{["should be keyword"]}
; :address {:city ["missing required key"]
; :lonlat [nil ["should be double"]]}}
Error messages can be customized with :error/message
and :error/fn
properties:
(-> [:map
[:id int?]
[:size [:enum {:error/message "should be: S|M|L"}
"S" "M" "L"]]
[:age [:fn {:error/fn '(fn [{:keys [value]} _] (str value ", should be > 18"))}
'(fn [x] (and (int? x) (> x 18)))]]]
(m/explain {:size "XL", :age 10})
(me/humanize
{:errors (-> me/default-errors
(assoc ::m/missing-key {:error/fn (fn [{:keys [in]} _] (str "missing key " (last in)))}))}))
;{:id ["missing key :id"]
; :size ["should be: S|M|L"]
; :age ["10, should be > 18"]}
Messages can be localized:
(-> [:map
[:id int?]
[:size [:enum {:error/message {:en "should be: S|M|L"
:fi "pitäisi olla: S|M|L"}}
"S" "M" "L"]]
[:age [:fn {:error/fn {:en '(fn [{:keys [value]} _] (str value ", should be > 18"))
:fi '(fn [{:keys [value]} _] (str value ", pitäisi olla > 18"))}}
'(fn [x] (and (int? x) (> x 18)))]]]
(m/explain {:size "XL", :age 10})
(me/humanize
{:locale :fi
:errors (-> me/default-errors
(assoc-in ['int? :error-message :fi] "pitäisi olla numero")
(assoc ::m/missing-key {:error/fn {:en '(fn [{:keys [in]} _] (str "missing key " (last in)))
:fi '(fn [{:keys [in]} _] (str "puuttuu avain " (last in)))}}))}))
;{:id ["puuttuu avain :id"]
; :size ["pitäisi olla: S|M|L"]
; :age ["10, pitäisi olla > 18"]}
Top-level humanized map-errors are under :malli/error
:
(-> [:and [:map
[:password string?]
[:password2 string?]]
[:fn {:error/message "passwords don't match"}
'(fn [{:keys [password password2]}]
(= password password2))]]
(m/explain {:password "secret"
:password2 "faarao"})
(me/humanize))
; {:malli/error ["passwords don't match"]}
Errors can be targetted using :error/path
property:
(-> [:and [:map
[:password string?]
[:password2 string?]]
[:fn {:error/message "passwords don't match"
:error/path [:password2]}
'(fn [{:keys [password password2]}]
(= password password2))]]
(m/explain {:password "secret"
:password2 "faarao"})
(me/humanize))
; {:password2 ["passwords don't match"]}
For closed schemas, key spelling can be checked with:
(-> [:map [:address [:map [:street string?]]]]
(mu/closed-schema)
(m/explain
{:name "Lie-mi"
:address {:streetz "Hämeenkatu 14"}})
(me/with-spell-checking)
(me/humanize))
;{:address {:street ["missing required key"]
; :streetz ["should be spelled :street"]}
; :name ["disallowed key"]}
(require '[malli.transform :as mt])
Two-way schema-driven value transformations with m/decode
and m/encode
using a m/Transformer
.
Default Transformers include: string-transformer
, json-transformer
, strip-extra-keys-transformer
, default-value-transformer
and key-transformer
.
(m/decode int? "42" mt/string-transformer)
; 42
(m/encode int? 42 mt/string-transformer)
; "42"
Transformations are recursive:
(m/decode
Address
{:id "Lillan",
:tags ["coffee" "artesan" "garden"],
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322 23.7854658]}}
mt/json-transformer)
;{:id "Lillan",
; :tags #{:coffee :artesan :garden},
; :address {:street "Ahlmanintie 29"
; :city "Tampere"
; :zip 33100
; :lonlat [61.4858322 23.7854658]}}
Transform map keys:
(m/decode
Address
{:id "Lillan",
:tags ["coffee" "artesan" "garden"],
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322 23.7854658]}}
(mt/key-transformer {:decode name}))
;{"id" "Lillan",
; "tags" #{:coffee :artesan :garden},
; "address" {"street" "Ahlmanintie 29"
; "city" "Tampere"
; "zip" 33100
; "lonlat" [61.4858322 23.7854658]}}
Transformers can be composed with mt/transformer
:
(def strict-json-transformer
(mt/transformer
mt/strip-extra-keys-transformer
mt/json-transformer)
(m/decode
Address
{:id "Lillan",
:EVIL "LYN"
:tags ["coffee" "artesan" "garden"],
:address {:street "Ahlmanintie 29"
:DARK "ORKO"
:city "Tampere"
:zip 33100
:lonlat [61.4858322 23.7854658]}}
strict-json-transformer)
;{:id "Lillan",
; :tags #{:coffee :artesan :garden},
; :address {:street "Ahlmanintie 29"
; :city "Tampere"
; :zip 33100
; :lonlat [61.4858322 23.7854658]}}
Schema properties can be used to override default transformations:
(m/decode
[string? {:decode/string 'str/upper-case}]
"kerran" mt/string-transformer)
; => "KERRAN"
Decoders and encoders as interceptors (with :enter
and :leave
stages):
(m/decode
[string? {:decode/string {:enter 'str/upper-case}}]
"kerran" mt/string-transformer)
; => "KERRAN"
(m/decode
[string? {:decode/string {:enter '#(str "olipa_" %)
:leave '#(str % "_avaruus")}}]
"kerran" mt/string-transformer)
; => "olipa_kerran_avaruus"
To access Schema (and options) use :compile
:
(m/decode
[int? {:math/multiplier 10
:decode/math {:compile '(fn [schema _]
(let [multiplier (:math/multiplier (m/properties schema))]
(fn [x] (* x multiplier))))}}]
12
(mt/transformer {:name :math}))
; => 120
Going crazy:
(m/decode
[:map
{:decode/math {:enter '#(update % :x inc)
:leave '#(update % :x (partial * 2))}}
[:x [int? {:decode/math {:enter '(partial + 2)
:leave '(partial * 3)}}]]]
{:x 1}
(mt/transformer {:name :math}))
; => {:x 42}
Applying default values:
(m/decode [:and {:default 42} int?] nil mt/default-value-transformer)
; => 42
Single sweep of defaults & string encoding:
(m/encode
[:map {:default {}}
[:a [int? {:default 1}]]
[:b [:vector {:default [1 2 3]} int?]]
[:c [:map {:default {}}
[:x [int? {:default 42}]]
[:y int?]]]
[:d [:map
[:x [int? {:default 42}]]
[:y int?]]]
[:e int?]]
nil
(mt/transformer
mt/default-value-transformer
mt/string-transformer))
;{:a "1"
; :b ["1" "2" "3"]
; :c {:x "42"}}
(require '[malli.util :as mu])
Updating Schema properties:
(mu/update-properties [:vector int?] assoc :min 1)
; => [:vector {:min 1} int?]
Lifted clojure.core
function to work with schemas: select-keys
, dissoc
, get
, assoc
, update
, get-in
, assoc-in
, update-in
(mu/get-in Address [:address :lonlat])
; => [:tuple double? double?]
(mu/update-in Address [:address] mu/assoc :country [:enum "fi" "po"])
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]
; [:country [:enum "fi" "po"]]]]]
(-> Address
(mu/dissoc :address)
(mu/update-properties assoc :title "Address"))
;[:map {:title "Address"}
; [:id string?]
; [:tags [:set keyword?]]]
Making keys optional or required:
(mu/optional-keys [:map [:x int?] [:y int?]])
;[:map
; [:x {:optional true} int?]
; [:y {:optional true} int?]]
(mu/required-keys [:map [:x {:optional true} int?] [:y int?]])
;[:map
; [:x int?]
; [:y int?]]
Closing and opening all :map
schemas recursively:
(def abcd
[:map {:title "abcd"}
[:a int?]
[:b {:optional true} int?]
[:c [:map
[:d int?]]]])
(mu/closed-schema abcd)
;[:map {:title "abcd", :closed true}
; [:a int?]
; [:b {:optional true} int?]
; [:c [:map {:closed true}
; [:d int?]]]]
(-> abcd
mu/closed-schema
mu/open-schema)
;[:map {:title "abcd"}
; [:a int?]
; [:b {:optional true} int?]
; [:c [:map
; [:d int?]]]]
Merging Schemas (last value wins):
(mu/merge
[:map
[:name string?]
[:description string?]
[:address
[:map
[:street string?]
[:country [:enum "finland" "poland"]]]]]
[:map
[:description {:optional true} string?]
[:address
[:map
[:country string?]]]])
;[:map
; [:name string?]
; [:description {:optional true} string?]
; [:address [:map
; [:street string?]
; [:country string?]]]]
Schema unions (merged values of both schemas are valid for union schema):
(mu/union
[:map
[:name string?]
[:description string?]
[:address
[:map
[:street string?]
[:country [:enum "finland" "poland"]]]]]
[:map
[:description {:optional true} string?]
[:address
[:map
[:country string?]]]])
;[:map
; [:name string?]
; [:description {:optional true} string?]
; [:address [:map
; [:street string?]
; [:country [:or [:enum "finland" "poland"] string?]]]]]
Adding generated example values to Schemas:
(m/accept
[:map
[:name string?]
[:description string?]
[:address
[:map
[:street string?]
[:country [:enum "finland" "poland"]]]]]
(m/schema-visitor
(fn [schema]
(mu/update-properties schema assoc :examples (mg/sample schema {:size 2, :seed 20})))))
;[:map
; {:examples ({:name "", :description "", :address {:street "", :country "poland"}}
; {:name "W", :description "x", :address {:street "8", :country "finland"}})}
; [:name [string? {:examples ("" "")}]]
; [:description [string? {:examples ("" "")}]]
; [:address
; [:map
; {:examples ({:street "", :country "finland"} {:street "W", :country "poland"})}
; [:street [string? {:examples ("" "")}]]
; [:country [:enum {:examples ("finland" "poland")} "finland" "poland"]]]]]
Writing and Reading schemas as EDN, no eval
needed.
(require '[malli.edn :as edn])
(-> [:and
[:map
[:x int?]
[:y int?]]
[:fn '(fn [{:keys [x y]}] (> x y))]]
(edn/write-string)
(doto prn) ; => "[:and [:map [:x int?] [:y int?]] [:fn (fn [{:keys [x y]}] (> x y))]]"
(edn/read-string)
(doto (-> (m/validate {:x 0, :y 1}) prn)) ; => false
(doto (-> (m/validate {:x 2, :y 1}) prn))) ; => true
;[:and
; [:map
; [:x int?]
; [:y int?]]
; [:fn (fn [{:keys [x y]}] (> x y))]]
Closed dispatch with :multi
schema and :dispatch
property:
(m/validate
[:multi {:dispatch :type}
[:sized [:map [:type keyword?] [:size int?]]]
[:human [:map [:type keyword?] [:name string?] [:address [:map [:country keyword?]]]]]]
{:type :sized, :size 10})
; true
Any (serializable) function can be used for :dispatch
:
(m/validate
[:multi {:dispatch 'first}
[:sized [:tuple keyword? [:map [:size int?]]]]
[:human [:tuple keyword? [:map [:name string?] [:address [:map [:country keyword?]]]]]]]
[:human {:name "seppo", :address {:country :sweden}}])
; true
:dispatch
values should be decoded before actual values:
(m/decode
[:multi {:dispatch :type
:decode/string '#(update % :type keyword)}
[:sized [:map [:type [:= :sized] [:size int?]]]
[:human [:map [:type [:= :human]] [:name string?] [:address [:map [:country keyword?]]]]]]
{:type "human"
:name "Tiina"
:age "98"
:address {:country "finland"
:street "this is an extra key"}}
(mt/transformer mt/strip-extra-keys-transformer mt/string-transformer))
;{:type :human
; :name "Tiina"
; :address {:country :finland}}
Local Registy allows an easy way to create recursive schemas:
(m/validate
[:schema {:registry {::cons [:maybe [:tuple pos-int? [:ref ::cons]]]}}
::cons]
[16 [64 [26 [1 [13 nil]]]]])
; => true
Mutual recursion works too:
(m/validate
[:schema {:registry {::ping [:maybe [:tuple [:= "ping"] [:ref ::pong]]]
::pong [:maybe [:tuple [:= "pong"] [:ref ::ping]]]}}
::ping]
["ping" ["pong" ["ping" ["pong" ["ping" nil]]]]])
; => true
Nested registries, last definition wins:
(m/validate
[:schema {:registry {::ping [:maybe [:tuple [:= "ping"] [:ref ::pong]]]
::pong any?}} ;; effectively unreachable
[:schema {:registry {::pong [:maybe [:tuple [:= "pong"] [:ref ::ping]]]}}
::ping]]
["ping" ["pong" ["ping" ["pong" ["ping" nil]]]]])
; => true
Schemas can be used to generate values:
(require '[malli.generator :as mg])
;; random
(mg/generate keyword?)
; => :?
;; using seed
(mg/generate [:enum "a" "b" "c"] {:seed 42})
;; => "a"
;; using seed and size
(mg/generate pos-int? {:seed 10, :size 100})
;; => 55740
;; regexs work too
(mg/generate
[:re #"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,63}$"]
{:seed 42, :size 10})
; => "[email protected]"
;; gen/elements (note, are not validated)
(mg/generate
[:and {:gen/elements ["kikka" "kukka" "kakka"]} string?]
{:seed 10})
; => "kikka"
;; portable gen/fmap
(mg/generate
[:and {:gen/fmap '(partial str "kikka_")} string?]
{:seed 10, :size 10})
;; => "kikka_WT3K0yax2"
(require '[clojure.test.check.generators :as gen])
;; gen/gen (note, not serializable)
(mg/generate
[:sequential {:gen/gen (gen/list gen/neg-int)} int?]
{:size 42, :seed 42})
; => (-37 -13 -13 -24 -20 -11 -34 -40 -22 0 -10)
Generated values are valid:
(mg/generate Address {:seed 123, :size 4})
;{:id "H7",
; :tags #{:v?.w.t6!.QJYk-/-?s*4
; :_7U
; :QdG/Xi8J
; :*Q-.p*8*/n-J9u}
; :address {:street "V9s"
; :city ""
; :zip 3
; :lonlat [-2.75 -0.625]}}
(m/validate Address (mg/generate Address))
; => true
Sampling values:
;; sampling
(mg/sample [:and int? [:> 10] [:< 100]] {:seed 123})
; => (25 39 51 13 53 43 57 15 26 27)
Inspired by F# Type providers:
(require '[malli.provider :as mp])
(def samples
[{:id "Lillan"
:tags #{:artesan :coffee :hotel}
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322, 23.7854658]}}
{:id "Huber",
:description "Beefy place"
:tags #{:beef :wine :beer}
:address {:street "Aleksis Kiven katu 13"
:city "Tampere"
:zip 33200
:lonlat [61.4963599 23.7604916]}}])
(mp/provide samples)
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map
; [:street string?]
; [:city string?]
; [:zip number?]
; [:lonlat [:vector double?]]]]
; [:description {:optional true} string?]]
All samples are valid against the inferred schema:
(every? (partial m/validate (mp/provide samples)) samples)
; => true
Schemas can converted into map-syntax (with keys :type
and optionally :properties
and :children
):
(def Schema
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:map
[:street string?]
[:lonlat [:tuple double? double?]]]]])
(m/to-map-syntax Schema)
;{:type :map,
; :children [[:id nil {:type string?}]
; [:tags nil {:type :set
; :children [{:type keyword?}]}]
; [:address nil {:type :map,
; :children [[:street nil {:type string?}]
; [:lonlat nil {:type :tuple
; :children [{:type double?} {:type double?}]}]]}]]}
... and back:
(-> Schema (m/to-map-syntax) (m/from-map-syntax) (mu/equals Schema))
; => true
Schemas can be transformed using the Visitor Pattern.
The identity visitor:
(m/accept
Address
(m/schema-visitor identity))
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map
; [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]]]]
Adding :title
property to schemas:
(m/accept
Address
(m/schema-visitor #(mu/update-properties % assoc :title (name (m/type %)))))
;[:map {:title "map"}
; [:id [string? {:title "string?"}]]
; [:tags [:set {:title "set"} [keyword? {:title "keyword?"}]]]
; [:address
; [:map {:title "map"}
; [:street [string? {:title "string?"}]]
; [:city [string? {:title "string?"}]]
; [:zip [int? {:title "int?"}]]
; [:lonlat [:tuple {:title "tuple"} [double? {:title "double?"}] [double? {:title "double?"}]]]]]]
Adding :path
property to schemas:
(m/accept
Address
(fn [schema children in options]
(m/into-schema
(m/type schema)
(cond-> (m/properties schema) (seq in) (assoc :path in))
children options)))
;[:map [:id [string? {:path [:id]}]]
; [:tags [:set {:path [:tags]}
; [keyword? {:path [:tags :malli.core/in]}]]]
; [:address
; [:map {:path [:address]}
; [:street [string? {:path [:address :street]}]]
; [:city [string? {:path [:address :city]}]]
; [:zip [int? {:path [:address :zip]}]]
; [:lonlat
; [:tuple {:path [:address :lonlat]}
; [double? {:path [:address :lonlat 0]}]
; [double? {:path [:address :lonlat 1]}]]]]]]
Transforming schemas into nested maps:
(m/accept
Address
(fn [schema children _ _]
(-> (m/properties schema)
(assoc :malli/type (m/type schema))
(cond-> (seq children) (assoc :malli/children children)))))
;{:malli/type :map,
; :malli/children [[:id nil {:malli/type string?}]
; [:tags nil {:malli/type :set
; :malli/children [{:malli/type keyword?}]}]
; [:address nil {:malli/type :map,
; :malli/children [[:street nil {:malli/type string?}]
; [:city nil {:malli/type string?}]
; [:zip nil {:malli/type int?}]
; [:lonlat nil {:malli/type :tuple
; :malli/children [{:malli/type double?}
; {:malli/type double?}]}]]}]]}
Transforming Schemas into JSON Schema:
(require '[malli.json-schema :as json-schema])
(json-schema/transform Address)
;{:type "object",
; :properties {:id {:type "string"},
; :tags {:type "array"
; :items {:type "string"}
; :uniqueItems true},
; :address {:type "object",
; :properties {:street {:type "string"},
; :city {:type "string"},
; :zip {:type "integer", :format "int64"},
; :lonlat {:type "array",
; :items [{:type "number"} {:type "number"}],
; :additionalItems false}},
; :required [:street :city :zip :lonlat]}},
; :required [:id :tags :address]}
Custom transformation via :json-schema
namespaced properties:
(json-schema/transform
[:enum
{:title "Fish"
:description "It's a fish"
:json-schema/type "string"
:json-schema/default "perch"}
"perch" "pike"])
;{:title "Fish"
; :description "It's a fish"
; :type "string"
; :default "perch"
; :enum ["perch" "pike"]}
Full override with :json-schema
property:
(json-schema/transform
[:map {:json-schema {:type "file"}}
[:file any?]])
; {:type "file"}
Transforming Schemas into Swagger2 Schema:
(require '[malli.swagger :as swagger])
(swagger/transform Address)
;{:type "object",
; :properties {:id {:type "string"},
; :tags {:type "array"
; :items {:type "string"}
; :uniqueItems true},
; :address {:type "object",
; :properties {:street {:type "string"},
; :city {:type "string"},
; :zip {:type "integer", :format "int64"},
; :lonlat {:type "array",
; :items {},
; :x-items [{:type "number", :format "double"}
; {:type "number", :format "double"}]}},
; :required [:street :city :zip :lonlat]}},
; :required [:id :tags :address]}
Custom transformation via :swagger
and :json-schema
namespaced properties:
(swagger/transform
[:enum
{:title "Fish"
:description "It's a fish"
:swagger/type "string"
:json-schema/default "perch"}
"perch" "pike"])
;{:title "Fish"
; :description "It's a fish"
; :type "string"
; :default "perch"
; :enum ["perch" "pike"]}
Full override with :swagger
property:
(swagger/transform
[:map {:swagger {:type "file"}}
[:file any?]])
; {:type "file"}
Validation:
(require '[clojure.spec.alpha :as s])
(require '[criterium.core :as cc])
;; 40ns
(let [spec (s/and int? (s/or :pos-int pos-int? :neg-int neg-int?))
valid? (partial s/valid? spec)]
(cc/quick-bench
(valid? spec 0)))
;; 5ns
(let [valid? (m/validator [:and int? [:or pos-int? neg-int?]])]
(cc/quick-bench
(valid? 0)))
Coercion:
(require '[spec-tools.core :as st])
(s/def ::id int?)
(s/def ::name string?)
;; 14µs
(let [spec (s/keys :req-un [::id ::name])
transform #(st/coerce spec % st/string-transformer)]
(cc/quick-bench
(transform {:id "1", :name "kikka"})))
;; 140ns
(let [schema [:map [:id int?] [:name string?]]
transform (m/decoder schema transform/string-transformer)]
(cc/quick-bench
(transform {:id "1", :name "kikka"})))
Schemas are looked up using a malli.registry/Registry
protocol, which is effectively a map from schema type
to a schema recipe (schema ast, Schema
or IntoSchema
instance).
Custom Registry
can be passed in to all/most malli public apis via the optional options map using :registry
key. If omitted, malli.core/default-registry
is used.
;; the default registry
(m/validate [:maybe string?] "kikka")
; => true
;; registry as explicit options
(m/validate [:maybe string?] "kikka" {:registry m/default-registry})
; => true
The default immutable registry is merged from the following parts, enabling easy re-composition of custom schema sets:
Contains both function values and unqualified symbol representations for all relevant core predicates. Having both representations enables reading forms from both code (function values) and EDN-files (symbols): any?
, some?
, number?
, integer?
, int?
, pos-int?
, neg-int?
, nat-int?
, float?
, double?
, boolean?
, string?
, ident?
, simple-ident?
, qualified-ident?
, keyword?
, simple-keyword?
, qualified-keyword?
, symbol?
, simple-symbol?
, qualified-symbol?
, uuid?
, uri?
, decimal?
, inst?
, seqable?
, indexed?
, map?
, vector?
, list?
, seq?
, char?
, set?
, nil?
, false?
, true?
, zero?
, rational?
, coll?
, empty?
, associative?
, sequential?
, ratio?
and bytes?
.
Class-based schemas, contains java.util.regex.Pattern
& js/RegExp
.
Comparator functions as keywords: :>
, :>=
, :<
, :<=
, :=
and :not=
.
Contains :and
, :or
, :map
, :map-of
, :vector
, :list
, :sequential
, :set
, :tuple
, :enum
, :maybe
, :multi
, :re
and :fn
.
Example to create a custom registry without the default core predicates and with :bool
and :int
Schemas:
(def registry
(merge
(m/class-schemas)
(m/comparator-schemas)
(m/base-schemas)
{:int (m/fn-schema :int int?)
:bool (m/fn-schema :bool boolean?)}))
(m/validate [:or :int :bool] 'kikka {:registry registry})
; => false
(m/validate [:or :int :bool] 123 {:registry registry})
; => true
Predicate Schemas don't work anymore:
(m/validate int? 123 {:registry registry})
; Syntax error (ExceptionInfo) compiling
; :malli.core/invalid-schema
Any schema can define a local registry using :registry
schema property:
(def Adult
[:map {:registry {::age [:and int? [:> 18]]}}
[:age ::age]])
(mg/generate Adult {:size 10, :seed 1})
; => {:age 92}
Local registries can be persisted:
(-> Adult
(malli.edn/write-string)
(malli.edn/read-string)
(m/validate {:age 46}))
; => true
See also Recursive Schemas.
Using custom registries via :registry
option is a simple solution, but this needs to be done for all public api calls. Also, with ClojureScript, the large (100+ schemas) default registry is not subject to any Dead Code Elimination (DCE), even if the schemas are not used in the application.
Malli allows the default registry to be replaced, with the following compiler/jvm bootstrap:
- cljs:
:closure-defines {malli.registry/type "custom"}
- clj:
:jvm-opts ["-Dmalli.registry/type=custom"]
It changes the default registry to empty one, which can be changed using malli.registry/set-default-registy!
. Empty default registry enableds DCE for all unsed schema implementations.
Malli supports multiple types of registries.
(require '[malli.registry :as mr])
;; - cljs: :closure-defines {malli.registry/type "custom"}
;; - clj: :jvm-opts ["-Dmalli.registry/type=custom"]
(mr/set-default-registry!
{:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)})
(m/validate
[:map [:maybe [:maybe :string]]]
{:maybe "sheep"})
; => true
;; gzipped malli.core size as js down from 12Kb -> 1.2Kb
clojure.spec introduces a mutable global registry for specs. The mutable registry in malli forced you to bring in your own state atom and functions how to work with it:
Using a custom registry atom:
(def registry*
(atom {:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)}))
(defn register! [type ?schema]
(swap! registry* assoc type ?schema))
;; - cljs: :closure-defines {malli.registry/type "custom"}
;; - clj: :jvm-opts ["-Dmalli.registry/type=custom"]
(mr/set-default-registry!
(mr/mutable-registry registry*))
(register! :non-empty-string [:string {:min 1}])
(m/validate :non-empty-string "malli")
; => true
The mutable registry can also passed in as explicit option:
(def registry (mr/mutable-registry registry*))
(m/validate :non-empty-string "malli" {:registry registry})
; => true
If you know what you are doing, you can also use dynamic scope to pass in default schema registry:
;; - cljs: :closure-defines {malli.registry/type "custom"}
;; - clj: :jvm-opts ["-Dmalli.registry/type=custom"]
(mr/set-default-registry!
(mr/dynamic-registry))
(binding [mr/*registry* {:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)
:non-empty-string [:string {:min 1}]}]
(m/validate :non-empty-string "malli"))
; => true
Registries can be composed:
(require '[malli.core :as m])
(require '[malli.registry :as mr])
;; bring your own evil
(def registry (atom {}))
(defn register! [type schema]
(swap! registry assoc type schema))
;; - cljs: :closure-defines {malli.registry/type "custom"}
;; - clj: :jvm-opts ["-Dmalli.registry/type=custom"]
(mr/set-default-registry!
;; linear search
(mr/composite-registry
;; immutable registry
{:map (m/-map-schema)}
;; mutable (spec-like) registry
(mr/mutable-registry registry)
;; on the perils of dynamic scope
(mr/dynamic-registry)))
;; mutate like a boss
(register! :maybe (m/-maybe-schema))
;; ☆.。.:*・°☆.。.:*・°☆.。.:*・°☆.。.:*・°☆
(binding [mr/*registry* {:string (m/-string-schema)}]
(m/validate
[:map [:maybe [:maybe :string]]]
{:maybe "sheep"}))
; => true
Schemas as just data, so they can be either inlined (values) or referenced (entities) in other schemas. For validation, they work the same way, but for model documentation, they are kept as separate.
Schemas can be represented as abstract schema syntax and referenced as values:
(def Age
[:and int? [:> 18]])
(def User
[:map
[:name string?]
[:age Age]])
(m/validate
User
{:name "Mirjami", :age 62})
; => true
NOTE: Schema format validation only occurs when a m/schema
is called, so here Age
and User
could contain syntax errors.
Wrapping schemas into m/schema
makes them first class entities. Here User
is an entity, while Age
is a (embedded) value.
(def Age
[:and int? [:> 18]])
(def User
(m/schema
[:map
[:name string?]
[:age Age]]))
(m/validate
User
{:name "Mirjami", :age 62})
; => true
We are building dynamic multi-tenant systems where data-models should be first-class: they should drive the runtime value transformations, forms and processes. We should be able to edit the models at runtime, persist them and load them back from database and over the wire, for both Clojure and ClojureScript. Think of JSON Schema, but for Clojure/Script.
Hasn't the problem been solved (many times) already?
There is Schema, which is awesome, proven and collaborative open source project, and we absolutely love it. We still use it in most of our projects. Sad part: serializing & de-serializing schemas is non-trivial and there is no back-tracking on branching.
Spec is the de facto data specification library for Clojure. It has many great ideas, but it is based on macros, it has a global registry and it doesn't support runtime transformations. Spec-tools was created to "fix" some of the things, but after three years of developing it, it's still kinda hack and not fun to maintain.
So, we decided to spin out our own library, which would do all the things we feel is important for dynamic system development. It's based on the best parts of the existing libraries and several project-specific tools we have done over the years.
If you have expectations (of others) that aren't being met, those expectations are your own responsibility. You are responsible for your own needs. If you want things, make them.
- Rich Hickey, Open Source is Not About You
- Schema https://github.com/plumatic/schema
- Clojure.spec https://clojure.org/guides/spec
- Spell-spec https://github.com/bhauman/spell-spec
- JSON Schema https://json-schema.org/understanding-json-schema/index.html
- Spec-provider: https://github.com/stathissideris/spec-provider
- F# Type Providers: https://docs.microsoft.com/en-us/dotnet/fsharp/tutorials/type-providers/
- Core.typed https://github.com/clojure/core.typed
- TypeScript https://www.typescriptlang.org/
- Struct https://funcool.github.io/struct/latest/
- JOI https://github.com/hapijs/joi
We use Kaocha as a test runner. Before running the tests, you need to install NPM dependencies.
npm install
bin/kaocha
clj -Ajar
clj -Ainstall
npx shadow-cljs run shadow.cljs.build-report app /tmp/report.html
npx shadow-cljs release app --pseudo-names
Copyright © 2019-2020 Metosin Oy and contributors.
Available under the terms of the Eclipse Public License 2.0, see LICENSE
.