GithubHelp home page GithubHelp logo

metosin / spec-tools Goto Github PK

View Code? Open in Web Editor NEW
592.0 37.0 92.0 1.08 MB

Clojure(Script) tools for clojure.spec

License: Eclipse Public License 2.0

Clojure 99.04% Shell 0.96%
clojure clojure-spec json-schema swagger clojurescript metosin-inactive

spec-tools's Introduction

spec-tools Build Status cljdoc badge

Clojure/Script utilities on top of clojure.spec. Bundled in one project but features in separate namespaces.

Metosin Open Source Status: Inactive

Posts:

Latest version

Clojars Project

Requires Java 1.8, tested with Clojure 1.10.0 and ClojureScript 1.10.520+.

Status: Alpha (as spec is still alpha too).

License

Copyright © Metosin Oy and contributors. Distributed under the Eclipse Public License 2.0, see LICENSE.

Picture, Noveau Larousse Ilustré (Larousse XIXs. 1866-1877). Public domain, via Wikimedia Commons.

The spell-spec library, Copyright © 2018 Bruce Hauman, is distributed under the Eclipse Public License as well.

spec-tools's People

Contributors

akond avatar arichiardi avatar c0rrzin avatar carocad avatar christoph-frick avatar dehli avatar deraen avatar f-f avatar heyarne avatar ikitommi avatar javahippie avatar jhacksworth avatar just-sultanov avatar kalekale avatar kennyjwilli avatar maacl avatar magemasher avatar metametadata avatar miikka avatar nenadalm avatar nhurden avatar olical avatar opqdonut avatar prepor avatar slipset avatar sumbach avatar theikkila avatar wandersoncferreira avatar wandferreira avatar yogthos avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spec-tools's Issues

README: `clojure.lang.ArityException: Wrong number of args (3) passed to: core/spec` following the "Example usage" code

user=> (require '[clojure.spec :as s])
nil
user=> (require '[spec-tools.core :as st])
nil
user=> (def my-integer? (st/spec integer?))
#'user/my-integer?
user=> my-integer?
#spec_tools.core.Spec
{:form clojure.core/integer?,
 :pred #<Fn@21aa40c8 clojure.core/integer_QMARK_>,
 :type :long}
user=> (my-integer? 1)
true
user=> (s/valid? my-integer? 1)
true
user=> (assoc my-integer? :info {:description "It's a int"})
#spec_tools.core.Spec
{:form clojure.core/integer?,
 :info {:description "It's a int"},
 :pred #<Fn@21aa40c8 clojure.core/integer_QMARK_>,
 :type :long}
user=> (eval (s/form (st/spec ::st/long integer? {:description "It's a int"})))

CompilerException clojure.lang.ArityException: Wrong number of args (3) passed to: core/spec, compiling:(/tmp/form-init2605111354228652905.clj:1:15) 

Data spec doesn't produce same results as long form

I believe the following code should all conform the same way but, under spec-tools 0.5.1, but the spec created through data-spec fails.

(:require 
            [clojure.spec.alpha :as s]
            [spec-tools.core :as st]
            [spec-tools.spec :as spec]
            [spec-tools.data-spec :as ds])

(let [conformer (st/type-conforming
                    (assoc
                      spec-tools.conform/json-type-conforming
                      :timestamp
                      (fn [a b]
                        (if-not (satisfies? Inst b)
                          (java.util.Date. b)
                          b))))]
    (s/def :test/timestamp (st/spec spec/inst? {:type :timestamp}))
    (s/def :test/ts :test/timestamp)
    (println (st/conform :test/timestamp
                         1508703225865
                         conformer))
    (println (st/conform (s/keys :req [:test/ts])
                         {:test/ts 1508703225865}
                         conformer))
    (println (st/conform (ds/spec :test {:test/ts :test/timestamp})
                         {:test/ts 1508703225865}
                         conformer))
    (println (st/explain-data (ds/spec :test {:test/ts :test/timestamp})
                              {:test/ts 1508703225865}
                              conformer)))

Class cast exception when attempting to coerce spec

I have a spec which is giving the error:
java.lang.ClassCastException: clojure.lang.PersistentList cannot be cast to clojure.lang.Named
when attempting to load the swagger UI for a compojure-api app.
The failing spec appears to be one that uses conditional keys in a :req parameter of a clojure.spec/keys declaration.

Environment:

  • [org.clojure/clojure "1.9.0-alpha17"]
  • [metosin//compojure-api "2.0.0-alpha7"]
  • [metosin/spec-tools "0.4.0-SNAPSHOT"]

select-spec not removing extra keys

Hey there. select-spec looked like the perfect tool for the job when I was hoping to take a map of X + N keys and pull out only X. Alas, it doesn't seem to be doing anything for me. Perhaps it's a bug, but more likely I'm missing something simple.

user=> (require '[spec-tools.core :as st])
nil

; First try unqualified
user=> (s/def ::meow (s/keys :req-un [::a ::b ::c]))
:user/meow
user=> (st/conform ::meow {:a 1 :b 2 :c 3 :d 4} st/strip-extra-keys-conforming)
{:a 1 :b 2 :c 3 :d 4} ; Nope. :(

; How about qualified?
user=> (s/def ::kitty (s/keys :req [::a ::b ::c]))
:user/kitty
user=> (st/conform ::kitty {::a 1 ::b 2 ::c 3 ::d 4} st/strip-extra-keys-conforming)
{:user/a 1
 :user/b 2
 :user/c 3
 :user/d 4} ; Nope. :(

Conform strings while leaving everything else unformed

This is either a question of a feature request, depending on whether or not it's already available. Based on what I've read from the docs, this hasn't been mentioned.

Use case

I'm working with ClojureScript, where my map values which are keyword are often turned into strings against my will. In hopes of fighting this problem, I've been trying out spec-tools. He's an example of the conforming I'm doing:

user=> (require '[cljs.spec.alpha :as s]
                '[spec-tools.core :as st]
                '[spec-tools.spec :as sts])
user=> (s/def ::foo (s/keys :req [::bar ::spam]))
:user/foo
user=> (s/def ::bar (s/and sts/keyword? #{::meow}))
:user/bar
user=> (s/def ::spam sts/string?)
:user/spam
user=> (st/conform ::foo {::bar "user/meow" ::spam "user/meow"} st/string-conforming)
{:user/bar :user/meow :user/spam "user/meow"}

Desired functionality

As shown above, this conforming is sweeet. Unfortunately, it also does the normal conforming stuff, like turning s/or and s/alt into their named pairs. Here's an example, based on the same REPL session above.

user=> (s/def ::spam (s/or :string sts/string? :integer sts/integer?))
:user/spam
user=> (st/conform ::foo {::bar "user/meow" ::spam "user/meow"} st/string-conforming)
{:user/bar :user/meow :user/spam [:string "user/meow"]}

As soon as ::spam has an alternative, the value I get back is less than ideal.

Short-term work-around

For now, I've been subsequently unforming my conform, which adds the keywords in the right spot, but removes the named pairs. As you can imagine, for performance, efficiency, and cleanliness, this isn't ideal. Here's an example, again, based on the above session:

user=> (s/unform ::foo (st/conform ::foo {::bar "user/meow" ::spam "user/meow"} st/string-conforming))
{:user/bar :user/meow :user/spam "user/meow"}

Long-term solution

As this is a common problem in both ClojureScript and Clojure code working with JSON and other formats which don't support EDN keywords, a cleaner solution would be preferred. I'd be happy to discuss more.

Thanks!

better s/merge for JSON Schema

(require '[clojure.spec :as s])

(s/def :int/a int?)
(s/def :string/a string?)
(s/def ::merge
  (s/merge
    (s/keys :req-un [:int/a])
    (s/keys :req-un [:string/a])))

(s/explain ::merge {:a 1})
; In: [:a] val: 1 fails spec: :string/a at: [:a] predicate: string?

(s/explain ::merge {:a "1"})
; In: [:a] val: "1" fails spec: :int/a at: [:a] predicate: int?

=> for JSON Schema, should emit :allOf from this?

Handle defaults via conforming

(s/def ::int
  (st/spec
    {:spec integer?
     :default 42}))

(st/conform ::int nil nil)
; ::s/invalid

(st/conform ::int nil st/default-conforming)
; 42

should contribute to JSON Schema too.

Better JSON Schema generation for s/tuple

related: metosin/compojure-api#327

;; heterogeneous lists not supported
(defmethod accept-spec 'clojure.spec.alpha/tuple [_ _ children _]
{:type "array"
:items {}
:x-items children})

As Swagger doesn't support heterogeneous lists, the current implementation converts to "array of anything" (with full info as vendor extension), which renders poorly on the UI. If all the tuple-elements are of same type, we could use that as the type. Also, as it could set the :minItems and :maxItems to the number of the elements in the tuple.

Failing on extra keys

I have interest in checking my data-structures for erroneous keys (any key that is not in the spec in this case.) I thought that spec-tools 'fail-on-extra-keys` seems perfect for exactly that.

But I can't figure out how to use it. Is this a bug?

(require '[spec-tools.core :as spec-tools])
(spec/def ::attr #{"value"})
(spec/def ::has-key (spec/keys opt [::attr]))

(spec-tool/conform
 ::has-key
 #:noextrakey.clj.core
 {:attr "value"
  :other-attr "still-valid?"}
 spec-tool/fail-on-extra-keys-conforming)

Composing conformers & types

Separating conformers from specs is a good idea, but still it would be nice to be able to compose conformers and add multiple types to specs. Currently, there can be only one installed conformer per :type and specs can have only one :type.

Something like should be available:

(st/spec ::person {:type [:map ::human]})

(st/conform 
  ::person 
  {"age" 20, "height" 200, "weight" 80, "level" :boss} 
  {:map [conform/strip-extra-keys (conform/transform-keys keyword)]
   ::human bmi-conform})
; => {:age 20, :height 200, :weight 80, :bmi 20}

Support nested maps, vectors & set with the simple format

With current "Concise Maps", it's still hard to define nested anonymous specs. Adding support for core data stuctures would help. Also, nillable value should be defined, maybe st/maybe? Top-level form could be any of those, so the st/map could be renamed too?

  • support {}
  • support []
  • support #{}
  • support st/maybe
  • rename st/map to .... st/coll-spec?
(s/def ::mrn string?)

(s/def ::patient
  (st/coll-spec
    ::patient
    {:demographics {:name string?
                    :age int?
                    :mrn ::mrn}
     :vitals {:weights [{:value number?
                         :units string?}]}}))

(s/valid?
  ::patient
  {:demographics {:name "xyz" :age 100 :mrn "abc"}
   :vitals {:weights [{:value 100 :units "lb"}]}})

Generate Schemas from Specs

Spec conformation is (alpha10) much slower than Schema coercion. If the perf doesn't get better with the upcoming versions, we could generate Schemas out of Specs and use Schema coercion.

Spiked to work on simple maps & predicates.

Way to have instance of Spec record (with metadata) when explain

Hi,

Thanks a lot for this awesome lib!

Can you please help me, is this possible to have instance of Spec record when conform failed.

Consider this scenario.

(def my-integer? (st/spec ::st/long integer? {:error "not_integer"}))
(s/explain-data my-integer? "a")

I expect to have Spec instance to be able to indicate "not_integer" to user.

Maybe there is spect-tools/explain?

Please advice how I can solve this, thanks!

local-keys (with data-specs)

Currently, data-specs register the map-spec leaf-specs into global registry with generated names. To support anonymous map-specs, a local-keys implementation would be nice, not to pollute the global registry.

;; to register the sub-specs
(ds/spec ::user {:id int?, :name string?})

;; anonymous
(ds/spec {:id int?, :name string?})

ds/opt vs ds/maybe

Hello, i find it difficult to understand when should i use ds/opt vs when to use ds/maybe?

Could someone explain that, may it should be documented?

Swagger stackoverflow with recursive specs

copied from: metosin/compojure-api#339

---8<---
When I set the coercion to :spec and use a recursive spec like :

(s/def ::user nil?) ;; will be redefined below
(s/def ::name string)
(s/def ::parent (s/nilable ::user))
(s/def ::user (s/keys :req-un [::name ::parent]))

Swagger fail with this error:

{"type":"unknown-exception","class":"java.lang.StackOverflowError"}

It seems that the clojure.spec -> swagger conversion keep walking the recursive spec until the stack blows up.

Thoughts about fdef utils?

Just wanted to stir up a conversation about this. I personally find specing functions to be tedious for most, simple functions.

Assume we had this spec:

(ds/spec ::user {::id   spec/uuid?
                 ::age  spec/pos-int?
                 ::name spec/string?})

Assume we want to spec this function:

(defn teenager?
  [user]
  (<= 13 (::age user) 19))

It would be neat if we could somehow spec this function easier

(ds/specf teenager? 
  [::user] ; args
  spec/boolean? ; ret
  nil ; fn
)

Or even with a Schema like macro:

(ds/defn teenager? :- spec/boolean?
  [user :- ::user]
  (<= 13 (::age user) 19))

I haven't actually fleshed out any of my thoughts, just wanted to see what you thought about some function specing utilities.

Support ClojureScript

spec-tools.visitor and spec-tools.json-schema should be made work as .cljc files. I quickly tried just renaming them, but tests didn't pass. Needs more investigation.

Resolve specs out of values

with spec-swagger, we need to resolve type for enums like #{1 2 3} - which is a enum of numbers.

maybe some goodness from spec-provider is needed. Would not like to take any new deps for spec-tools.

Possible improvement to data-spec/spec

Working with the data-spec/spec function I find myself repeating myself a lot to achieve functionality in a spec manner, by that in how I need things to feel like I am still working with spec itself.

I love the functionality of it but due to my approach, I find I'm not using it optimally and could pretty much stick to just Clojure spec by itself to achieve my goals. Anyway enough of my crazyish rant and to explain with a bit of code.

Using some fake code as an example following your guidelines as follows, I'm not sure about the second map with :description and :reason if I am using right or even using it in the right context.

(def fake-tag
  (st-ds/spec
    ::fake-tag
    {:tag (s/and keyword? #{:fake-tag})}
    {:description "fake tag"
     :reason "doesn't match format"}))

(st-c/conform fake-tag :fake-tag)

As you can see I create the spec then check if it conforms but what I am finding myself doing a lot is the following, in fact all the time, when using data-spec/spec is the following.

(s/def ::fake-tag
  (st-ds/spec
    ::fake-tag
    {:tag (s/and keyword? #{:fake-tag})}
    {:description "fake tag"
     :reason "doesn't match format"}))

(st-c/conform ::fake-tag :fake-tag)

As you can see I duplicate the namespaced key and using spec/def to register the spec in a more consistent spec manner just so I can use conform and the other related functions with a namespace key.

So what I am trying to say is I'd like to see data-spec/spec changed slight if possible to like the following, again if possible. Make the data-spec/spec registration be top level like the Clojure spec s/def to reduce the need for extra binding.

(st-ds/spec
  ::fake-tag
  {:tag (s/and keyword? #{:fake-tag})}
  {:description "fake tag"
   :reason "doesn't match format"}))

(st-c/conform ::fake-tag :fake-tag)

Sorry for my ramble at the beginning but hope it helps convey what I mean.

Support for set #{} as a spec predicate/collection

In clojure.spec you can use #{} as predicates in specs but using it with spec-tools I have to wrap it in clojure.spec/spec for it to be valid otherwise, it results in a vague error about the issue.

CompilerException java.lang.Exception: Unable to resolve spec: :deg, compiling:(css_spec.clj:33:14) 

that error is the result of the following, and any subsequent code using #{} directly will produce the same error

(s/def ::deg (st-ds/spec ::deg {:magnitude spec/int? :unit #{:deg}}))

the solution is fairly straight forward, just wrap in clojure.spec/spec

(s/def ::deg (st-ds/spec ::deg {:magnitude spec/int? :unit (s/spec #{:deg})}))

but it would be nice to have first class support for them like the rest of the collection types.

Create a spec conformer statically

Like dynamic conforming, but the conformer for a map is created at the same time as the spec is created. This avoids unnecessary walking of the spec at the runtime.

Identify map-specs in `spec-tools.core/spec` and collect keys for later manipulation

Enable easy implementation of conformers like remove-extra-keys& fail-on-extra-keys.

(defn remove-extra-keys [spec x]
  (if (map? x)
    (if-let [spec-keys (find-spec-keys spec)]
      (s/conform spec (select-keys x spec-keys))
      x)))

(s/def ::a clojure.core/int?)
(s/def ::b clojure.core/keyword?)
(s/def ::c clojure.core/string?)
(s/def ::ab (spec ::map (s/keys :req [::a ::b])))
(s/def ::cab (spec ::map (s/keys :req [::c] :opt [::ab])))

(conform
  ::cab
  {::c "c"
   ::ab {::a 1
         ::b :b
         ::EVIL "HACKER"}
   ::EVIL "HACKER"}
  {::map remove-extra-keys})
; => #:spec-tools.core{:c "c", :ab #:spec-tools.core{:a 1, :b :b}}

Validation by comparing keys.

Hello, my problem is - need compare two values from Map for validate. For example that value of one key less than value of another key.
{ :floor 5 :total-floor 10 } - For flat on 5 floor in building with 10 floors.
Can achieves this with spec-tools and clojure.spec?

Remove ns from Spec-keys

:spec/type => :type. Having a ns actually makes the spec-validation of the keys harder and has more to type. All extensions should use fully qualified names.

Extract reified clojure.spec Specs as Records

if (st/spec (s/keys :req [::id ::name]) would return a spec-tools.core/KeysSpec instead of spec-tools.core/Spec, we could add custom protocols ourselves, like Coercion and JSONSchema. Coercion would allow selective conforming without the dynamic vars. Nice.

Readme examples fail

It appears that the following examples fail:

https://github.com/metosin/spec-tools#more-complex-example
https://github.com/metosin/spec-tools#map-conforming

From the first one :

(s/def ::name string?)
(s/def ::birthdate spec/inst?)

(s/def ::languages
  (s/coll-of
    (s/and spec/keyword? #{:clj :cljs})
    :into {}))

(s/def ::user
  (s/keys
    :req-un [::name ::languages ::age]
    :opt-un [::birthdate]))

(def data
  {:name "Ilona"
   :age "48"
   :languages ["clj" "cljs"]
   :birthdate "1968-01-02T15:04:05Z"})

(st/conform ::user data st/string-conforming)

IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword  clojure.lang.RT.seqFrom (RT.java:547)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.