katydid / katydid-haskell

An Encoding Agnostic Validation Language

Katydid

Katydid is a toolkit for trees.

Currently there are three tools in the katydid toolkit:

• Relapse: a regular expression type language for trees that matches up to 1000000s of records per second,
• A collection of parsers (protobuf, json, xml, reflected go structures, yaml) which are easily extendable, and
• A collection of encoders (protobuf, json, xml, reflected go structures) which are useful for dynamic transcoding.

more...

katydid-haskell's Issues

Xml - consider simple helpers

functions "maybe" and "either" are handy... Or try to internalize more of the prelude.

Use aeson's token stream for json parser

I haven't thought deeply about your use case, but I almost always default to aeson

Missing files for test suite

This usually means you're missing some other-modules info in your .cabal file:

Building test suite 'katydid-test' for katydid-0.1.0.0..
[1 of 1] Compiling Main             ( test/Spec.hs, dist/build/katydid-test/katydid-test-tmp/Main.o )

test/Spec.hs:10:1: error:
    Could not find module ‘ParserSpec’
    Use -v to see a list of the files searched for.
   |
10 | import qualified ParserSpec
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^

test/Spec.hs:11:1: error:
    Could not find module ‘Suite’
    Use -v to see a list of the files searched for.
   |
11 | import qualified Suite
   | ^^^^^^^^^^^^^^^^^^^^^^

test/Spec.hs:12:1: error:
    Could not find module ‘RelapseSpec’
    Use -v to see a list of the files searched for.
   |
12 | import qualified RelapseSpec
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Consider xeno for faster xml

https://github.com/ocramz/xeno

Optimize

Useful ideas for optimizing:

• Finish benchmarks on protobufs and compare with go implementation before starting anything and document progress.
• Flamegraph: https://www.fpcomplete.com/blog/2015/04/ghc-prof-flamegraph
• Try strict ! on the leaves of data structures.

Gadt for expr

Try refactoring to use GADTs if you want. There are examples on the web of such work. I didn't read deep enough to see if this would be easy or not, but my kneejerk reaction to seeing repetition on ADT for expression is to consider GADTs

Fewer type aliases?

Somewhat of a style choice... But I would tend towards not aliasing the types or using a full newtype

Trivial, but might be of interest - test groups

Tasty or hspecs features for grouping tests might be interesting to you

Zip module, consider ADT instead of special ints

data ZipEntry = Val Int | Special1 | Special2

I don't read deep enough yet to give proper names for special1, etc, with respect to -1 and -2

Concise use cases

Can you mention a few real world use cases you have in mind for this, and point out where this fits within prior work? I skimmed docs briefly, but I don't have a strong understanding of when I would choose this over other options.

Also, when would I choose this, if I am exclusively dealing with json?

Would I want to use this validating a tree of data expressed as in-memory Haskell data types?

Utilities for tagged types

Ideally, when I have external data, I want to:

• parse it
• validate it
• construct a new value with a type tagged to note it has been validated

Maybe you can address the final step of creating tagged types?

katydid 0.3 build failure with ghc 8.4

Related to the semigroup changes in ghc 8.4.
See also: https://ghc.haskell.org/trac/ghc/wiki/Migration/8.4#SemigroupMonoidsuperclasses

[ 3 of 23] Compiling Expr             ( src/Expr.hs, dist/build/Expr.o )

src/Expr.hs:283:39: error:
    Ambiguous occurrence ‘<>’
    It could refer to either ‘Prelude.<>’,
                             imported from ‘Prelude’ at src/Expr.hs:4:8-11
                             (and originally defined in ‘GHC.Base’)
                          or ‘Expr.<>’, defined at src/Expr.hs:276:1
    |
283 | cmp a b = compare (_hash a) (_hash b) <>
    |                                       ^^

src/Expr.hs:284:33: error:
    Ambiguous occurrence ‘<>’
    It could refer to either ‘Prelude.<>’,
                             imported from ‘Prelude’ at src/Expr.hs:4:8-11
                             (and originally defined in ‘GHC.Base’)
                          or ‘Expr.<>’, defined at src/Expr.hs:276:1
    |
284 |     compare (_name a) (_name b) <>
    |                                 ^^

src/Expr.hs:285:55: error:
    Ambiguous occurrence ‘<>’
    It could refer to either ‘Prelude.<>’,
                             imported from ‘Prelude’ at src/Expr.hs:4:8-11
                             (and originally defined in ‘GHC.Base’)
                          or ‘Expr.<>’, defined at src/Expr.hs:276:1
    |
285 |     compare (length (_params a)) (length (_params b)) <>
    |                                                       ^^

src/Expr.hs:286:11: error:
    Ambiguous occurrence ‘<>’
    It could refer to either ‘Prelude.<>’,
                             imported from ‘Prelude’ at src/Expr.hs:4:8-11
                             (and originally defined in ‘GHC.Base’)
                          or ‘Expr.<>’, defined at src/Expr.hs:276:1
    |
286 |     foldl (<>) EQ (zipWith cmp (_params a) (_params b)) <>
    |           ^^^^

src/Expr.hs:286:57: error:
    Ambiguous occurrence ‘<>’
    It could refer to either ‘Prelude.<>’,
                             imported from ‘Prelude’ at src/Expr.hs:4:8-11
                             (and originally defined in ‘GHC.Base’)
                          or ‘Expr.<>’, defined at src/Expr.hs:276:1
    |
286 |     foldl (<>) EQ (zipWith cmp (_params a) (_params b)) <>
    |                                                         ^^

I was able to reproduce this locally like so:

stack unpack katydid-0.3.0.0 && cd katydid-0.3.0.0
echo 'resolver: nightly-2018-05-06' > stack.yaml
stack build --haddock --test --bench --no-run-benchmarks