haskell-waargonaut / waargonaut Goto Github PK

ghci> jNumberToScientific $ _JNumberScientific # ( fromFloatDigits @Double 5 )
Just 5.0
ghci> jNumberToScientific $ _JNumberScientific # ( fromFloatDigits @Double 0.5 )
Just 5.0

This seems to be because of an off-by-one error in _JNumberScientific. I don't understand why the testsuite doesn't catch this.

Fix in #85.

Hi! I'm unsuccessfully trying to build a project with recommended extra-deps.

stack.yaml:

resolver: lts-19.18

# some stuff ...

extra-deps:
  - waargonaut-0.8.0.2
  - digit-0.11@sha256:d3b42df305e64756505e46dabad32d08df3721b2f58e7c4741cfd6a73ccaaef2,3888
  - hoist-error-0.2.1.0@sha256:b7ccf72fe0dc2339a06956a0388115a33550cbe08089c7c2da310d1ef7aa7b9c,1353
  - lens-4.19.2@sha256:d4d704141d7c322bbfb746157b0709f3b966dfec92421f571c34069893af08cc,16062
  - natural-0.3.0.6@sha256:6149f7df782b5fa0d7e6685319fed73cacbe3efb0721acff4d2c34e0f6d92205,2042
  - witherable-0.3.5@sha256:9febcda439f514aec8f430bd3c75a7cda20b28978a8cbad1b2f8f097a3a16b42,1542
  - Cabal-3.2.1.0@sha256:8743076ec022296f9771d962000c9ca3a0fe02e68c37b992c63e382a675f791d,27482
  - template-haskell-2.16.0.0@sha256:71d8c2e92b712695cb4fd0be9cd1a41b05c25315e9748cfef1132bf4d02b6556,1917

package.yaml:

dependencies:
  - base >= 4.7 && < 5
  - aeson
  - bytestring
  - text
  - unordered-containers
  - containers
  - stm
  - wai
  - wai-logger
  - warp
  - servant
  - servant-server
  - websockets
  - wai-websockets
  - servant-websockets
  - wai-app-static
  - hashable
  - waargonaut
  - digit
  - hoist
  - lens
  - natural
  - witherable
  - Cabal
  - template-haskell

# some stuff ...

Without the last two (Cabal and template-haskell) got such message:

Error: While constructing the build plan, the following exceptions were encountered:

Dependency cycle detected in packages:
    [aeson,monoidal-containers,witherable,aeson,backend]

In the dependencies for backend-0.1.0.0:
    hoist needed, but the stack configuration has no specified version (no package with that name found, perhaps there is a typo in a package's build-depends or an
          omission from the stack.yaml packages list?)
    witherable dependency cycle detected: witherable, monoidal-containers, witherable, aeson, backend
needed since backend is a build target.

In the dependencies for lens-4.19.2:
    Cabal-3.4.1.0 from stack configuration does not match >=1.10 && <3.3  (latest matching version is 3.2.1.0)
    template-haskell-2.17.0.0 from stack configuration does not match >=2.4 && <2.17  (latest matching version is 2.16.0.0)
needed due to backend-0.1.0.0 -> lens-4.19.2

In the dependencies for monoidal-containers-0.6.2.0:
    aeson dependency cycle detected: aeson, monoidal-containers, witherable, aeson, backend
    witherable dependency cycle detected: witherable, monoidal-containers, witherable, aeson, backend
needed due to backend-0.1.0.0 -> monoidal-containers-0.6.2.0

In the dependencies for servant-0.19:
    aeson dependency cycle detected: aeson, monoidal-containers, witherable, aeson, backend
needed due to backend-0.1.0.0 -> servant-0.19

In the dependencies for servant-server-0.19.1:
    aeson dependency cycle detected: aeson, monoidal-containers, witherable, aeson, backend
needed due to backend-0.1.0.0 -> servant-server-0.19.1

In the dependencies for servant-websockets-2.0.0:
    aeson dependency cycle detected: aeson, monoidal-containers, witherable, aeson, backend
needed due to backend-0.1.0.0 -> servant-websockets-2.0.0

In the dependencies for waargonaut-0.8.0.2:
    witherable dependency cycle detected: witherable, monoidal-containers, witherable, aeson, backend
needed due to backend-0.1.0.0 -> waargonaut-0.8.0.2

In the dependencies for wai-extra-3.1.12.1:
    aeson dependency cycle detected: aeson, monoidal-containers, witherable, aeson, backend
needed due to backend-0.1.0.0 -> wai-extra-3.1.12.1

Dependency cycle detected in packages:
    [witherable,monoidal-containers,witherable,aeson,backend]

Unknown package: witherable-class

In my opinion the current API for creating a 'map-like' JSON object using the
mapLikeObj and atKey functions is awkward and not as straight-forward as I would like.

In the current form, it looks like this:

personEncoder :: Applicative f => Encoder f Person
personEncoder = E.mapLikeObj $ \p ->
  E.atKey' "name" E.text (_personName p) .
  E.atKey' "age" E.int (_personAge p) .
  E.atKey' "address" E.text (_personAddress p) .
  E.atKey' "numbers" (E.list E.int) (_personFavouriteLotteryNumbers p)

With the atKey' functions being composed together to create the final object.
However the available generalisation over f complicates things a bit when both
the mapLikeObj function AND the atKey functions both have general and
Identity implementations. But neither mapLikeObj function works when used
with the generalised over f atKey function.

Also, the fact that these functions are composed isn't always obvious to newer users
of the library.

Also the general type of the atKey functions can produce errors that are difficult to untangle:

atKey :: (At t, IxValue t ~ Json, Applicative f) => Index t -> Encoder f a -> a -> t -> f t
atKey' :: (At t, IxValue t ~ Json) => Index t -> Encoder' a -> a -> t -> t

Contrast this to the glorious ease of decoding the equivalent object:

personDecoder2 :: Monad f => Decoder f Person
personDecoder2 = Person
  <$> D.atKey "name" D.text
  <*> D.atKey "age" D.int
  <*> D.atKey "address" D.text
  <*> D.atKey "numbers" (D.list D.int)

I would like to have something for creating 'map-like' objects that has a more
obvious interface as well as being more difficult to use incorrectly.

Maybe something like:

data ObjKV f a = OKV
  { _objkvKey :: Text
  , _objkvEncoder :: Encoder f v
  , _objkvGetter :: a -> (v ????)
  }

mkObj :: (Applicative f, Foldable g) => g (OKV f a) -> a -> Encoder f a

-- Then encoding a map-like object is more obviously:
mapLike :: Applicative f => Encoder f a
mapLike = E.encodeA . mkObj
  [ OKV "name" E.text  _personName
  , OKV "age" E.int _personAge
  , OKV "address" E.text _personAddress
  , OKV "numbers" (E.list E.int) _personFavouriteLotteryNumbers
  ]

I'm not sure, I hacked that out as an idea...

Try to keep the suggestions reasonable. Whilst I'm not against burning the
Encoder structure to the ground in the name of a far superior alternative, I
would expect a sufficiently compelling justification. You need more than
"I/we/this package/that package/bob/susan does it this way and I like it, so you
should do that."

Likewise, nerdsnipes like "Build a better representation in ATS2 with linear
types and just use the FFI" are ❤️ , but no.

Given a record like this:

  data Property = SomeProperty {
      name :: Text,
      description :: Text
    }
    | FancyProperty {
      name :: Text,
      description :: Text

      total :: Maybe Int,
      average :: Maybe Int,
      required :: Maybe Bool
    }
    deriving (Show)

I then want to write a property encoder that looks something like this:

import qualified Waargonaut.Encode as WE

encodeProperty :: Applicative f => WE.Encoder f Property
encodeProperty = WE.mapLikeObj $ \jss ->
  WE.textAt "name" (name jss) .
  WE.textAt "description" (description jss) .
  WE.textAt "type" (encodePropertyType jss) .
  WE.boolAt "required" (required jss)   -- this field only exists in one of the Property record's constructors

encodePropertyType :: Property -> Text
encodePropertyType x = case x of
  SomeProperty {} -> "some"
  FancyProperty {} -> "fancy"

When I use gEncoder to generate an encoder for a data type, setting an _optionsFieldName field name modifier function, that function appears to be called twice, which is a problem if it isn't idempotent. To reproduce:

{-# LANGUAGE DataKinds             #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE TemplateHaskell       #-}
{-# LANGUAGE TypeFamilies          #-}
module Test where

import Data.Text          (Text)
import Generics.SOP.TH    (deriveGeneric)
import Waargonaut.Encode  (Encoder)
import Waargonaut.Generic (GWaarg, JsonEncode (..), Options (..),
Tagged (..),
                           defaultOpts, gEncoder, untag)

data Foo = Foo
  { abc :: Text
  }
$(deriveGeneric ''Foo)

instance JsonEncode GWaarg Foo where
  mkEncoder = gEncoder (defaultOpts { _optionsFieldName  = tail })

fooEncoder :: Applicative f => Encoder f Foo
fooEncoder = untag (mkEncoder :: Applicative f => Tagged GWaarg
(Encoder f Foo))

Now to test in ghci:

λ simpleEncodeText fooEncoder (Foo "test")
"{"c":"test"}"

I expected to see:

λ simpleEncodeText fooEncoder (Foo "test")
"{"bc":"test"}"

Errors need to be more descriptive and precise. We don't have an error not finding a key on an object, for example. We have the history, but it'd be nice if the message was more expressive.

Also the history reporting hasn't been thoroughly battle tested so more than likely needs tweaking as well.

The moveToKey function is a O(n) search from its current position to the desired key or an error from the end of the object. The succinct data structures make this incredibly fast and the movements are super efficient.

But how efficient this is remains to be measured. Create benchmarks for an increasing in size ordered key object and find out where we run into performance problems by looking for the last key.

This isn't what I was expecting.

D.withCursor $ \c -> D.down c >> pure ()

will fail on "{}" or "[]". I was expecting the D.down to succeed, but then any D.moveRight to cause an error. Is this documented somewhere?

Update digit dependency to 0.6.

The overlay in waargonaut-deps.nix only applies to the default haskellPackages and not to the one selected if the compiler is changed (by passing the compiler argument to (shell|default).nix.

To reproduce:

nix-shell --argstr compiler ghc862

Expected: dropped in a nix shell

Actual: error saying that generics-sop-0.3.2.0 cannot be built. The overlay specifies version 0.4.0.1 of generics-sop, so it appears the overlays are not working.

I'm surprised there exists no encoder for that: https://hackage.haskell.org/package/waargonaut-0.8.0.1/docs/Waargonaut-Encode.html#g:4

Would you be interested in a PR that included a few default parsing functions (similar to or exactly like https://github.com/qfpl/waargonaut/blob/master/test/Types/Common.hs#L255) along with docs related to them?

My team recently decided to try Waargonaut and it took us a little while to piece together what was needed to construct a parser and we thought we could help reduce that time for another team.

I'm asking because I'm not sure if this would seem to be stepping on the toes of "BYO parsing library" by suggesting a default implementation.

Help appreciated.

OK, so there are no tests here to cheat from and I found if I passed in an Int to the Prettier functions (as the docs currently read to me) they'd both reject it.

This is what I've come up with to get it working, but I figure I've probably taken a detour somewhere:

import qualified Waargonaut.Encode as WE
import qualified Waargonaut.Prettier as WEP
import qualified Waargonaut.Types as WT
import qualified Data.Text.Lazy as TL

encode :: [J.Schema] -> [TL.Text]
encode xs =
    let spaces = WEP.NumSpaces $ successor' $ successor' zero' -- this is fun with Natural instead of 2::Int
        indent = WEP.IndentStep $ successor' $ successor' zero'
        beautify = WEP.simpleEncodePretty WEP.ArrayOnly indent spaces WE.json'
        json = WE.asJson' encodeSchema <$> xs
    in runIdentity . beautify <$> json

To get a decent print out in GHCI I am then calling it with:

import qualified Data.Text.Lazy.IO as T

main :: IO ()
main = do
  params <- -- fetches data here of type J.Schema
  foldr (const . T.putStr) (pure ()) $ JS.encode params

J.Schema looks like this:

data Schema = Schema {
  id :: Text,
  schema :: Text,
  schemaDescription :: Text,
  schemaProperties :: Properties
} deriving (Show)

What's the best way of utilising the Prettier stuff and how does one overcome the Natural impediment?

Build fails for the following reason:

Configuring concurrent-output-1.10.4...
Setup: At least the following dependencies are missing:
process >=1.6.0 && <1.7.0

Not sure if this is a straight forward fix as process doesn't exist in haskell.packages.ghc7103 due to being a boot library for GHC (https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory). So overriding it causes more terrible errors.

Waargonaut happily builds for these versions of GHC when nix isn't involved (see TravisCI).

My nix skills aren't enough for this, so help would very much be appreciated.

Currently the parser would parse "\u2705" into T.pack "\x05". And to get T.pack "\x2705" you have to put "\u00e2\u009c\u0085" in the JSON.

Hi, thank you for creating this library!
I just started learning it and I like it.

Reading through Waargonaut.Decode module, I realized that the either function attemps the Left decoder first, even though the document states "Right decoder is attempted first.".

For example:

> D.simpleDecode (D.either D.scientific D.int) parseBS "123"
Right (Left 123.0)

If the Right decoder is attempted first, the result should be Right (Right 123).

Currently CI always fails on GHC 8.0.2, which is annoying. Local builds are ok. It would be good to either fix it or remove GHC 8.0.2 support. My preference is the former.

The CI is conservative and builds with installed versions of boot dependencies. I think if we simply undid that, 8.0.2 would build again.

Waargonaut currently has a strict upper bound on generics-sop that prevents it from using the latest version (and thereby restricting Waargonaut from being used with servant-0.15).

Waargonaut's movement code uses Natural from Numeric.Natural, which doesn't actually enforce that a number is natural via the type system, but rather throws an impure arithmetic exception when provided with negative input.

Reproduction:

λ> :i manyMoves
manyMoves ::
  Monad m => GHC.Natural.Natural -> (b -> m b) -> b -> m b
  	-- Defined at /Users/jkachmar/code/waargonaut/src/Waargonaut/Decode.hs:215:1
λ> manyMoves (-1) undefined undefined
*** Exception: arithmetic underflow

Ok, this is a pedantic issue, I'm not sure how you feel about it. The readme should rather say

The data structures, parser, and printer are built to comply with the following *pseudo* properties

Since parse . print is more akin to Right rather than id :-)

The haddock for simpleEncodePretty says that it prints to a ByteString. In fact, it prints to a Text.

I noticed that you don't support GHC 8.10. Could you?

> preview _Empty (review _Empty () :: CommaSeparated WS (JAssoc WS Json))
Nothing

The result should be Just ().

I am using the resolver 18.10, which is very recent, and it seems like this package is not available: https://www.stackage.org/lts-18.10/hoogle?q=waargonaut

It would be a lot easier to use for stack users, if it were available in Stackage. I have attempted to add the package using extra-deps and the versions that stack suggest, but possibly due to the lack of a solver, Stack suggests a broken configuration:

Dependency cycle detected in packages:
    [witherable,witherable-class,witherable,waargonaut,argus]

In the dependencies for witherable-class-0.0.1:
    witherable dependency cycle detected: witherable, witherable-class, witherable, waargonaut