jvican / dijon Goto Github PK

Hi, How do you convert the resultant json from here back to dataframe using spark scala

Example:

scala> import com.github.pathikrit.dijon._
scala > val test = """ {"id":3,"content":"src = \"\", desc = \"HI\"","cls":"Figure"} """
scala> val out = parse(test)

Now if convertig out to String representation the masked quotes (") from
test aren't masked anymore:

scala> out.toString
String = {"cls" : "Figure", "content" : "src = "", desc = "HI"", "id" : 3.0}

Even dijon itself could not handle this:

scala> parse(out.toString)
java.lang.IllegalArgumentException: Could not convert to JSON

In my own code I'm using a little workaround:

scala> out.content = out.content.as[String].get.replace("\"", "\\\"")

Now dijon and other json parser can handle this:

scala> parse(out.toString)

Maybe you can built in this workaround? The problem is, that this must
applied recursivly on the values only, otherwise if the keys are also
"replaced" the json would get corrupt.

The full file and output is here:
https://gist.github.com/tylerprete/275e60183a52095ecc47

But the gist is as follows:
val d = parse( """{"name": {"title": "Rat"}}""")
val wrong: Option[String] = d.name.wrong.as[String]
val lowerWrong = wrong map {
_.toLowerCase
}
val lowerWrongStr = lowerWrong.getOrElse("none")
println(lowerWrongStr)

This fails with "scala.None$ cannot be cast to java.lang.String"

I've ran through it in a debugger, and the value of "wrong" before the map is: Some(None),
which should not be possible for an Option[String].

It seems like there is a type-unsafe cast in there somewhere.

scala/scala@3e1a075a7e8b91

@pathikrit Hi, Pathikrit!

The latest version of jsoniter-scala breaks backward binary compatibility.
It was done for easier evolving when adding new configuration flags or new routines to support more types/representations for versions 1.x until 2.0.0.

Please release the Dijon v0.4.0 with all merged changes.

Thanks,
Andriy.

Is dijon scala.js friendly? I can work on it but not sure if it is something possible.

When will dijon support scala 2.11.x as release?

val aoeu = json"""
{"took":1182,"timed_out":false,"_shards":{"total":2,"successful":2,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]},"facets":{"failures":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]}}}
"""
println(aoeu.toString())

Sorry I haven't taken the time to whittle down the JSON to something minimal.

Hi...If I run this

val (name, age) = ("Tigri", 7)
val cat = json"""
  {
    "name": "$name",
    "age": $age,
    "hobbies": ["eating", "purring"],
    "is cat": true
  }
"""

cat.age.as[Int].getOrElse(2)

I get an error...cat.age.as[Int] has the type Option[Int]...so is pretty weird behaviour...what could be the best option for get a json field and pass as paramete to a method..because this works

cat.age == 7

but this doesnt:

cat.age * 2

I might be doing something wrong:

val json = dijon.parse(jsonString)
json.timestamp.as[Long]

The following should be valid:

val query = json"""
{
'name': 'name'
}
"""

but currently throws an exception indicating that it's invalid JSON.

Example:

import com.github.pathikrit.dijon.JsonStringContext
import com.github.pathikrit.dijon

scala> json"""{"hi": "ho\nho"}"""
java.lang.IllegalArgumentException: Invalid JSON
...

But if using parse it works as expected:

scala> dijon.parse("""{"hi": "ho\nho"}""")
res11: com.github.pathikrit.dijon.Json[_] = {"hi" : "ho\nho"}

https://github.com/jrudolph/json-lenses

There are no artifacts for Scala 2.11 and 2.12

Bellow is a flame graph that shows the problem which was detected by these benchmarks.

Hi...I'm not so newbie with scala but I can't understand why this works

json""" {....}"""

but this not
json("""{ .. }""") or json.apply("""{ }""")

error: not found: value json
              json(q)

how can format a string, basically I need pass my string as parameter but just now dijon is doing some black magic than I can't understand...thanks!!

I'm handling with multiline strings with json, but here's a bug.

Welcome to Scala version 2.10.3 (OpenJDK 64-Bit Server VM, Java 1.6.0_30).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import com.github.pathikrit.dijon._
import com.github.pathikrit.dijon._

scala> val json = `{}`
json: com.github.pathikrit.dijon.Json[_] = {}

scala> json.hi = 10
json.hi: com.github.pathikrit.dijon.Json[_] = 10

scala> json
res0: com.github.pathikrit.dijon.Json[_] = {"hi" : 10}

scala> json.ml = """this is
     | a multiline string
     | """
json.ml: com.github.pathikrit.dijon.Json[_] = "this is
a multiline string
"

scala> json
res1: com.github.pathikrit.dijon.Json[_] = 
{"hi" : 10, "ml" : "this is
a multiline string
"}

scala> json.ml
res2: com.github.pathikrit.dijon.Json[_] = 
"this is
a multiline string
"

scala> json.toString
res3: String = 
{"hi" : 10, "ml" : "this is
a multiline string
"}

The problem is, that res3 is invald json and won't be interpreded. Is there a clever workaround?

https://github.com/kmizu/jsonda
https://github.com/maffoo/jsonquote

Please consider dropping support of Scala 2.11.x because latest versions of jsoniter-scala and Scala Native don't support this outdated version

Hi there,

maybe with an example it is clearer:

import com.github.pathikrit.dijon.`[]`
val x = `[]`
x(0) = 1
x(1) = 2

x.toString
[1, 2]

Is it possible to do something like that?

x -- 1
x.toString
[1]

The json specification allows null as value: http://www.json.org

import com.github.pathikrit.dijon
dijon.parse("""{"hi": null}""")

If pased to a scala structure it should be of type None?
Tested with version 0.2.3.

@pathikrit I've updated results of benchmarks and taking a pause in adding PRs...

Please review last PRs merged by me and cut the 0.3.0 release if you will not spot any issue...