Timings comparaison with Play json about circe

[ktoso]

For writing benchmarks please consider using sbt-jmh (it's wrapping OpenJDK's JMH) as it will give more accurate results than running something in a loop or a few times in the repl – there's all kinds of warm-up method JIT-ing and other effects in play here (no pun intended 😉) so it's hard to measure properly without the right infrastructure (i.e. what JMH does around benchmark code).

I'd be very interested in comparing the various libs, but with the right tools :-)
I'm expecting circe to be faster due to it's nature, still curious how much faster.

[travisbrown]

Thanks, @ktoso. We're very happily using your sbt-jmh for circe and Finch's benchmarks, and I'll close this issue when I've added Play JSON to the circe benchmarks (which currently only compare against Argonaut).

(And right, with a little warmup circe looks faster with @crakjie's code.)

[crakjie]

Why play json don't have this "problem" ?

[ktoso]

Great, thanks @travisbrown :-)

(I've got this knee-jerk reaction when I see hand-rolled benchmarks to recommend proper tools, sorry for the noise 😉)

[travisbrown]

@crakjie There's so much stuff happening in any JVM implementation that you just can't meaningfully compare the performance of operations like this without warmup, etc.

For example, even just running the Play version first pretty much evens things out:

scala> for (i <- 1 to 3 ) { Test.time(play.api.libs.json.Json.toJson(v)) }
 elapsed time :57.044ms
 elapsed time :0.113ms
 elapsed time :0.089ms

scala> for(i <- 1 to 3) { Test.time(v.asJson) }
 elapsed time :61.344ms
 elapsed time :0.067ms
 elapsed time :0.039ms

But this still doesn't really tell you anything.