jjenkov / parsers-in-java Goto Github PK

If I did not misread test sources, input is taken as java.lang.String.
I think this is incorrect and should be changed to a byte source.

This is fundamentally wrong choice, given that input for parsing practically always comes as a byte source (stream, ByteBuffer, byte array). And although the next step for textual data formats like JSON and XML typically is decoding into characters, efficiency of doing so varies a lot. Same is true for ability to stream (or not) from arbitrary length content; but I guess that may be ignored for now, and just note that certain optimizations are only possible if the exact length of input is known.

This is especially true for libraries that do allow either more efficient char decoding than what JDK offers (for XML that'd be Woodstox at least), or combine parts of decoding, validation and lexing straight from byte sources (for JSON that's FastJSON and Jackson at least).

For better take on performance comparisons in this respect, you could consider checking out:

https://github.com/eishay/jvm-serializers

which uses byte[] as the common input source: libraries that expect character-based sources can then construct either Strings or Readers.

In parse() method you assert that the json stream to parse is always an object, but the specification of the format says that json stream can be an array or an object.

I've tried to optimize things (smaller inlinable methods, switches instead of if/elseif, cached parser instances).

So far, the only thing that did performed better was turning DataCharBuffer into an interface, and add an implementation that wraps a String instead of a char array, thus saving an array copy.

This improved performance on big JSON messages of ~15%.

I'm still puzzled about your implementation being beaten by gson and json-smart on smaller messages. I suspect IndexBuffer's arrays allocation.

Hi,

I wanted to give your JSON parser a spin after reading your post on InfoQ.

Getting an ArrayIndexOutOfBoundsException with the following JSON string:

{ "store": {
    "book": [ 
      { "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      { "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      },
      { "category": "fiction",
        "author": "Herman Melville",
        "title": "Moby Dick",
        "isbn": "0-553-21311-3",
        "price": 8.99
      },
      { "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  }
}

Full stacktrace:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 715
    at com.jenkov.parsers.json.JsonTokenizer.parseStringToken(JsonTokenizer.java:67)
    at com.jenkov.parsers.json.JsonTokenizer.parseToken(JsonTokenizer.java:56)
    at com.jenkov.parsers.json.JsonParser.parseObject(JsonParser.java:43)
    at com.jenkov.parsers.json.JsonParser.parseArray(JsonParser.java:77)
    at com.jenkov.parsers.json.JsonParser.parseObject(JsonParser.java:47)
    at com.jenkov.parsers.json.JsonParser.parse(JsonParser.java:22)

Regards,

Stéphane

The first version of the parser only supported string values. It did not support numbers and booleans.