markreedz / mrjson

JSON encoder and decoder for python written in C/C++

MrJSON

MrJSON is a JSON encoder and decoder written in C/C++ with bindings for Python 2.5+ and 3. I needed faster performance for long strings and floats and MrJSON benchmarks at 2-3 times faster than the fastest python parsers for my use cases.

To install it just run Pip as usual:

    $ pip install mrjson

Usage

May be used as a replacement for json

    >>> import mrjson as json
    >>> json.dumps([{"key": "value"}, 81, True])
    '[{"key":"value"},81,true]'
    >>> json.loads("""[{"key": "value"}, 81, true]""")
    [{'key': 'value'}, 81, True]

Encoder differences

ensure_ascii defaults to true in the base json module, but defaults to false here for space and performance reasons.

indent, separators, and sortKeys are not supported as pretty printing doesn't need the performance. Write an issue if you have a use case.

allow_nan is unsupported as NaN and infinity are supported by default.

Custom objects are supported if they implement an json() method returning a valid json string.

Benchmarks

Benchmark your own files as results can vary significantly:

$ python -m timeit -s "import mrjson as json;st = open('canada.json').read();" "json.loads(st)"
100 loops, best of 3: 8.05 msec per loop

$ python -m timeit -s "import json;st = open('canada.json').read();" "json.loads(st)"
10 loops, best of 3: 32.7 msec per loop

Or run bench.py to test MrJSON against some other modules. MrJSON does particularly well decoding long strings and floating point numbers thanks to intel's AVX2 instructions and Milo Yip at Tencent for publishing C++ code implementing Florian Loitsch's float to string algorithms.

Loads

Only 128 byte long strings.

Mostly floating point numbers - canada.json from The Native JSON Benchmark, the fastest C++ JSON parser comes in at 7.9 milliseconds on this machine for comparison

A single tweet from twitter - twit.json

citm_catalog.json from The Native JSON Benchmark

Dumps

Only 128 byte long strings.

Mostly floating point numbers - canada.json from The Native JSON Benchmark

A single tweet from twitter - twit.json

citm_catalog.json from The Native JSON Benchmark

mrjson's Issues

cgi.parse_header is slow

[delete]

Unexpected character

JSONLint says the following is okay, but I'm seeing this error:

>>> j = '[[{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0},{"c":"blw_2IgLc","up":true,"type":"upv","s":0,"t":0}],{"username":"1","id":1}]'
>>> import mrjson
>>> mrjson.loads(j)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unexpected character at pos 551
>>> j[551:]
'd":1}]'

Error messages are not useful

If this was a large json file this may not be useful for figuring out the problem.

>>> import mrjson
>>> mrjson.loads("tru")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: JSON_BAD_IDENTIFIER

Negative infinity doesn't work

While looking at the tests I noticed this doesn't work and probably should?

>>> mrjson.loads("-Infinity")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: JSON_BAD_NUMBER;