ultrajson / ultrajson

Ultra fast JSON decoder and encoder written in C with Python bindings

UltraJSON

UltraJSON is an ultra fast JSON encoder and decoder written in pure C with bindings for Python 3.8+.

Install with pip:

python -m pip install ujson

Project status

Warning

UltraJSON's architecture is fundamentally ill-suited to making changes without risk of introducing new security vulnerabilities. As a result, this library has been put into a maintenance-only mode. Support for new Python versions will be added and critical bugs and security issues will still be fixed but all other changes will be rejected. Users are encouraged to migrate to orjson which is both much faster and less likely to introduce a surprise buffer overflow vulnerability in the future.

Usage

May be used as a drop in replacement for most other JSON parsers for Python:

>>> import ujson
>>> ujson.dumps([{"key": "value"}, 81, True])
'[{"key":"value"},81,true]'
>>> ujson.loads("""[{"key": "value"}, 81, true]""")
[{'key': 'value'}, 81, True]

Encoder options

encode_html_chars

Used to enable special encoding of "unsafe" HTML characters into safer Unicode sequences. Default is False:

>>> ujson.dumps("<script>John&Doe", encode_html_chars=True)
'"\\u003cscript\\u003eJohn\\u0026Doe"'

ensure_ascii

Limits output to ASCII and escapes all extended characters above 127. Default is True. If your end format supports UTF-8, setting this option to false is highly recommended to save space:

>>> ujson.dumps("åäö")
'"\\u00e5\\u00e4\\u00f6"'
>>> ujson.dumps("åäö", ensure_ascii=False)
'"åäö"'

escape_forward_slashes

Controls whether forward slashes (/) are escaped. Default is True:

>>> ujson.dumps("https://example.com")
'"https:\\/\\/example.com"'
>>> ujson.dumps("https://example.com", escape_forward_slashes=False)
'"https://example.com"'

indent

Controls whether indentation ("pretty output") is enabled. Default is 0 (disabled):

>>> ujson.dumps({"foo": "bar"})
'{"foo":"bar"}'
>>> print(ujson.dumps({"foo": "bar"}, indent=4))
{
    "foo":"bar"
}

Benchmarks

UltraJSON calls/sec compared to other popular JSON parsers with performance gain specified below each.

Test machine

Linux 5.15.0-1037-azure x86_64 #44-Ubuntu SMP Thu Apr 20 13:19:31 UTC 2023

Versions

• CPython 3.11.3 (main, Apr 6 2023, 07:55:46) [GCC 11.3.0]
• ujson : 5.7.1.dev26
• orjson : 3.9.0
• simplejson : 3.19.1
• json : 2.0.9

	ujson	orjson	simplejson	json
Array with 256 doubles
encode	18,282	79,569	5,681	5,935
decode	28,765	93,283	13,844	13,367
Array with 256 UTF-8 strings
encode	3,457	26,437	3,630	3,653
decode	3,576	4,236	522	1,978
Array with 256 strings
encode	44,769	125,920	21,401	23,565
decode	28,518	75,043	41,496	42,221
Medium complex object
encode	11,672	47,659	3,913	5,729
decode	12,522	23,599	8,007	9,720
Array with 256 True values
encode	110,444	425,919	81,428	84,347
decode	203,430	318,193	146,867	156,249
Array with 256 dict{string, int} pairs
encode	14,170	72,514	3,050	7,079
decode	19,116	27,542	9,374	13,713
Dict with 256 arrays with 256 dict{string, int} pairs
encode	55	282	11	26
decode	48	53	27	34
Dict with 256 arrays with 256 dict{string, int} pairs, outputting sorted keys
encode	42		8	27
Complex object
encode	462		397	444
decode	480	618	177	310

Above metrics are in call/sec, larger is better.

Build options

For those with particular needs, such as Linux distribution packagers, several build options are provided in the form of environment variables.

Debugging symbols

UJSON_BUILD_NO_STRIP

By default, debugging symbols are stripped on Linux platforms. Setting this environment variable with a value of 1 or True disables this behavior.

Using an external or system copy of the double-conversion library

These two environment variables are typically used together, something like:

export UJSON_BUILD_DC_INCLUDES='/usr/include/double-conversion'
export UJSON_BUILD_DC_LIBS='-ldouble-conversion'

Users planning to link against an external shared library should be aware of the ABI-compatibility requirements this introduces when upgrading system libraries or copying compiled wheels to other machines.

UJSON_BUILD_DC_INCLUDES

One or more directories, delimited by os.pathsep (same as the PATH environment variable), in which to look for double-conversion header files; the default is to use the bundled copy.

UJSON_BUILD_DC_LIBS

Compiler flags needed to link the double-conversion library; the default is to use the bundled copy.

ultrajson's Issues

ujson.loads returns str object for unicode strings

I found a minor issue, in JSONtoObj.c you call PyString_FromString, which results in a python str object that could potentially contain unicode data (which causes issues if you want to try and change the string encoding). What it should be returning is a unicode object if the string has unicode characters.

For unicode strings I think what you need to convert the string to use 16 bit characters rather than 8 bit characters and then call PyUnicode_FromUnicode. This will then return a unicode object rather than a str object.

Encoding a floating point end with zeros

I found a minor issue in encoding floating points.
A floating point that ends with zeros is encoded to an integer (e.g. 1.0 -> 1).

Consider implementing the Python layer using Cython

It would probably result in a lot less code to maintain
http://www.cython.org/

ujson.loads a big int value overflow

Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit (Intel)] onwin32
the same on x86_64 x86_64 GNU/Linux

import json, ujson
ujson.version
'1.19'

ujson.loads('[18446098363113800555]')

[-645710595751061L]

json.loads('[18446098363113800555]')

[18446098363113800555L]

Too lenient with a trailing comma in arrays

>>> import json, ujson
>>> json.loads('[42,]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
>>> ujson.loads('[42,]')
[42]

Unicode as input for the decode()

Unfortunately, ujson makes assumption that incoming string is in UTF-8 and throws exception if it is in unicode already.

Is it possible that ujson will work with unicode strings internally?

datetime types not handled correctly - and an alternative to the toDict idea

JSON doesn't specify how date/time/datetime types should be serialized. The python stdlib json module raises a TypeError if you try to serialize these types.

Currently ujson tries to serialize/deserialize datetime.date and datetime.datetime (it raises an OverflowError for datetime.time) but the result is nonstandard and I think also incorrect, e.g.

now = datetime.datetime.now()
str(now)
'2011-12-27 15:17:54.879000'
enc = ujson.dumps(now)
enc
'1324999074' # hmmm, looks like a time_t?
ujson.loads()
1324999074 # now we have just an int, not a datetime
str(datetime.datetime.fromtimestamp())
'2011-12-27 10:17:54' # why isn't this the same as now above?

Using the toDict approach used for other unknown types would be preferable, then the user could decide how they want these types to be serialized/deserialized. Unfortunatley the toDict method for these types is tricky to implement because it's difficult to monkey patch built in types.

I wonder if instead of the toDict approach it might be better in general for ujson to look for two global functions to handle serialization and deserialization of unknown types, e.g. something like

def serialize_to_json(obj):
t = type(obj)
if t == datetime.date:
return "date:%s" % obj.isoformat()
elif t == bitset:
return "bitset:%s" % obj.to01()
elif t == numpy.ndarray:
return "ndarray:%s" % repr(obj.tolist())
...

def deserialize_from_json(s):
t, rep = s.split(":")
if t == "date":
return dateutil.parse(rep)
elif t == "bitset":
return bitset(rep)
... etc...

The serialize function should be able to return anything that ujson already knows how to serialize to JSON. The deserialize function should take str/unicode and return a Python object. This is more general than the toDict approach since there is more control over the JSON representation--it doesn't have to be a dictionary/object

Yet another approach would be to try to duplicate the JSONEncoder/JSONDecoder thing that the stdlib json package does.

Any thoughts?

difference between json.dumps and ujson.dumps

>>> json.dumps({None:1})
'{"null": 1}'
>>> ujson.dumps({None:1})
'{"None":1}'

>>> ujson.__version__
'1.18'

Test failures on Python 2.5 / win32

C:\code\ultrajson>c:\python25\x32\Scripts\nosetests.exe tests
............................................EE.E.............C:\Python25\x32\lib\unittest.py:332: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if not first == second:
F.F.F..F.........
======================================================================
ERROR: test_encodeDoubleInf (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 291, in test_encodeDoubleInf
    input = float('inf')
ValueError: invalid literal for float(): inf

======================================================================
ERROR: test_encodeDoubleNan (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 282, in test_encodeDoubleNan
    input = float('nan')
ValueError: invalid literal for float(): nan

======================================================================
ERROR: test_encodeDoubleNegInf (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 300, in test_encodeDoubleNegInf
    input = -float('inf')
ValueError: invalid literal for float(): inf

======================================================================
FAIL: test_encodeToUTF8 (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 251, in test_encodeToUTF8
    self.assertEquals(enc, json.dumps(input, encoding="utf-8", ensure_ascii=False))
AssertionError: '"\xe6\x97\xa5\xd1\x88"' != u'"\u65e5\u0448"'

======================================================================
FAIL: test_encodeUnicode4BytesUTF8 (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 137, in test_encodeUnicode4BytesUTF8
    self.assertEquals(dec, json.loads(enc))
AssertionError: u'\u1030TRAILINGNORMAL' != u'\U00011030TRAILINGNORMAL'

======================================================================
FAIL: test_encodeUnicode4BytesUTF8Highest (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 146, in test_encodeUnicode4BytesUTF8Highest
    self.assertEquals(dec, json.loads(enc))
AssertionError: u'\uffffTRAILINGNORMAL' != u'\U000fffffTRAILINGNORMAL'

======================================================================
FAIL: test_encodeUnicodeSurrogatePair (tests.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\code\ultrajson\tests\tests.py", line 129, in test_encodeUnicodeSurrogatePair
    self.assertEquals(dec, json.loads(enc))
AssertionError: u'\u0346' != u'\U00010346'

----------------------------------------------------------------------
Ran 78 tests in 6.828s

FAILED (errors=3, failures=4)

C:\code\ultrajson>'C:\code\ultrajson' is not recognized as an internal or external command,
operable program or batch file.

[]] doesn't throw an error

>>> ujson.loads('[]]')
[]
>>> json.loads('[]]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 2 - line 1 column 3 (char 2 - 3)

toDict not working?

Using this test program

--- snip ---

import ujson

class thing:
def init(self):
pass
def toDict(self):
print "toDict called!"
return {"foo" : 1}

t = thing()
print ujson.dumps(t)

--- snip ---

I see that toDict is called but the returned dictionary is not used in the serialization to JSON. Am I doing something wrong?

Thanks for your help--ujson is great!

Undefined symbol: PyUnicodeUCS2_AsUTF8String

I got an issue on Debian Squeeze (however ujson works well on MacOS).

(virtualenv)# python -v
Python 2.6.7 (r267:88850, Aug  3 2011, 11:33:52) 
[GCC 4.6.1] on linux2

(virtualenv)# pip install ujson
Downloading/unpacking ujson
  Downloading ujson-1.18.zip (185Kb): 185Kb downloaded
  Running setup.py egg_info for package ujson
Installing collected packages: ujson
  Running setup.py install for ujson
    building 'ujson' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/usr/include/python2.6 -c ./python/ujson.c -o build/temp.linux-x86_64-2.6/./python/ujson.o
    ./python/ujson.c:6:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/usr/include/python2.6 -c ./python/objToJSON.c -o build/temp.linux-x86_64-2.6/./python/objToJSON.o
    ./python/objToJSON.c:70:6: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
    ./python/objToJSON.c: In function ‘PyLongToINT64’:
    ./python/objToJSON.c:101:12: warning: unused variable ‘obj’ [-Wunused-variable]
    ./python/objToJSON.c: In function ‘objToJSON’:
    ./python/objToJSON.c:695:3: warning: initialization from incompatible pointer type [enabled by default]
    ./python/objToJSON.c:695:3: warning: (near initialization for ‘encoder.beginTypeContext’) [enabled by default]
    ./python/objToJSON.c:706:3: warning: initialization from incompatible pointer type [enabled by default]
    ./python/objToJSON.c:706:3: warning: (near initialization for ‘encoder.releaseObject’) [enabled by default]
    ./python/objToJSON.c: At top level:
    ./python/objToJSON.c:85:14: warning: ‘PyIntToINT32’ defined but not used [-Wunused-function]
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/usr/include/python2.6 -c ./python/JSONtoObj.c -o build/temp.linux-x86_64-2.6/./python/JSONtoObj.o
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/usr/include/python2.6 -c ./lib/ultrajsonenc.c -o build/temp.linux-x86_64-2.6/./lib/ultrajsonenc.o
    ./lib/ultrajsonenc.c:121:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringUnvalidated’:
    ./lib/ultrajsonenc.c:199:3: warning: value computed is not used [-Wunused-value]
    ./lib/ultrajsonenc.c: At top level:
    ./lib/ultrajsonenc.c:414:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringValidated’:
    ./lib/ultrajsonenc.c:378:6: warning: ‘ucs’ may be used uninitialized in this function [-Wuninitialized]
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/usr/include/python2.6 -c ./lib/ultrajsondec.c -o build/temp.linux-x86_64-2.6/./lib/ultrajsondec.o
    ./lib/ultrajsondec.c:55:40: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:75:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:283:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:301:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:323:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:341:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:391:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c: In function ‘decode_string’:
    ./lib/ultrajsondec.c:424:14: warning: pointer targets in assignment differ in signedness [-Wpointer-sign]
    ./lib/ultrajsondec.c: At top level:
    ./lib/ultrajsondec.c:620:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c: In function ‘decode_array’:
    ./lib/ultrajsondec.c:633:4: warning: value computed is not used [-Wunused-value]
    ./lib/ultrajsondec.c: At top level:
    ./lib/ultrajsondec.c:670:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    ./lib/ultrajsondec.c:746:1: warning: ‘fastcall’ attribute ignored [-Wattributes]
    gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/./python/ujson.o build/temp.linux-x86_64-2.6/./python/objToJSON.o build/temp.linux-x86_64-2.6/./python/JSONtoObj.o build/temp.linux-x86_64-2.6/./lib/ultrajsonenc.o build/temp.linux-x86_64-2.6/./lib/ultrajsondec.o -o build/lib.linux-x86_64-2.6/ujson.so
Successfully installed ujson
Cleaning up...

I got the error at the import step

(virtualenv)# python myApp.py 
Traceback (most recent call last):
  File "myApp.py", line 7, in <module>
    import ujson
ImportError: /www/myapp/lib/python2.6/site-packages/ujson.so: undefined symbol: PyUnicodeUCS2_AsUTF8String

When I inspect ujson.so everything seem fine:

# objdump -T ujson.so 

ujson.so:     file format elf64-x86-64

DYNAMIC SYMBOL TABLE:
0000000000002c98 l    d  .init  0000000000000000              .init
0000000000000000      D  *UND*  0000000000000000              PyList_New
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.2.5 free
0000000000000000      D  *UND*  0000000000000000              _Py_ZeroStruct
0000000000000000      D  *UND*  0000000000000000              PyArg_ParseTupleAndKeywords
0000000000000000      D  *UND*  0000000000000000              PyFloat_Type
0000000000000000      D  *UND*  0000000000000000              PyObject_CallMethod
0000000000000000      D  *UND*  0000000000000000              PyUnicodeUCS2_AsUTF8String
[...]

# ldd ujson.so
    linux-vdso.so.1 =>  (0x00007fff8797e000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f32a2d45000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f32a29be000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f32a3173000)

Any ideas ?

Strange result when trying to serialize a lambda

Python 2.7 (r27:82525, Jul 4 2010, 09:01:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import ujson
ujson.dumps(lambda x : x)

'{"func_closure":null,"func_code":{"co_argcount":1,"co_cellvars":[],"co_code":"|\u0000\u0000S","co_consts":[null],"co_filename":"","co_firstlineno":1,"co_flags":67,"co_fre
evars":[],"co_lnotab":"","co_name":"","co_names":[],"co_nlocals":1,"co_stacksize":1,"co_varnames":["x"]},"func_defaults":null,"func_dict":{},"func_doc":null,"func_globals":
{"builtins":{"Ellipsis":{},"False":false,"None":null,"NotImplemented":{},"True":true},"name":"main","ujson":{},"doc":null,"package":null},"func_name":""
}'

Double escaped forward slash

I'm not sure if this is expected behavior, but ujson is add two backslashes when it receives a string with a forward slash. ujson does properly reverse this, but I'm having compatibility issues with a few other programs.

>>> ujson.dumps('forward/slash')
'"forward\\/slash"'
>>> ujson.loads(ujson.dumps('forward/slash'))
u'forward/slash'

At least the built in json and simplejson do not exhibit this behavior.

>>> json.dumps('forward/slash')
'"forward/slash"'
>>> simplejson.dumps('forward/slash')
'"forward/slash"'

Problem at installing with pip

Sys: ubuntu 12.04 updated
python 3.2

log file:

---

/usr/bin/tornado2.3_pip run on Thu Aug 2 16:00:46 2012
Downloading/unpacking ujson

Running setup.py egg_info for package ujson

running egg_info
writing pip-egg-info/ujson.egg-info/PKG-INFO
writing top-level names to pip-egg-info/ujson.egg-info/top_level.txt
writing dependency_links to pip-egg-info/ujson.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found


reading manifest file 'pip-egg-info/ujson.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'setup.y'

writing manifest file 'pip-egg-info/ujson.egg-info/SOURCES.txt'
****************************************************************************************************

If you want to run the tests be sure to run 2to3 on them first, e.g. `2to3 -w tests/tests.py`.

****************************************************************************************************

Installing collected packages: ujson

Running setup.py install for ujson

Running command /virtualenv/tornado2.3/bin/python3 -c "import setuptools;__file__='/virtualenv/tornado2.3/build/ujson/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-6sx1k9-record/install-record.txt --install-headers /virtualenv/tornado2.3/include/site/python3.2
running install
running build
running build_ext
building 'ujson' extension

creating build
creating build/temp.linux-x86_64-3.2
creating build/temp.linux-x86_64-3.2/python
creating build/temp.linux-x86_64-3.2/lib
gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC -I./python -I./lib -I/usr/include/python3.2mu -c ./python/ujson.c -o build/temp.linux-x86_64-3.2/./python/ujson.o

In file included from ./python/ujson.c:34:0:

./python/py_defines.h:34:20: erro fatal: Python.h: Arquivo ou diretório não encontrado

compilação terminada.

error: command 'gcc' failed with exit status 1

Complete output from command /virtualenv/tornado2.3/bin/python3 -c "import setuptools;__file__='/virtualenv/tornado2.3/build/ujson/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-6sx1k9-record/install-record.txt --install-headers /virtualenv/tornado2.3/include/site/python3.2:

running install

running build

running build_ext

building 'ujson' extension

creating build

creating build/temp.linux-x86_64-3.2

creating build/temp.linux-x86_64-3.2/python

creating build/temp.linux-x86_64-3.2/lib

gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC -I./python -I./lib -I/usr/include/python3.2mu -c ./python/ujson.c -o build/temp.linux-x86_64-3.2/./python/ujson.o

In file included from ./python/ujson.c:34:0:

./python/py_defines.h:34:20: erro fatal: Python.h: Arquivo ou diretório não encontrado

compilação terminada.

error: command 'gcc' failed with exit status 1

---

Command /virtualenv/tornado2.3/bin/python3 -c "import setuptools;file='/virtualenv/tornado2.3/build/ujson/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --single-version-externally-managed --record /tmp/pip-6sx1k9-record/install-record.txt --install-headers /virtualenv/tornado2.3/include/site/python3.2 failed with error code 1 in /virtualenv/tornado2.3/build/ujson

Exception information:
Traceback (most recent call last):
File "/virtualenv/tornado2.3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/basecommand.py", line 104, in main
status = self.run(options, args)
File "/virtualenv/tornado2.3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/commands/install.py", line 250, in run
requirement_set.install(install_options, global_options)
File "/virtualenv/tornado2.3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/req.py", line 1133, in install
requirement.install(install_options, global_options)
File "/virtualenv/tornado2.3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/req.py", line 577, in install
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
File "/virtualenv/tornado2.3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/init.py", line 256, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command /virtualenv/tornado2.3/bin/python3 -c "import setuptools;file='/virtualenv/tornado2.3/build/ujson/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --single-version-externally-managed --record /tmp/pip-6sx1k9-record/install-record.txt --install-headers /virtualenv/tornado2.3/include/site/python3.2 failed with error code 1 in /virtualenv/tornado2.3/build/ujson

Write tests for UTF-16 escape decoding

Should test:
Unpaired lead surrogate
Unpaired trailing surrogate
Invalid value (some UTF-16 values are invalid)
Invalid length (\uXXX)

Write tests for exponent decoding

expose ujson.__version__

It would be helpful for versioning as i integrate ujson into some other things if the ujson version number were exposed via ujson.__version__ as the string '1.4'

I would expect the version string to match up with setup.py and pypi, but setup.py has 1.0, and pypi has 1.4

Breaks when it has trailling data

When pulling data from twitter api I cant use ujson because of the trailing data. I ended up having to rstrip to remove that. Simplejson an the default json works without changes

Support encoding from UCS-4 and UTF-16 directly

Strings in either UCS-4 (Python Unicode on Linux) or UTF-16 (Python Unicode on Windows) should be possible to direct encode instead of first converting to UTF-8 and then encoding.

README.rst not included in pypi sdist

README.rst isnt included pypi sdist just released:

While updating the FreeBSD port for ujson:

===> Configuring for py27-ujson-1.21
Traceback (most recent call last):
File "setup.py", line 48, in
f = open('README.rst')
IOError: [Errno 2] No such file or directory: 'README.rst'
*** Error code 1

Unable to decode or load Google Finance JSON

URL: 'http://www.google.com/finance/info?infotype=infoquoteall&q=CI:NYSE,CINF:Nasdaq,CTAS:Nasdaq,CSCO:Nasdaq,C:NYSE,CTXS:Nasdaq,CLF:NYSE,CLX:NYSE,KO:NYSE,CCE:NYSE,CTSH:Nasdaq,CL:NYSE,CMCSA:Nasdaq,CMA:NYSE,CSC:NYSE,CAG:NYSE'

I'm relatively new to python, but I believe this may be due to UJSON not being able to escape certain parts of the Google Finance feed.

CJSON is able to decode this using cjson.decode.

Given the benchmarks, I would love to be able to use ujson for this

ujson dumps strings differently than json/simplejson

ujson appears to insert backslashes in places that other json libs dont. This api difference seems like a big deal. Can you identify it as either a bug or a feature?

>>> import ujson as json
>>> json.dumps("Hi/")
'"Hi\\/"'
>>> import simplejson as json
>>> json.dumps("Hi/")
'"Hi/"'
>>> import json
>>> json.dumps("Hi/")
'"Hi/"'

document edge cases for acceptable values

I'm using ujson 1.19. It looks like it will:

1. Permissively encode arbitrary objects into JSON dictionaries
2. Encode datetime.datetime and datetime.date to their integer POSIX timestamp
3. Refuse to encode float(nan), by raising OverflowError

Is it possible to document what values are valid inputs to ujson, and what kinds of objects have special-cased encodings? Thanks very much.

64-bit issues with encoding integers on Linux

The test "test_encodeLongNegConversion" fails on x64 Linux.

Works on x64 Windows and x86 Linux. Not sure about OSX.

Must be fixed

Problem installing on Windows 7

Can anybody help me out? I do not have this problem when building other packages (e.g. Pandas)

python setup.py build --compiler=mingw32
running build
running build_ext
building 'ujson' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\python
creating build\temp.win32-2.7\Release\lib
c:\Rtools\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -I./python -I./lib -IC:\P
ython27\include -IC:\Python27\PC -c ./python/ujson.c -o build\temp.win32-2.7\Rel
ease.\python\ujson.o
c:\Rtools\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -I./python -I./lib -IC:\P
ython27\include -IC:\Python27\PC -c ./python/objToJSON.c -o build\temp.win32-2.7
\Release.\python\objtojson.o
In file included from ./python/objToJSON.c:4:0:
./lib/ultrajson.h:83:26: error: conflicting types for 'uint32_t'
c:\rtools\mingw\bin../lib/gcc/mingw32/4.5.0/../../../../include/stdint.h:32:20:
note: previous declaration of 'uint32_t' was here
./python/objToJSON.c: In function 'PyLongToINT64':
./python/objToJSON.c:101:12: warning: unused variable 'obj'
./python/objToJSON.c: In function 'objToJSON':
./python/objToJSON.c:695:3: warning: initialization from incompatible pointer ty
pe
./python/objToJSON.c:706:3: warning: initialization from incompatible pointer ty
pe
./python/objToJSON.c: At top level:
./python/objToJSON.c:92:14: warning: 'PyIntToINT64' defined but not used
error: command 'gcc' failed with exit status 1

Support PDP Endian

I'mnot sure who uses PDP endian anymore but we should probably support it

Documentation

Would be nice if someone could contribute somekind of documentation to the module.

Like dir(ujson) from Python

Most anything is better than what we have today, so feel free to fork!

Compile error on windows - objToJSON

Downloaded the files and ran python setup.py build
It compiles the ujson.c, but fails when trying the objToJSON.c

I get this error when compiling with VS2008 - windows 7.
Running with a virtual environment (numpy installed there)

c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I./python -I./lib -IC:\Python27\Tools\virtualenv\gafront
lib\site-packages\numpy\core\include -IC:\Python27\include -IC:\Python27\Tools\virtualenv\gafront\PC /Tc./python/objToJSON.c /Fobuild\temp.win32-2.7\Release./p
ython/objToJSON.obj
objToJSON.c
./python/objToJSON.c(104) : error C2275: 'PyObject' : illegal use of this type as an expression
c:\python27\include\object.h(108) : see declaration of 'PyObject'
./python/objToJSON.c(104) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(105) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(105) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'int'
./python/objToJSON.c(105) : warning C4024: 'PyObject_GetAttrString' : different types for formal and actual parameter 1
./python/objToJSON.c(106) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(106) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'int'
./python/objToJSON.c(106) : warning C4024: 'PyObject_GetAttrString' : different types for formal and actual parameter 1
./python/objToJSON.c(107) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(107) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'int'
./python/objToJSON.c(107) : warning C4024: 'PyObject_GetAttrString' : different types for formal and actual parameter 1
./python/objToJSON.c(108) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(108) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(108) : error C2065: 'mod_frame' : undeclared identifier
./python/objToJSON.c(140) : warning C4244: 'function' : conversion from 'unsigned long' to 'npy_half', possible loss of data
./python/objToJSON.c(239) : warning C4133: '=' : incompatible types - from 'PyArrayObject *' to 'PyObject *'
./python/objToJSON.c(319) : error C2143: syntax error : missing ';' before 'type'
./python/objToJSON.c(320) : error C2065: 'i' : undeclared identifier
./python/objToJSON.c(320) : error C2065: 'i' : undeclared identifier
./python/objToJSON.c(320) : error C2065: 'i' : undeclared identifier
./python/objToJSON.c(549) : error C2275: 'Py_ssize_t' : illegal use of this type as an expression
c:\python27\include\pyport.h(170) : see declaration of 'Py_ssize_t'
./python/objToJSON.c(549) : error C2146: syntax error : missing ';' before identifier 'index'
./python/objToJSON.c(549) : error C2065: 'index' : undeclared identifier
./python/objToJSON.c(551) : error C2065: 'index' : undeclared identifier
./python/objToJSON.c(557) : error C2065: 'index' : undeclared identifier
./python/objToJSON.c(563) : error C2065: 'index' : undeclared identifier
./python/objToJSON.c(913) : warning C4133: '==' : incompatible types - from 'PyObject *' to '_typeobject *'
./python/objToJSON.c(913) : warning C4133: 'function' : incompatible types - from 'PyObject *' to 'PyTypeObject *'
./python/objToJSON.c(924) : warning C4133: '==' : incompatible types - from 'PyObject *' to '_typeobject *'
./python/objToJSON.c(924) : warning C4133: 'function' : incompatible types - from 'PyObject *' to 'PyTypeObject *'
./python/objToJSON.c(924) : warning C4133: '==' : incompatible types - from 'PyObject *' to '_typeobject *'
./python/objToJSON.c(924) : warning C4133: 'function' : incompatible types - from 'PyObject *' to 'PyTypeObject *'
error: Command "c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I./python -I./lib -IC:\Python27\Tools\vir
tualenv\gafront\lib\site-packages\numpy\core\include -IC:\Python27\include -IC:\Python27\Tools\virtualenv\gafront\PC /Tc./python/objToJSON.c /Fobuild\temp.win32
-2.7\Release./python/objToJSON.obj" failed with exit status 2

OverflowError: Maximum recursion level reached when trying to serialize function

this code:

def foo():
pass

import ujson
ujson.dumps(foo)

results in
Traceback (most recent call last):
File "test2.py", line 5, in
ujson.dumps(foo)
OverflowError: Maximum recursion level reached

API compatibility with other Python JSON libraries

If ujson included loads/dumps aliases it would be possible for users to do things like this to optionally support faster encoding/decoding:

try:
    import ujson as json
except ImportError:
    print "Using slower stdlib json library"
    import json

ValueError: Unrecognized escape sequence when decoding 'string'

Further to this thread: #44 (comment)

I've isolated a specific cause of this, which can be avoided in cjson through the 'all_unicode=True' option. Hopefully this will help with a solution.

Names such as "Abercrombie \x26 Fitch Co." cause this error in ujson:

j = json.decode(feed)

Traceback (most recent call last):
File "", line 1, in
ValueError: Unrecognized escape sequence when decoding 'string'

default parser if unknown type

This is not actually an issue but a nice to havve feature

simplejson, cjson, jsonlilb2 provides a second argument in json encoding if in case there are some datatypes that cant be encoded like:

json.dumps(obj, default=)

if some datatypes cant be encoded it will resort to the default function

Push current to PyPI?

Looks like ujson on PyPI hasn't been updated since 1.19; could a push to PyPI for the current version (1.21 as of the time of this writing) be done?

ujson.dumps eats \x00 characters

Here's simple test case to reproduce:

>>> simplejson.dumps(u"  \u0000\r\n ")
'"  \\u0000\\r\\n "'
>>> ujson.dumps(u"  \u0000\r\n ")
'"  "'

Unable to decode Google Finance JSON (cont'd)

This is meant as a continuation of the issue at the link below, which I believe was prematurely closed.

#44 (comment)

Even when I look up the " [ " with .find and limit my decode with that, I still get an error. I've tried decoding staring at the " { " as well, and I get the same thing.

Traceback (most recent call last):
File "", line 1, in
jsonobj = json.decode(info[z:y])
LookupError: unknown encoding: {

Specifically... even I tried what @mthurlin suggested in the previous thread and got this.

jsonob = json.decode(info[5:])

Traceback (most recent call last):
File "<pyshell#33>", line 1, in
jsonob = json.decode(info[5:])
LookupError: unknown encoding:
{
"id": "22144"
,"t" : "AAPL"
,"e" : "NASDAQ"
,"l" : "567.84"
,"l_cur" : "567.84"
,"s": "0"
,"ltt":"11:26AM EDT"
,"lt" : "May 22, 11:26AM EDT"
,"c" : "+6.56"
,"cp" : "1.17"
,"ccol" : "chg"
,"eo" : ""
,"delay": ""
,"op" : "569.55"
,"hi" : "573.88"
,"lo" : "565.50"
,"vo" : "11.53M"
,"avvo" : "23.76M"
,"hi52" : "644.00"
,"lo52" : "310.50"
,"mc" : "530.97B"
,"pe" : "13.84"
,"fwpe" : ""
,"beta" : "1.26"
,"eps" : "41.02"
,"shares" : "935.06M"
,"inst_own" : "69%"
,"name" : "Apple Inc."
,"type" : "Company"
}
,{
"id": "284784"
,"t" : "INTC"
,"e" : "NASDAQ"
,"l" : "26.06"
,"l_cur" : "26.06"
,"s": "0"
,"ltt":"11:26AM EDT"
,"lt" : "May 22, 11:26AM EDT"
,"c" : "-0.09"
,"cp" : "-0.35"
,"ccol" : "chr"
,"eo" : ""
,"delay": ""
,"op" : "26.27"
,"hi" : "26.28"
,"lo" : "25.77"
,"vo" : "20.45M"
,"avvo" : "40.23M"
,"hi52" : "29.27"
,"lo52" : "19.16"
,"mc" : "131.10B"
,"pe" : "11.04"
,"fwpe" : ""
,"beta" : "1.07"
,"eps" : "2.36"
,"shares" : "5.03B"
,"inst_own" : "62%"
,"name" : "Intel Corporation"
,"type" : "Company"
}
]

Support decoding to UCS-4 and UTF-16 directly

When decoding the incoming string should be able to be directly decoded to UTF-8, UCS-4 or UTF-16 depending on configuration, Python and Platform.

Support Big and Little Endian

ujson dump python set() as an empty object.

It does not make sense, and does not behavior like the built-in json module.

>>> import ujson
>>> ujson.dumps(set())
'{}'
>>> ujson.dumps(set([1, 2, 3]))
'{}'
>>> import json
>>> json.dumps(set())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: set([]) is not JSON serializable

ujson.loads("3") returns string instead of ValueError

Is this expected behavior?

ujson.loads("3")
3

How is "3" a valid JSON? It should throw a ValueError like the case of:

ujson.loads("asdsad asdads")
Traceback (most recent call last):
File "", line 1, in
ValueError: Expected object or value

Using Python 2.7.3 ujson version 1.23

Support dump/load functions (Python interface)

Most of Python serialization modules (e.g. standard json, pickle or third party simplejson, phpserialize) include following four functions:

• dump(obj, file): Serializes an obj into the file which is file object.
• dumps(obj): Serializes an obj into a string and returns it.
• load(file): Unserializes from the file which is file object.
• loads(string): Unserializes from the string.

But ujson currently supports only two of these. Following the de facto standard serialization module interface would be useful because the interface make ujson able to be interchangable and a drop-in replacement for another JSON modules or serialization modules. For example:

try:
    import ujson as json # use the faster one if exists
except ImportError:
    import json # use the standard module as counterpart
import sys

print json.load(sys.stdin)

Generators support

Ultrajson doesn't support generators yet, therefore we have to consume more memory/CPU explicitly converting generator objects to lists before dumping to JSON (the mainstream Python json module has the same problem, but it's not optimized anyway, so it won't matter whether it dumps data for 30s or 30.1s). This is easy to fix, and I'm attaching the patch fixing this problem. Please let me know if I need to improve the patch or include additional stuff.

P.S. If you don't want to include the patch, just let me know - I'll fork the project, but I would really like to see the changes in the original project, because now we have to modify our unit-tests to monkey-patch ujson (unit-tests environment is using the original version of ujson, and unfortunately we can't modify the environment so easily).

Make it possible to specify output format from encoder to be UTF-8 or ASCII

The current implementation will JSON Unicode escape (\uXXXX) anything that isn't ASCII (7-bit).

The code has no problem to support UTF-8 output and might even become faster from doing this. Languages with many characters outside the ASCII/Latin alphabet would also benefit in size.

How would the Python interface for this be implemented?

Documentation in ultrajson.h is mostly BS

Needs a major overhaul indeed

Installing with pip/easy_install yields lots of warnings

If these warnings are safe to ignore, why are they printed?
Built with gcc/Xcode on Mac OS X.

Installing collected packages: ujson
  Running setup.py install for ujson
    building 'ujson' extension
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I./python -I./lib -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c ./python/ujson.c -o build/temp.macosx-10.6-universal-2.6/./python/ujson.o
    ./python/ujson.c:6: warning: function declaration isn’t a prototype
    ./python/ujson.c:6: warning: function declaration isn’t a prototype
    ./python/ujson.c:6: warning: function declaration isn’t a prototype
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I./python -I./lib -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c ./python/objToJSON.c -o build/temp.macosx-10.6-universal-2.6/./python/objToJSON.o
    In file included from ./python/objToJSON.c:4:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    ./python/objToJSON.c:71: warning: function declaration isn’t a prototype
    ./python/objToJSON.c: In function ‘PyLongToINT64’:
    ./python/objToJSON.c:101: warning: unused variable ‘obj’
    ./python/objToJSON.c: In function ‘objToJSON’:
    ./python/objToJSON.c:695: warning: initialization from incompatible pointer type
    ./python/objToJSON.c:706: warning: initialization from incompatible pointer type
    ./python/objToJSON.c: At top level:
    ./python/objToJSON.c:93: warning: ‘PyIntToINT64’ defined but not used
    ./python/objToJSON.c:71: warning: function declaration isn’t a prototype
    ./python/objToJSON.c: In function ‘PyLongToINT64’:
    ./python/objToJSON.c:101: warning: unused variable ‘obj’
    ./python/objToJSON.c: In function ‘objToJSON’:
    ./python/objToJSON.c:695: warning: initialization from incompatible pointer type
    ./python/objToJSON.c:706: warning: initialization from incompatible pointer type
    ./python/objToJSON.c: At top level:
    ./python/objToJSON.c:93: warning: ‘PyIntToINT64’ defined but not used
    In file included from ./python/objToJSON.c:4:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    ./python/objToJSON.c:71: warning: function declaration isn’t a prototype
    ./python/objToJSON.c: In function ‘PyLongToINT64’:
    ./python/objToJSON.c:101: warning: unused variable ‘obj’
    ./python/objToJSON.c: In function ‘objToJSON’:
    ./python/objToJSON.c:695: warning: initialization from incompatible pointer type
    ./python/objToJSON.c:706: warning: initialization from incompatible pointer type
    ./python/objToJSON.c: At top level:
    ./python/objToJSON.c:86: warning: ‘PyIntToINT32’ defined but not used
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I./python -I./lib -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c ./python/JSONtoObj.c -o build/temp.macosx-10.6-universal-2.6/./python/JSONtoObj.o
    In file included from ./python/JSONtoObj.c:2:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    In file included from ./python/JSONtoObj.c:2:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I./python -I./lib -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c ./lib/ultrajsonenc.c -o build/temp.macosx-10.6-universal-2.6/./lib/ultrajsonenc.o
    In file included from ./lib/ultrajsonenc.c:37:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringUnvalidated’:
    ./lib/ultrajsonenc.c:199: warning: value computed is not used
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringValidated’:
    ./lib/ultrajsonenc.c:216: warning: ‘ucs’ may be used uninitialized in this function
    ./lib/ultrajsonenc.c:121: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringUnvalidated’:
    ./lib/ultrajsonenc.c:199: warning: value computed is not used
    ./lib/ultrajsonenc.c: At top level:
    ./lib/ultrajsonenc.c:414: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringValidated’:
    ./lib/ultrajsonenc.c:216: warning: ‘ucs’ may be used uninitialized in this function
    In file included from ./lib/ultrajsonenc.c:37:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    ./lib/ultrajsonenc.c:121: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringUnvalidated’:
    ./lib/ultrajsonenc.c:199: warning: value computed is not used
    ./lib/ultrajsonenc.c: At top level:
    ./lib/ultrajsonenc.c:414: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsonenc.c: In function ‘Buffer_EscapeStringValidated’:
    ./lib/ultrajsonenc.c:216: warning: ‘ucs’ may be used uninitialized in this function
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I./python -I./lib -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c ./lib/ultrajsondec.c -o build/temp.macosx-10.6-universal-2.6/./lib/ultrajsondec.o
    In file included from ./lib/ultrajsondec.c:37:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    ./lib/ultrajsondec.c: In function ‘decode_string’:
    ./lib/ultrajsondec.c:424: warning: pointer targets in assignment differ in signedness
    ./lib/ultrajsondec.c: In function ‘decode_array’:
    ./lib/ultrajsondec.c:633: warning: value computed is not used
    ./lib/ultrajsondec.c:55: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:75: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:283: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:301: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:323: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:341: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:391: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c: In function ‘decode_string’:
    ./lib/ultrajsondec.c:424: warning: pointer targets in assignment differ in signedness
    ./lib/ultrajsondec.c: At top level:
    ./lib/ultrajsondec.c:620: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c: In function ‘decode_array’:
    ./lib/ultrajsondec.c:633: warning: value computed is not used
    ./lib/ultrajsondec.c: At top level:
    ./lib/ultrajsondec.c:670: warning: ‘fastcall’ attribute directive ignored
    ./lib/ultrajsondec.c:746: warning: ‘fastcall’ attribute directive ignored
    In file included from ./lib/ultrajsondec.c:37:
    ./lib/ultrajson.h:122:1: warning: "__LITTLE_ENDIAN__" redefined
    <built-in>: warning: this is the location of the previous definition
    ./lib/ultrajsondec.c:55: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:75: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:283: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:301: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:323: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:341: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:391: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c: In function ‘decode_string’:
    ./lib/ultrajsondec.c:424: warning: pointer targets in assignment differ in signedness
    ./lib/ultrajsondec.c: At top level:
    ./lib/ultrajsondec.c:620: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c: In function ‘decode_array’:
    ./lib/ultrajsondec.c:633: warning: value computed is not used
    ./lib/ultrajsondec.c: At top level:
    ./lib/ultrajsondec.c:670: warning: ‘fastcall’ attribute ignored
    ./lib/ultrajsondec.c:746: warning: ‘fastcall’ attribute ignored

Publish Python extension on PyPi

ujson crashes with Fatal Python error: GC object already tracked

Hey, here is a testcase I wrote to demonstrate this crash.

#ujson_test.py

import ujson

json_str = '[{"name" : "vitaly"}]'
print ujson.loads(json_str)

print "##########################################"

json_str = "[{'name' : 'vitaly'}]"
print ujson.loads(json_str)

I am on Ubuntu 10.10 and when I run this locally this is what I get:

(ujson) vbabiy@vbabiy-ubuntu ~ ❯❯ python ujson_test.py 
[{'name': 'vitaly'}]
##########################################
Traceback (most recent call last):
  File "ujson_test.py", line 9, in <module>
    print ujson.loads(json_str)
ValueError: Unexpected character in found when decoding array value
*** glibc detected *** python: corrupted double-linked list: 0x00000000015e6920 ***
======= Backtrace: =========
/lib/libc.so.6(+0x774b6)[0x7f304a0b44b6]
/lib/libc.so.6(+0x7a921)[0x7f304a0b7921]
/lib/libc.so.6(cfree+0x73)[0x7f304a0bac83]
python(PyDict_Fini+0x28)[0x44c288]
python(Py_Finalize+0x15f)[0x4c68af]
python(Py_Main+0x486)[0x417b46]
/lib/libc.so.6(__libc_start_main+0xfe)[0x7f304a05bd8e]
python[0x4172c9]
======= Memory map: ========
00400000-0061a000 r-xp 00000000 08:05 4199234                            /home/vbabiy/.virtualenvs/hydra/bin/python
00819000-0081a000 r--p 00219000 08:05 4199234                            /home/vbabiy/.virtualenvs/hydra/bin/python
0081a000-0087c000 rw-p 0021a000 08:05 4199234                            /home/vbabiy/.virtualenvs/hydra/bin/python
0087c000-0088b000 rw-p 00000000 00:00 0 
0153d000-01634000 rw-p 00000000 00:00 0                                  [heap]
7f3044000000-7f3044021000 rw-p 00000000 00:00 0 
7f3044021000-7f3048000000 ---p 00000000 00:00 0 
7f3049765000-7f304977a000 r-xp 00000000 08:05 8257615                    /lib/libgcc_s.so.1
7f304977a000-7f3049979000 ---p 00015000 08:05 8257615                    /lib/libgcc_s.so.1
7f3049979000-7f304997a000 r--p 00014000 08:05 8257615                    /lib/libgcc_s.so.1
7f304997a000-7f304997b000 rw-p 00015000 08:05 8257615                    /lib/libgcc_s.so.1
7f304997b000-7f304998c000 r-xp 00000000 08:05 2890291                    /usr/lib/python2.6/lib-dynload/datetime.so
7f304998c000-7f3049b8c000 ---p 00011000 08:05 2890291                    /usr/lib/python2.6/lib-dynload/datetime.so
7f3049b8c000-7f3049b8d000 r--p 00011000 08:05 2890291                    /usr/lib/python2.6/lib-dynload/datetime.so
7f3049b8d000-7f3049b91000 rw-p 00012000 08:05 2890291                    /usr/lib/python2.6/lib-dynload/datetime.so
7f3049b91000-7f3049b98000 r-xp 00000000 08:05 4200828                    /home/vbabiy/.virtualenvs/hydra/lib/python2.6/site-packages/ujson.so
7f3049b98000-7f3049d97000 ---p 00007000 08:05 4200828                    /home/vbabiy/.virtualenvs/hydra/lib/python2.6/site-packages/ujson.so
7f3049d97000-7f3049d98000 r--p 00006000 08:05 4200828                    /home/vbabiy/.virtualenvs/hydra/lib/python2.6/site-packages/ujson.so
7f3049d98000-7f3049d99000 rw-p 00007000 08:05 4200828                    /home/vbabiy/.virtualenvs/hydra/lib/python2.6/site-packages/ujson.so
7f3049d99000-7f304a03d000 r--p 00000000 08:05 2622855                    /usr/lib/locale/locale-archive
7f304a03d000-7f304a1b7000 r-xp 00000000 08:05 8257841                    /lib/libc-2.12.1.so
7f304a1b7000-7f304a3b6000 ---p 0017a000 08:05 8257841                    /lib/libc-2.12.1.so
7f304a3b6000-7f304a3ba000 r--p 00179000 08:05 8257841                    /lib/libc-2.12.1.so
7f304a3ba000-7f304a3bb000 rw-p 0017d000 08:05 8257841                    /lib/libc-2.12.1.so
7f304a3bb000-7f304a3c0000 rw-p 00000000 00:00 0 
7f304a3c0000-7f304a442000 r-xp 00000000 08:05 8257848                    /lib/libm-2.12.1.so
7f304a442000-7f304a641000 ---p 00082000 08:05 8257848                    /lib/libm-2.12.1.so
7f304a641000-7f304a642000 r--p 00081000 08:05 8257848                    /lib/libm-2.12.1.so
7f304a642000-7f304a643000 rw-p 00082000 08:05 8257848                    /lib/libm-2.12.1.so
7f304a643000-7f304a659000 r-xp 00000000 08:05 8257730                    /lib/libz.so.1.2.3.4
7f304a659000-7f304a859000 ---p 00016000 08:05 8257730                    /lib/libz.so.1.2.3.4
7f304a859000-7f304a85a000 r--p 00016000 08:05 8257730                    /lib/libz.so.1.2.3.4
7f304a85a000-7f304a85b000 rw-p 00017000 08:05 8257730                    /lib/libz.so.1.2.3.4
7f304a85b000-7f304a9be000 r-xp 00000000 08:05 8257557                    /lib/libcrypto.so.0.9.8
7f304a9be000-7f304abbe000 ---p 00163000 08:05 8257557                    /lib/libcrypto.so.0.9.8
7f304abbe000-7f304abcb000 r--p 00163000 08:05 8257557                    /lib/libcrypto.so.0.9.8
7f304abcb000-7f304abe4000 rw-p 00170000 08:05 8257557                    /lib/libcrypto.so.0.9.8
7f304abe4000-7f304abe7000 rw-p 00000000 00:00 0 
7f304abe7000-7f304ac32000 r-xp 00000000 08:05 8257572                    /lib/libssl.so.0.9.8
7f304ac32000-7f304ae31000 ---p 0004b000 08:05 8257572                    /lib/libssl.so.0.9.8
7f304ae31000-7f304ae33000 r--p 0004a000 08:05 8257572                    /lib/libssl.so.0.9.8
7f304ae33000-7f304ae39000 rw-p 0004c000 08:05 8257572                    /lib/libssl.so.0.9.8
7f304ae39000-7f304ae3b000 r-xp 00000000 08:05 8257846                    /lib/libutil-2.12.1.so
7f304ae3b000-7f304b03a000 ---p 00002000 08:05 8257846                    /lib/libutil-2.12.1.so
7f304b03a000-7f304b03b000 r--p 00001000 08:05 8257846                    /lib/libutil-2.12.1.so
7f304b03b000-7f304b03c000 rw-p 00002000 08:05 8257846                    /lib/libutil-2.12.1.so
7f304b03c000-7f304b03e000 r-xp 00000000 08:05 8257857                    /lib/libdl-2.12.1.so
7f304b03e000-7f304b23e000 ---p 00002000 08:05 8257857                    /lib/libdl-2.12.1.so
7f304b23e000-7f304b23f000 r--p 00002000 08:05 8257857                    /lib/libdl-2.12.1.so
7f304b23f000-7f304b240000 rw-p 00003000 08:05 8257857                    /lib/libdl-2.12.1.so
7f304b240000-7f304b258000 r-xp 00000000 08:05 8257843                    /lib/libpthread-2.12.1.so
7f304b258000-7f304b457000 ---p 00018000 08:05 8257843                    /lib/libpthread-2.12.1.so
7f304b457000-7f304b458000 r--p 00017000 08:05 8257843                    /lib/libpthread-2.12.1.so
7f304b458000-7f304b459000 rw-p 00018000 08:05 8257843                    /lib/libpthread-2.12.1.so
7f304b459000-7f304b45d000 rw-p 00000000 00:00 0 
7f304b45d000-7f304b47d000 r-xp 00000000 08:05 8257849                    /lib/ld-2.12.1.so
7f304b524000-7f304b5d7000 rw-p 00000000 00:00 0 
7f304b5d8000-7f304b65f000 rw-p 00000000 00:00 0 
7f304b67a000-7f304b67d000 rw-p 00000000 00:00 0 
7f304b67d000-7f304b67e000 r--p 00020000 08:05 8257849                    /lib/ld-2.12.1.so
7f304b67e000-7f304b67f000 rw-p 00021000 08:05 8257849                    /lib/ld-2.12.1.so
7f304b67f000-7f304b680000 rw-p 00000000 00:00 0 
7fffd906d000-7fffd908f000 rw-p 00000000 00:00 0                          [stack]
7fffd9122000-7fffd9123000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted

Feature request: Decimal support

Trying to encode a python Decimal raises OverflowError:

import ujson
from decimal import Decimal
ujson.dumps(Decimal(1.2))
Traceback (most recent call last):
File "", line 1, in
OverflowError: Maximum recursion level reached

Would be great to add a "use_decimal=True" parameter as in simplejson 2.1+

Small floats become zero

I found that small numbers are encoded as zero. Is it an intended behavior?
Example:

>>> ujson.dumps(1e-6)
'0.0'

Json in python2.7 is not doing such rounding:

>>> json.dumps(1e-6)
'1e-06'

Support for OrderedDict?

Any way to support ordered dicts? Is it even possible (efficiently) from C as it is a PyObject that overrides iter so without slow python calls it would probably be impossible unless iter implementation would be moved to C (detect object type and act accordingly)?

It seems that no C json encoder supports it and this is probably the reason. Is that right?