rxi / json.lua Goto Github PK
View Code? Open in Web Editor NEWA lightweight JSON library for Lua
License: MIT License
A lightweight JSON library for Lua
License: MIT License
Any chance you could provide the benchmark results in a section of the README? It helps to evaluate the options out there. I appreciate how concise and portable your module is.
% lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio
> json=require'json'
> json.encode(json.decode('{}'))
[]
Why doesn't return '{}'? (Maybe related to #19)
Thanks,
Consider the following benchmark which encodes the deeply nested structure [1000, [999, [998, [...]]]]
:
local json = require"json"
for _ = 1, 1e3 do
local t = {}
for i = 1, 1e3 do
t = {i, t}
end
json.encode(t)
end
this should run in a fraction of a second. Instead, it takes ~2 seconds on my device. This is because string concatenations / recursive table.concat
are internally used to encode the JSON:
return "[" .. table.concat(res, ",") .. "]"
This means that the outer loop will have to add the [
and ]
brackets in time linear in table.concat(res, ",")
, which in our example will be just marginally shorter, which in turn will only differ in length by a constant. Thus you do a linear time operation linearly often, resulting in worst-case quadratic encoding runtime in the depth of the nested structure.
The proper fix is to use a shared rope and have all recursive calls write to the same rope. You then only concatenate strings once at the end. Building the string then runs in linear time. You can find an example of this approach in my JSON implementation.
in lua:
> dump( json.decode("{1: 1, '1':2}"))
{ [1] = 1,["1"] = 2,}
whereas in javacript we have:
> Object.entries({1: 1, '1':2}).length
1
so in lua we should probably have:
> dump( json.decode("{1: 1, '1':2}"))
{["1"] = 2,}
I'm using Tengine. How to add this lib for lua?
local z = {
[2] = 3
}
local testVar = {
[1] = {"2", "3"},
[31] = z
}
local str = json.encode(testVar)
print(str)
expected ':' after key at line 1 col 5
Please publish json.lua on LuaRocks, the de facto standard Lua packages repository. This issue has been already raised in #3, but the submitter closed it without any reason.
I've tried to wrap json.decode
within a pcall
yet it is not succeeding as expected. Below is the minimum code that can produce this issue; using jit (lua 5.1) built within love2d 11.3
local json = require("json")
local str = [[{"foo":"bar"}]]
local contents = json.decode(str) -- works as expected
local success, contents = pcall(json.decode(str)) -- fails, success returns false
if not success then
error(contents) -- Throws "attempt to call a table value"
end
The error message attempt to call a table value
. This might be just my poor understanding of pcall
as it's one of the first times I've started to use it.
Hi,
This example seems to cause errors:
{"this":["is", "a", "minimal", "example"]}
It gives an error at symbol 10; the opening square bracket.
If table contains function, this happens: " json.lua:130: unexpected type 'function' "
If I comment the line 170 it works, if not :
json.lua:170: unexpected character 's' at line 1 col 1
stack traceback:
[C]: in function 'error'
/home/.../json.lua:170: in function 'decode_error'
/home/.../json.lua:368: in function 'parse'
/home/.../json.lua:376: in function 'decode'
const.lua:566: in function 'json_test'
const.lua:572: in main chunk
[C]: in function 'dofile'
...urin/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: in ?
Also, how to read a json file in lua and write the result of encode back to file?
When used .. in loops, the lua will create a lot of garbage and thus affect performance. I changed your script (json-fix.lua)
c:\test>busybox diff -U 0 json.lua json-fix.lua
--- json.lua
+++ json-fix.lua
@@ -41 +41 @@
-local json = { _version = "0.1.2" }
+local json = { _version = "0.1.2fix" }
@@ -235 +235 @@
- local res = ""
+ local res = {}
@@ -246 +246 @@
- res = res .. str:sub(k, j - 1)
+ table.insert(res, str:sub(k, j - 1))
@@ -253 +253 @@
- res = res .. parse_unicode_escape(hex)
+ table.insert(res, parse_unicode_escape(hex))
@@ -259 +259 @@
- res = res .. escape_char_map_inv[c]
+ table.insert(res, escape_char_map_inv[c])
@@ -264,2 +264,2 @@
- res = res .. str:sub(k, j - 1)
- return res, j + 1
+ table.insert(res, str:sub(k, j - 1))
+ return table.concat(res), j + 1
I created a .HAR file (size 313971312 bytes) using firefox and tested it with a simple Lua script.
local function read(file)
local f, err = io.open(file, "r");
if not f then
return;
else
local data = f:read("a");
f:close();
return (data);
end
end
local function test(obj)
print (obj._version)
local last = os.time();
local har = obj.decode(read(file));
print ('decode - ' .. os.time() - last .. " sec")
end
json = require "json";
jsonfix = require "json-fix";
file = 'Archive [21-05-25 11-43-43].har';
test(json)
test(jsonfix)
I got results like this on my computer (Intel(R) Core(TM) i3-9100 CPU @ 3.60GHz)
c:\test>lua read_har.lua
0.1.2
decode - 145 sec
0.1.2fix
decode - 23 sec
c:\test>lua read_har.lua
0.1.2
decode - 144 sec
0.1.2fix
decode - 24 sec
c:\test>lua read_har.lua
0.1.2
decode - 147 sec
0.1.2fix
decode - 22 sec
Currently the lib does not seem to support nested tables a.k.a. unlimited depth, e.g.
luaTable = { innerTable = { value1 = "value", value2 = "value" }, value1 = "value }
won't be supported currently. So I would like to request to support any kind of nested structure by using recursion.
A web service I need to use is returning the following in its JSON:
"Id":1618065507111835497
But when using this Lua library to decode it, the result is losing precision as "Id" is returned as:
1.6080655071118e+18
In this case I can't change what the webservice is providing. I'm quite happy referring the data as a string - it's just an id value.
I am limited to only using Lua 5.1.4.
Any help appreciated (e.g. pass back numbers that can't be correctly converted as strings?).
Thanks,
Rob
Suggestion / Request to use annotated tags for version tag numbers.
This is considered "best practice" by a number of open source projects and annotated tags have a number of advantages for official releases.
For our purposes, it will simplify the way that we use 'git describe' to collect information about packages that are a part of our build. By default, git describe ignores "lightweight tags"
Many thanks!
--
Please see a related request here:
inotify-tools/inotify-tools#124
local json_before = {"key":[{},{},{},{},{},{},{},{},{},{}],"roleAttr":{"3":0,"4":0,"5":0,"6":0,"7":0,"8":0,"9":0,"0":0,"1":0,"2":0}}
local list = json.decode(json_before );
local json_after = json.encode(list);
此时的json_afre为:{"key":[[],[],[],[],[],[],[],[],[],[]],"roleAttr":{"1":0,"0":0,"3":0,"2":0,"5":0,"4":0,"7":0,"6":0,"9":0,"8":0}}
为何不一样?
Are you going to distribute json.lua as a Luarock module? It seems to me, it would be very convenient - just add json.lua as a luarock dependency.
I installed nginx-full and trying to use json.lua with NGINX:
Example:
nginx.conf:
location /test { default_type 'text/plain'; content_by_lua_block { local json = require("json") ngx.say(json.decode('[1,2,3,{"x":10}]')) }
then curl http://localhost:8080/test and got 500 error. But everything is ok in lua console:
`> json = require("json")
json.encode({ 1, 2, 3, { x = 10 } })
[1,2,3,{"x":10}]`
What I am doing wrong?
Output is all on one line with \n instead of a return
and the return statement is still there.
local treeFile = io.open("TreeData/"..treeVersion.."/tree.lua", "r")
if treeFile then
treeText = treeFile:read("*a")
treeFile:close()
f = io.open("TreeData/"..treeVersion.."/tree.json", "w")
f:write(json.encode(treeText))
f:close()
end
What is the problem with parsing it like this?
local function encode_table(val, stack)
local res = {}
stack = stack or {}
-- Circular reference?
if stack[val] then error("circular reference") end
stack[val] = true
local isArray = true
for k in pairs(val) do
if type(k) ~= "number" then
isArray = false
break
end
end
if isArray then
-- Encode
for i=1, #val do
table.insert(res, encode(val[i], stack))
end
stack[val] = nil
return "[" .. table.concat(res, ",") .. "]"
else
-- Treat as an object
for k, v in pairs(val) do
if type(k) ~= "string" then
error("invalid table: mixed or invalid key types")
end
table.insert(res, encode(k, stack) .. ":" .. encode(v, stack))
end
stack[val] = nil
return "{" .. table.concat(res, ",") .. "}"
end
end```
And then allow numbers for keys too, is there a technical reason why it shouldn't be done?
Strings like '0123' or '1.' are invalid as described in ECMA-404 and can not be parsed to numbers. However, the function tonumber()
regards such strings convertible to numbers.
So the following code:
json.decode('1.')
will return the value 1 without any errors, which go against the JSON standard.
I have a lua object containing a large number:
local obj = {prop=1925036013156063406}
When the object is serialized, the JSON looks like:
{"prop":1.9250360131561e+18}
and can not be parsed by other JSON libraries on other programming languages.
The correct serializaion output would be:
{"prop":1925036013156063406}
For some reason json4lua's decode is commented out on bench_decode:
https://github.com/rxi/json.lua/blob/master/bench/bench_decode.lua#L8
When I uncomment it out, it locks up.
What's going on?
Parsing such document
{ "dt": 1548784800, "main": { "temp": -2.05, "temp_min": -2.18, "temp_max": -2.05, "pressure": 998.5, "sea_level": 1019.79, "grnd_level": 998.5, "humidity": 89, "temp_kf": 0.12 }, "weather": [ { "id": 800, "main": "Clear", "description": "clear sky", "icon": "01n" } ], "clouds": { "all": 36 }, "wind": { "speed": 1.56, "deg": 233.002 }, "rain": {}, "snow": { "3h": 0.004 }, "sys": { "pod": "n" }, "dt_txt": "2019-01-29 18:00:00" }
gives an error:
invalid number '-2.05' at line 4 col 17
Parsing JSON with empty arrays does not work.
Example:
local json = loadfile("json.lua")()
d = json.decode("{"tags":[]}")
local fmt = string.format
print( fmt("[test] %s ", d.tags) )
I am new to lua.
A case, i was built a nginx + luajit +redis2.
I was used json.lua and got follow errer message:
2021/02/07 08:42:02 [error] 37#37: *11 failed to run body_filter_by_lua*: /etc/nginx/redis.lua:238: bad argument #1 to 'rawget' (table expected, got string)
stack traceback:
[C]: in function 'rawget'
/etc/nginx/redis.lua:238: in function 'decode'
/etc/nginx/ueditor/ueditor_upload_file_rewrite_response.lua:22: in function </etc/nginx/ueditor/ueditor_upload_file_rewrite_response.lua:1>
code fragments
if eof then
local whole = table.concat(ngx.ctx.buffered)
local ret_body = whole
if whole then
ngx.ctx.buffered = nil
local body_json = json.decode(whole) #################### line number 22 ####################
local ret_table = {}
if body_json then
ret_table['original'] = body_json['originalFilename']
ret_table['size'] = body_json['size']
ret_table['state'] = 'SUCCESS'
ret_table['title'] = body_json['originalFilename']:match("(.+)%.%w+$")
ret_table['type'] = body_json['file']:match(".+(%.%w+)$")
local imageUrl = body_json['thumbnail']
if imageUrl and imageUrl ~= "" then
ret_table['url'] = imageUrl
else
ret_table['url'] = body_json["file"]
end
else
ret_table['state'] = 'Error: upload file failed!'
end
ret_body = json.encode(ret_table)
end
ngx.arg[1] = ret_body
end
Any help appreciated .
Thanks,
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.