GithubHelp home page GithubHelp logo

neslib / neslib.json Goto Github PK

View Code? Open in Web Editor NEW
77.0 13.0 23.0 252 KB

Fast and memory-efficient JSON for Delphi

License: Other

Pascal 100.00%
delphi json json-library json-parser json-documents json-array jsonpath neslib

neslib.json's Introduction

Neslib.Json - Fast and memory-efficient JSON for Delphi

A fast and memory-efficient JSON object model, with support for efficiently parsing and writing in JSON-compliant format.

Dependencies

This library only depends on the Neslib repository. It is included as submodule with this repository.

Loading and Saving JSON

The main entry point to this library is the IJsonDocument interface. It is used for parsing, loading and saving JSON documents and provides access to the JSON object model. You can parse a JSON string as follows:

var
  Doc: IJsonDocument;
begin
  Doc := TJsonDocument.Parse('{ "Answer" : 42 }');
end;

Note that, unlike the official JSON specification, this library does not require quotes around dictionary keys (as long as the key does not contain spaces or other non-identifier characters). So the following is valid as well:

Doc := TJsonDocument.Parse('{ Answer : 42 }');

You can also use the Load method to load from a file or stream.

On the output side, you use Save to save to a file or stream, or ToJson to output to a JSON string.

You can also create new JSON documents from scratch using the CreateArray or CreateDictionary methods:

var
  Doc: IJsonDocument;
begin
  Doc := TJsonDocument.CreateArray;
  Doc.Root.Add(42);
end;

As you can see in this example, you access the JSON document object model through the Root property.

JSON object model

At the heart of the JSON object model is the TJsonValue type. This is a record that can hold any type of JSON value.

It provides various implicit conversion operators to convert a TJsonValue to another (Delphi) type. In addition, there are various To* methods that try to convert a TJsonValue but return a provided default value if conversion fails.

You (can) never create TJsonValue's yourself; The only way to create a TJsonValue is by adding a value to JSON array or dictionary:

var
  Doc: IJsonDocument;
begin
  Doc := TJsonDocument.CreateArray;
  Doc.Root.Add(42);
end;

This example adds a TJsonValue (with value 42) to a JSON array. To create a new array of dictionary, you use the AddArray or AddDictionary methods instead:

var
  Doc: IJsonDocument;
  Dict: TJsonValue;
begin
  Doc := TJsonDocument.CreateArray;
  Dict := Doc.Root.AddDictionary;
  Dict.AddOrSetValue('answer', 42);
end;

This creates a new dictionary and adds it to the root array. Then, the value 42 is added to this dictionary under the name 'answer'.

To check the type of a value, use the TJsonValue.ValueType property or one of the TJsonValue.Is* methods.

When trying to use methods like Add (or AddOrSetValue) on values that are not arrays (or dictionaries), an exception will be raised.

However, accessing the items in an array (using the Items property) or the values in a dictionary (using the Values property) will never result in an exception, even if the array index is out of bounds. This allows for chaining multiple array/dictionary accesses together without having to check the validity of each intermediate step. For example:

I := Doc.Root.Items[3].Values['foo'].Values['bar'].Items[4].ToInteger(0);

This will always succeed, but return 0 if any of the intermediate values are unavailable.

Manually Reading and Writing JSON

The IJsonDocument interface makes it easy to read and write JSON into a document object model.

However, you can also choose to read or write JSON manually if you prefer (for example to avoid having to load an object model into memory). You can do this with the IJsonReader and IJsonWriter interfaces in the Neslib.Json.IO unit.

These interfaces are completely independent from any DOM implementation and don't even require the Neslib.Json unit. Using these interfaces is a bit more complicated and requires some more work though. See the Neslib.Json.IO unit for more information.

Querying JSON documents with JSONPath

There is also an XPath-like JSONPath implementation you can use for querying JSON documents.

There is no official JSONPath specification, but the most widely used version seems to be one developed by Stefan Goessner.

About JSONPath

A JSONPath looks like:

$.store.book[0].title

or

$['store']['book'][0]['title']

Both representation are identical: you can use either dot (.) or bracket ([]) notation to denote children of a dictionary. Brackets can also be used with numerical indices to denote children of an array by index.

JSONPath only uses single quotes (') within brackets. We also allow for double quotes (") since these are easier to use in Delphi strings.

In short:

  • Every path starts with a $ indicating the root, followed by zero or more child operators (. or []). A $ by itself matches the entire document.
  • A child name can be an identifier string or the asterisk (* or '*') wildcard to match all children. For example, $.store.book[*].author matches the authors of all books in the store.
  • In addition to a single dot (.), a double dot (..) can be used to search for any descendants instead of immediate children. For example, $..author matches all authors, regardless of depth. This is called recursive descent.
  • Children can also be accessed by one or more indices between brackets. These indices are 0-based and are only used with arrays. You can separate multiple indices with comma's. For example, $.store.book[0,2,3] matches the first, third and fourth books.
  • You can use the slice notation [Start:End:Step] to match a slice (range) of children. This matches all children from index Start up to (but not including) End, using a given Step size (usually 1). All are optional, but at least one value (and colon) must be given:
    • If Start is omitted, it is implied to be 0. A negative value indicates an offset from the end of the array.
    • If End is omitted, the slice extracts through the end of the array. A negative value indicates and offset from the end of the array.
    • If Step is omitted, is is implied to be 1.
    • Examples:
      • List[2:] matches the third and all following elements.
      • List[-2:] matches the last two elements.
      • List[:2] matches the first two elements.
      • List[:-2] matches all but the last two elements.
      • List[2:-2] matches all elements but the first two and last two.
      • List[-4:-2] matches the 3rd and 4rd elements from the end.
      • List[::2] matches all elements with an even index.

JSONPath also has an @ operator to allow custom script expressions. We do not support this operator.

JSONPath Examples

Example document:

{ "store": {
    "book": [
      { "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      { "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  }
}

Example paths:

Expression Result
$ Matches the root document (a single value)
$..* Matches all members in the document (lots of values)
$.store.book[*].author The authors of all books in the store
$..author All authors
$.store.* All things in store (2 books and a bicycle)
$.store..price The price of everything in the store
$..book[2] The third book
$..book[-1:] The last book in order
$..book[:2] The first two books

JSONPath in Delphi

The JSONPath API is short and simple. It consists of a TJsonPath record with only a couple of methods.

For one-off matching, use the static Match method:

var
  Doc: IJsonDocument;
  Matches: TArray<TJsonValue>;
begin
  Doc := TJsonDocument.Load(...);
  Matches := TJsonPath.Match(Doc, '$.store.book[*].author');
end;

If you plan to use the same path on multiple (sub)documents, then it is faster to parse the path once, and then apply it multiple times:

var
  Doc1, Doc2: IJsonDocument;
  Path: TJsonPath;
  Matches1, Matches2: TArray<TJsonValue>;
begin
  Doc1 := TJsonDocument.Load(...);
  Doc2 := TJsonDocument.Load(...);

  Path := TJsonPath.Create('$.store.book[*].author');
    
  Matches1 := Path.Match(Doc1);
  Matches2 := Path.Match(Doc2);
end;

You can also run the path on sub-trees:

var
  Doc: IJsonDocument;
  Store: TJsonValue;
  Matches: TArray<TJsonValue>;
begin
  Doc := TJsonDocument.Load(...);
  Store := Doc.Root.Values['store'];
  Matches := TJsonPath.Match(Store, '$.book[*].author');
end;

If you are only interested in a single (or the first) match, then you can use MatchSingle instead:

var
  Doc: IJsonDocument;
  Match: TJsonValue;
begin
  Doc := TJsonDocument.Load(...);
  if (TJsonPath.MatchSingle(Store, '$.book[*]', Match)) then
    ...
end;

Memory Management

All memory management in this JSON library is automatic. An IJsonDocument interface owns all TJsonValue's and destroys them when the document is destroyed (goes out of scope).

The only thing you need to be aware of is that you shouldn't use any TJsonValue records anymore after the document is destroyed. Doing so will lead to undefined behavior and possibly crashes.

Customization

You can customize some behavior using these conditional defines:

  • JSON_UTF8: to use UTF8String instead of String everywhere. All strings will be treated as 8-bit UTF-8 strings instead of 16-bit Unicode strings. This reduces memory consumption and speeds up parsing a bit. However, this means you will have to use this JSON library with UTF8Strings as well, otherwise Delphi will implicitly convert between Unicode strings and UTF8Strings, which can hurt performance.
  • JSON_STRING_INTERNING: to enable string interning for dictionary keys. This reduces memory consumption in case the same key is used a lot of times (which is common when JSON is exported from a database), but is a bit slower.

The Neslib.Json unit declares the JsonString type as either String or UTF8String, depending on the JSON_UTF8 define. However, this doesn't mean that you have to use JsonString as well. If you don't care about the JSON_UTF8 define, then you can just use regular strings with this library.

License

Neslib.Json is licensed under the Simplified BSD License.

See License.txt for details.

neslib.json's People

Contributors

erikvanbilsen avatar neslib avatar vincentparrett avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

neslib.json's Issues

Problem with stringifying a JSON string consisting of Chinese characters

The following code fails in a Delphi 11.1/Windows 64 Bit application with the conditional JSON_UTF8 defined in the Neslib.Json.* units:

program jsonwriter_err;

{$APPTYPE CONSOLE}

{$R *.res}

uses
System.SysUtils, Neslib.Json;

var
doc: IJsonDocument;
s1, s2: utf8string;

begin
s1 := '{"s":"クランクケース/全面カバー"}';
doc := TJsonDocument.Parse(s1);
s2 := doc.Root.ToJson(false);
WriteLn(Utf8ToString(s2));
ReadLn;
end.

Call stack:

:00007FFD1609CD29 ; C:\WINDOWS\System32\KERNELBASE.dll
System._RaiseAtExcept(???,???)
System.SysUtils.AssertErrorHandler('Assertion fehlgeschlagen','... Neslib.Json.IO.pas',1586,$712F5D)
System._Assert(???,???,???)
Neslib.Json.IO.TJsonWriter.WriteEscapedString('クランクケース/全面カバー')
Neslib.Json.IO.TJsonWriter.WriteString('クランクケース/全面カバー')
Neslib.Json.TJsonValue.WriteTo(TJsonWriter($258AEED80A0) as IJsonWriter)
Neslib.Json.TJsonValue.TJsonDictionary.WriteTo(TJsonWriter($258AEED80A0) as IJsonWriter)
Neslib.Json.TJsonValue.WriteTo(TJsonWriter($258AEED80A0) as IJsonWriter)
Neslib.Json.TJsonValue.ToJson(False)

Stringifying JSON array without indentation triggers error

program StringifyJsonArray;

{$APPTYPE CONSOLE}

{$R *.res}

uses
System.SysUtils,
Neslib.Json {latest version};

var
doc: IJsonDocument;
u: utf8string;

begin
try
doc := TJsonDocument.Parse('{"i":[-1,0,1]}');
u := doc.ToJson(false); // works if parameter is changed to true; tested with Delphi 11.1 and Win64 as target
WriteLn(Utf8ToString(u));
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
ReadLn;
end.

JSON Path - Match error

Hi,
The following JPATH example $.book[10].author will raise error if the child 10 do not exists.
I think that will be better to only return an empty array (no match) or at least to have the ability to check if the path exists before execution.
There is any way to check if path exists?

Thanks,
Paul

Example needed

Example needed
Unclear how to get / iterate a dictionary
ex: JSONDoc.Root.Values['test'].IsDictionary= true
How to iterate this dictionary ... there is no method such as
for Key in Dictionary.Keys do

JPath unable to escape "."

Hi,

How can I escape "." in the front/ end of an element?

I have the following JSON:
{
"error_summary": "expired_access_token/",
"error": {
".tag": "expired_access_token"
}
}

I need to select ".tag" using JPath
using https://jsonpath.com/ I select the element with $.error.['.tag']
Using Neslib.Json I have the following error "Child operator in JSON path is missing a member name"

Thank you in advance,

EAsertionFailed -> line 1280 Assert((UIntPtr(P) and TYPE_MASK) = 0);

Hi,

There is a problem Neslib.Json.pas on Linux, Android, and other platforms except Windows

The following lines will raise an Assertion Failed Error (line 1280 - Assert((UIntPtr(P) and TYPE_MASK) = 0); )

FJSONData:=TJsonDocument.CreateDictionary;
FJSONData.Root.AddOrSetValue('data','0'); <- will raise ASSERTION ERROR only if the VALUE IS one character long ON Android, Linux ... EXCEPT Windows

and in Neslib.Json

class function TJsonValue.Create(const AValue: JsonString): TJsonValue;
var
P: Pointer;
begin
P := nil;
JsonString(P) := AValue; // Increases ref count
Assert((UIntPtr(P) and TYPE_MASK) = 0); <- here
Result.FBits := TYPE_STR or UIntPtr(P);
end;

NOTE:

  1. I use delphi 10.4.2,
    http://docwiki.embarcadero.com/RADStudio/Sydney/en/Zero-based_strings_(Delphi)

(Workaround)
2. If I Typecast AValue to PChar the APP is working JsonString(P) := PChar(AValue); // Increases ref count
This change will impact the code in any other place?

Thank you,
Paul

Neslib vs GrijjyFoundation

Looking for a json parser that fully support jsonpath - and there seems to be some similarity between neslib and GrijjyFoundation (same developers?) - so which should I chose? This one seems to have more recent commits (well to the json code anyway).

TJsonValue: array property defaults (suggestion)

Suggestion is to add these two (or similar) to TJsonValue...

    property Default[const AIndex: Integer]: TJsonValue read GetItem; default;
    property Default[const AName: JsonString]: TJsonValue read GetValue; default;

To enable this:

  const doc = TJsonDocument.Parse('{ "Numbers" : ["Zero", "One", "Two", "Three"]}');

  const old = doc.Root.Values['Numbers'].Items[2];
  const new = doc.Root['Numbers'][2];

Can't get a value as string nested more than three levels deep.

If I try and retrieve a value that is nested more than four or more level deep as a string, I get an exception, is this expected behavior?

example

{ "store": {
"book": [
{ "category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95,
"Sale" : {
"OnSale" : "Yes"
}
},
{ "category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
}
}

Retrieving $.store.book[0].Sale.OnSale as toJson works, however if I try asString it does not. toJson returns {
"OnSale" : "Yes"
}, though!

procedure TForm5.Button3Click(Sender: TObject);
var
Matches: TArray;
begin
JSONDoc:=TJsonDocument.Parse(memo1.Text);
matches:=TJsonPath.Match(JSONDoc,'$.'+edit1.Text);
label1.Caption:=matches[0].ToJson();
label1.Caption:=matches[0].ToString();
end;

UTF8 encoding

I am not 100% certain to make the framework work with UTF8 content. I was under the impression that UTF8 would be used if the the string provided to parse is prepared correctly.
So I use this:

 LDoc := TJsonDocument.Parse( TEncoding.UTF8.GetString( ABytesArray ) );

Still, my content does not show unicode content correctly in a VCL app. So, I am wondering where I am not configuring it correctly.

Neslib.Utf8 - Compile error on Linux64: [Get|Reset|Set]MXCSR

Linux64:

[DCC Error] Neslib.Utf8.pas(1685): E2003 Undeclared identifier: 'GetMXCSR'
[DCC Error] Neslib.Utf8.pas(1687): E2003 Undeclared identifier: 'ResetMXCSR'
[DCC Error] Neslib.Utf8.pas(1783): E2003 Undeclared identifier: 'GetMXCSR'
[DCC Error] Neslib.Utf8.pas(1785): E2003 Undeclared identifier: 'SetMXCSR'

Windows64:

[dcc64 Warning] Neslib.Utf8.pas(1685): W1002 Symbol 'GetMXCSR' is specific to a platform
[dcc64 Warning] Neslib.Utf8.pas(1687): W1002 Symbol 'ResetMXCSR' is specific to a platform
[dcc64 Warning] Neslib.Utf8.pas(1783): W1002 Symbol 'GetMXCSR' is specific to a platform
[dcc64 Warning] Neslib.Utf8.pas(1785): W1002 Symbol 'SetMXCSR' is specific to a platform
[dcc64 Warning] Neslib.Utf8.pas(1853): W1002 Symbol 'SetMXCSR' is specific to a platform

UnitTest fails when decimal symbol is comma

My Windows is set to Icelandic regional settings and we use comma as a decimal separator.
Tried with decimal symbol as a dot and a comma and the test fails with a comma decimal separator
To Change The Format Goto: Control Panel -> Region -> Additional Settings -> Decimal Symbol

Tests Found : 187
Tests Ignored : 0
Tests Passed : 181
Tests Leaked : 0
Tests Failed : 6
Tests Errored : 0

Failing Tests

Tests.Neslib.Json.IO.TestJsonData.test_array_04
Message: Expected .=[]
.[0]=1
.[1]="abc"
.[2]=12.3
.[3]=-4 is not equal to actual .=[]
.[0]=1
.[1]="abc"
.[2]=12,3
.[3]=-4

Tests.Neslib.Json.IO.TestJsonData.test_basic_03
Message: Expected .=1.2345678 is not equal to actual .=1,2345678

Tests.Neslib.Json.IO.TestJsonData.test_real_04
Message: Expected .=1.2345678 is not equal to actual .=1,2345678

Tests.Neslib.Json.IO.TestJsonData.test_real_05
Message: Expected .=1234567.8 is not equal to actual .=1234567,8

Tests.Neslib.Json.IO.TestJsonData.test_real_06
Message: Expected .=-1.2345678 is not equal to actual .=-1,2345678

Tests.Neslib.Json.IO.TestJsonData.test_real_07
Message: Expected .=-1234567.8 is not equal to actual .=-1234567,8

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.