Hello. It looks like escaping characters in regex is problematic

#[test]
fn regex_filter_test(){
    let json: Box<Value> = Box::new(json!({
        "author":"d(Rees)",
    }));

    let path: Box<JsonPathInst> = Box::from(
        JsonPathInst::from_str("$.[?(@.author ~= 'd(Rees)')]").expect("the path is correct"),
    );
    let finder = JsonPathFinder::new(json.clone(), path);
    assert_eq!(finder.find_slice(),vec![Slice(&json!({"author":"d(Rees)",}))]);
}

assertion left == right failed left: [NoValue] right: [Slice(Object {"author": String("d(Rees)")})]

If you try to escape with \ it won't work either
'd\(Rees\)' won't compile, expecting an atom
'd\\(Rees\\)' compiles but it will look for \ in value and () is still treated like a capture group
And if you don't escape the () is considered a capture group

given json:

[
{"verb": "TEST","a":[1,2,3]},
{"verb": "TEST","a":[1,2,3]},
{"verb": "TEST"}, {"verb": "RUN"}
]

given jpath:
$.[?(@.verb == 'TEST')].a.length()

Result: [2]
Expected result: [3,3]

In the Goessner implementation the Json Path query can return either value or path. The Java implementation that we used to use has this feature too.
Would it be possible to add this feature to this library?

Hi, I am new to rust and jsonpath, so I am not sure whether this is a bug or not.
When comparing field values using logical expression I need to always check if the field is present or not.
If I use a logical expression with a field that does not exist, jsonpathfinder will return the whole string as a match.
See the example below:

use jsonpath_rust::JsonPathFinder;
use serde_json::json;

// This returns null as expected
fn main() {
    let finder = JsonPathFinder::from_str(r#"{"first":{"second":[{"active":1},{"passive":1}]}}"#, "$.first[?(@.does_not_exist && @.does_not_exist >= 1.0)]").unwrap();
    let res_js = finder.find();
    assert_eq!(res_js, json!(null));
}

// This returns the full string as a match
fn main() {
    let finder = JsonPathFinder::from_str(r#"{"first":{"second":[{"active":1},{"passive":1}]}}"#, "$.first[?(@.does_not_exist >= 1.0)]").unwrap();
    let res_js = finder.find();
    assert_eq!(res_js, json!(null));
}

Here is my Cargo toml

[package]
name = "jsonpath_test"
version = "0.1.0"
edition = "2021"

[dependencies]
jsonpath-rust = "0.3.0"
serde = {version = "1.0.160", features = ["derive"]}
serde_json = "1.0"

Context

After reading some bugfixes and playing around with edge cases, I think it would be great if the library could handle an invalid json path as an error instead of returning an empty array. This would avoid some bugs or confusing results I think. The specs recommend to return an empty array or false, but I think this is because of the lack of types at this moment (javascript / php). I think the version by the Internet engineering task force is the most recent one but it's globally the same as the initial version.

An alternative for jsonPath to return false in case of no match may be to return an empty array in future. [This is already done in the above.]

Here "no match" seems to cover both empty results and non-existent paths. It leaves a gray area on the implementation details.

Example

Let's take this Value as the data source:

let json_obj = json!({
      "shop": {
        "orders": [
          { "id": 1, "active": true },
          { "id": 2 },
          { "id": 3 },
          { "id": 4, "active": true }
        ],
        "past_orders": [ "AB01", "AB02" ],
        "returned_orders": []
      }
    });

length

Note: I don't know if the last fixes for length function have been released or not. So maybe some of the results here are already solved.

We can't differentiate an empty result from an non-existent path. Both json paths will return Array [ Number(0) ]

1. $.shop.inexistant[*].length()
2. $.shop.returned_orders[*].length()

I would expect an Err(JsonPathNotFound) instead for 1. (or similar).

false positives/negatives

It's more specific to the implementation I'm doing with Grillon but it's also the case for native assertions with the Rust test lib. I can't tell an empty result from a non-inexistent path.

If we keep the initial data source, it's all good. Let's say we want to verify AB01 and AB02 are not in returned_orders, the assertions passes :

let path = "$.shop.returned_orders[*]";
let val = json_obj.path(path).unwrap();
let should_not_be = json!(["AB01", "AB02"]);
assert_ne!(val, should_not_be);

println!(" val: {val:#?} \n should not be: {should_not_be:#?}");

warning: `jpath-test` (bin "jpath-test") generated 1 warning
    Finished dev [unoptimized + debuginfo] target(s) in 0.86s
     Running `target/debug/jpath-test`
 val: Array [] 
 should not be: Array [
    String("AB01"),
    String("AB02"),
]

But now let's remove returned_orders from the data, the result is the following:

warning: `jpath-test` (bin "jpath-test") generated 1 warning
    Finished dev [unoptimized + debuginfo] target(s) in 0.90s
     Running `target/debug/jpath-test`
 val: Array [] 
 should not be: Array [
    String("AB01"),
    String("AB02"),
]

We observe a false positive. We think there is no issue but in reality the path doesn't exist, the result isn't empty, the data doesn't contain what we're looking at. Instead I would like to handle it as an Error and produce meaningful information to my users. Like raising a warning, since it's not completely false, you don't have any returned orders but the fact that it's missing from the result shouldn't be ignored.

Conclusion

I don't think I have a way to make distinction right now. I also think it causes some bugs in the interpretation of results in the library and by users. The advantage of a strongly typed language is that we can take advantage on the type system to return meaningful information. I wonder if it would be possible to handle non-existent json paths in the library? Do we have any cases where handling an error could prevent basic functionality from working? If not, can we work on a new version?

Since the calculation happens in memory, there is a pivotal drawback that ensues with the processing of the big files.
But probably before all, the good idea is to attempt to measure it

I plan to use your library for Grillon and I was wondering if you would agree to add an extra step in your workflow so we can match the tags here with the deployed versions on crates.io? It's currently hard to follow what was integrated in your deployed version.

It's fairly simple in terms of process, but it needs to follow the trunk based development (+ continuous delivery which requires a human action). Each time you send a tag on the main branch, it triggers a github action to deploy on crates.io. Since tagging is a human action I found it quite good for controlling deployments while automating them.

Also github tag UI comes with release notes that you can use to attach to your tag and automatically link to milestones. It's great to get all the details of a tag 😄 .

Let me know and I can help you to setup this - you will need the crates.IO token to setup as a secret of your git repo.

Json doc:
[ {"verb", "TEST"}, {"verb", "RUN"}]

Json path:
$.[?(@.verb=='Test')]

Returns error:

|
1 | $.[?(@.verb=='Test')]
| ^---
|
= expected string_qt"'

The same expression tested out with online tools like this one: https://jsonpath.com/.

Hi,

Thanks for your work :)

To the best of my knowledge, python does not have a well maintained JSONPath library. Maybe it could be a good idea to create python bindings so that this library could be used from python.

the lib not support like this

{
    "data": [1, 2, 3]
}

$.data.length => [ 3 ]

how can i do？

JsonPathFinder seems to be redundant. Probably, better to revise the API a remove it

Have you seen the crate:json_dotpath?
This isn’t implementing the extensive search functionality that your JsonPathFindercan perform, but it might provide you with some ideas.
Maybe you could implement a trait over Value itself (taking either an &str or perhaps even a conversion trait (call it C) of sorts)? Then your user could simply call value.find(&str) or value::.find(C) and get a Result back?
That would avoid you having to expose JsonPathFinder to the user. Similarly, you could then implement two traits, one for finding references and one that returns mutable references?
All the user would need to ‘know’ then to use your crate would be how to construct the query language for finding those references. Like the dot_path crate, you could also utilise Serde’s DeserializedOwned trait to extend this beyond Value to any implementor of that trait. That would then be very flexible indeed. As I said, people may disagree, and I only scanned the code; just a few suggestions really. Hope that helps you along the way!

came from LinkedIn comments from Michael Radley

I want to use jsonpath-rust for pathfinding in jsonb project, which is a binary JSON format. But the parser module is private, can we make the parser module public so that it can be used independently by downstream projects?

Another issue is that JsonPathIndex relies on serde_json::Value, which does not seem necessary. Whether it is possible to use basic types to define the relevant fields?

It's helpful for debugging, when I try to create a JsonPathFinder and it fails I cannot easily debug the error.

Hey. I'm using this library at work and I have noticed some slowness when using regex, looks like creating a Regex is somehow slow. I have tested the theory using a global map for storing the compiled regex and it improved the speed ~50x. The implementation is not what we want, we should create the regex when creating the jsonpath and use it directly. I'm not sure how we can do that.

Here is the changes I made for testing:
https://github.com/catalinstochita/jsonpath-rust

Let me know if you have an idea of how we can do that. Thank you @besok

I'm trying this in online json-path evaluators and both i tried working. Maybe i'm doing something wrong? Basically need to express the following: "select all nodes with child node X"

I am not able to use jsonpath that has bool or null values, even though it is mentioned in README. For example this code based on example:

use serde_json::{json,Value};
use jsonpath_rust::json_path_value;
use jsonpath_rust::JsonPathFinder;
use jsonpath_rust::JsonPathValue;

fn main(){
    let  finder = JsonPathFinder::from_str(r#"{"first":{"second":[{"active":1},{"passive":1}]}}"#, "$.first.second[?(@.active == true)]").unwrap();
    let slice_of_data:Vec<JsonPathValue<Value>> = finder.find_slice();
    let js = json!({"active":1});
    assert_eq!(slice_of_data, json_path_value![&js,]);
}

It is based on the example. Notice the jsonpath string is: $.first.second[?(@.active == true)]
The error when running is:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: " --> 1:30\n  |\n1 | $.first.second[?(@.active == true)]\n  |                              ^---\n  |\n  = expected atom"', src/main.rs:7:139

Looks like this case is not added to grammar rules

Currently parse_json_path uses unwrap in a couple of places which is far from good practice.
Shift to normal process handling with Either and introduce an internal parser error

Placeholder to myself or somebody to find some time to refactor the module in the following points

• replace all expect with ok_or(..)
• refactor grammar (it feels like we need to squash some levels)
• rewrite ast prasing. (it feels there is the implementation that heavily relies on the knowledge of the grammar itself which can be painful to handle in some cases)
• ensure that all possible cases are covered with tests

If the object does not exist length() still returns 0. 0 is a valid length of an array and this can be a source of bugs. Current behaviour is:

let json: Box<Value> = Box::new(json!([{"verb": "TEST"},{"verb": "TEST"}, {"verb": "RUN"}]));
let path: Box<JsonPathInst> = Box::from(
    JsonPathInst::from_str("$.not.exist.length()").expect("the path is correct")
);
let finder = JsonPathFinder::new(json, path);
assert_eq!(finder.find(), json!([0]);

I would like to propose to return Value::Null in case of unsupported or unknown objects.
I prepared a PR that can fix it: #24

Hello 👋

I wanted to ask if it's possible to do something following with this crate.
I want to be able to run multiple queries on some JSON to eventually find a node that I'd like to modify/remove.
In the following example, I'm trying to remove the genre of the second book from the list.

use std::str::FromStr;
use serde_json::{Value};
use jsonpath_rust::{JsonPathFinder, JsonPathInst, JsonPathQuery};

fn main() {
    let data = r#"
    {
        "books": [
            {
                "title": "Title 1",
                "details": {
                    "description": "Description 1",
                    "genre": "Genre 1"
                }

            },
            {
                "title": "Title 2",
                "details": {
                    "description": "Description 2",
                    "genre": "Genre 2"
                }
            }
        ]
    }
  "#;

    let json: Value = serde_json::from_str(data).unwrap();
    let books_path = JsonPathInst::from_str("$.books").unwrap();
    let finder = JsonPathFinder::new(Box::from(json), Box::new(books_path)); // had to clone due to ownership

    let books = finder.find_slice().get(0).unwrap().clone().to_data();

    let second_book = books.get(1).unwrap();
    let genre_value = second_book.clone().path("$.details.genre").unwrap(); // had to clone due to ownership in path

    // remove genre_value
}

Do you have any suggestion how to achieve this?
If I'm not mistaken find returns new Values and find_slice return reference to the Value. However none of them return mutable reference, maybe adding it could help with this?
Or maybe once the feature mentioned in #31 is added I could find the paths that are to be removed/modified, clone whole "original" json and modify it using these paths 🤔
If you have any other idea please point me to the right direction 🙏

Thanks!

The following example does not work correctly:
Json:

[{"verb": "TEST"}]

Jsonpath:

$.[?(@.verb == 'TEST')].length()

Result of this query will be Null, while correct result should be 1 as one item was found.

It looks like this happens because if length of result is 1 we can not determine if we should return length of the object of length of the result array: https://github.com/besok/jsonpath-rust/blob/master/src/path/top.rs#L99

Hi!

I'm evaluating integration of jsonpath-rust into my app fhttp. I don't understand why JsonPathFinder.find returns a Vec<&Value> instead of just a Value, which could be a Value::Array. As someone integrating the lib and not knowing the data or the jsonpath expression used at compile time, i can't know whether i need to unpack the vec or not.

Why was it designed this way?

Not sure something like [?(@.category=='fiction' && @.price < 10)] support?

The current interface exposed to applications wants to own both the JSON value and the path. It also allows for updating the path and/or value. I would rather not have to pass ownership of my data to the library and simply borrow the data (both pre-parsed path and JSON value). The foundation for this functionality exists internally through a combination of parse_json_path, JsonPath, json_path_instance, and PathInstance.

By making those public on my own branch, I can get code like this to work:

for item in json_path_instance(&path, &value).find(&value).into_iter() {
  // ...
}

But clearly this is not a great user experience since value is redundantly provided to the library. I suggest a public interface be created using these internals.

Given the following input document:

  "a": {
      "b": {
          "c": {
              "d": true
          }
      }
  },

The query:
$.a[?(@.b.c.d)] returns Some

But
$.a[?(@.b.c.d==true)] returns None

Provide an update by path or return a mutable borrow.

example:

[?(!(...))]