simeji / jid

json incremental digger

jid

Json Incremental Digger

It's a very simple tool.
You can drill down JSON interactively by using filtering queries like jq.

Suggestion and Auto completion of this tool will provide you a very comfortable JSON drill down.

Demo

Installation

• With HomeBrew (for macOS)
• With MacPorts (for macOS)
• With pkg (for FreeBSD)
• With scoop (for Windows)
• Other package management system
• Simply use "jid" command
• Build

With HomeBrew (for macOS)

brew install jid

With MacPorts (for macOS)

sudo port install jid

With pkg (for FreeBSD)

pkg install jid

With scoop (for Windows)

scoop install jid

Other package management systems

Jid can install by package management systems of below OS.

Simply use "jid" command

If you simply want to use jid command, please download binary from below.

https://github.com/simeji/jid/releases

Build

go install github.com/simeji/jid/cmd/jid@latest

Usage

Quick start

• simple json example
• simple json example2
• with initial query
• with curl

simple json example

Please execute the below command.

echo '{"aa":"2AA2","bb":{"aaa":[123,"cccc",[1,2]],"c":321}}'| jid

then, jid will be running.

You can dig JSON data incrementally.

When you enter .bb.aaa[2], you will see the following.

[Filter]> .bb.aaa[2]
[
  1,
  2
]

Then, you press Enter key and output [1,2] and exit.

simple json example2

This json is used by demo section.

echo '{"info":{"date":"2016-10-23","version":1.0},"users":[{"name":"simeji","uri":"https://github.com/simeji","id":1},{"name":"simeji2","uri":"https://example.com/simeji","id":2},{"name":"simeji3","uri":"https://example.com/simeji3","id":3}],"userCount":3}}'|jid

With a initial query

First argument of jid is initial query. (Use JSON same as Demo)

with curl

Sample for using RDAP data.

curl -s http://rdg.afilias.info/rdap/domain/example.info | jid

Load JSON from a file

jid < file.json

Keymaps

key	description
TAB / CTRL + I	Show available items and choice them
CTRL + W	Delete from the cursor to the start of the word
CTRL + U	Delete whole query
CTRL + F / Right Arrow (:arrow_right:)	Move cursor a character to the right
CTRL + B / Left Arrow (:arrow_left:)	Move cursor a character to the left
CTRL + A	To the first character of the 'Filter'
CTRL + E	To the end of the 'Filter'
CTRL + J	Scroll json buffer 1 line downwards
CTRL + K	Scroll json buffer 1 line upwards
CTRL + G	Scroll json buffer to bottom
CTRL + T	Scroll json buffer to top
CTRL + N	Scroll json buffer 'Page Down'
CTRL + P	Scroll json buffer 'Page Up'
CTRL + L	Change view mode whole json or keys (only object)
ESC	Hide a candidate box

Option

option	description
First argument ($1)	Initial query
-h	print a help
-help	print a help
-version	print the version and exit
-q	Output query mode (for jq)
-M	monochrome output mode

jid's Issues

Please propose it to homebrew-core

So users don't have to tap.

panic on invalid json

$ echo '' | jid
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x469292]

goroutine 1 [running]:
panic(0x513220, 0xc42000a0e0)
        /usr/local/Cellar/go/1.7.1/libexec/src/runtime/panic.go:500 +0x1a1
github.com/simeji/jid.(*JsonManager).GetFilteredData(0x0, 0x0, 0x0, 0xc420039b00, 0xc420039be0, 0x43bb3b, 0xc420039be0, 0x42fdce, 0xc420039bb8, 0x4a51d1, ...)
        /Users/ts/go/src/github.com/simeji/jid/json_manager.go:63 +0x42
github.com/simeji/jid.(*JsonManager).GetPretty(0x0, 0x0, 0x0, 0x0, 0xbf00000004,
0x7f1c030000000a30, 0x1a13110001000415, 0xc420039cc8, 0x425d33, 0xc420039c98, ...)
        /Users/ts/go/src/github.com/simeji/jid/json_manager.go:54 +0x66
github.com/simeji/jid.(*Engine).getContents(0xc420090000, 0x544748, 0x543513, 0x543512)
        /Users/ts/go/src/github.com/simeji/jid/engine.go:160 +0x5d
github.com/simeji/jid.(*Engine).Run(0xc420090000, 0x0, 0x0)
        /Users/ts/go/src/github.com/simeji/jid/engine.go:92 +0xb4
main.run(0x5babe0, 0xc420090000, 0x5bab00, 0xc420090000)
        /Users/ts/go/src/github.com/simeji/jid/cmd/jid.go:24 +0x49
main.main()
        /Users/ts/go/src/github.com/simeji/jid/cmd/jid.go:19 +0xc5

.. not supported

Seems to either hang or not allow input

include `Keymaps` section in `man`page or `jid -h`

Don't exit on Enter

Can we change the way the program exits when you hit the 'Enter' button? Maybe make the default exit ctrl-C?

Pressing 'Enter' should select the given auto-complete suggestion.

Add support for []

Cannot do .[]

Replace gox with goreleaser

Go Releaser lets you do release management (generation of binaries, rpms, debs, brew taps, change logs etc) with a single yaml file.
You can also put in godownloader to generate an install script that can be used for downloading release binaries from github releases. It is made by the same goreleaser team

For inspiration you can take a look at the awesome golangci-lint project

Bottom line: Installation and release management becomes way cooler 😎

Feature request: Scroll with key arrows and pageup-down

Arrow up/down are not being used, why not map them to Ctrll+j/k. Same for page up/down

Support for less strict JSON

Could we get support for JSON that isn't so strictly formatted:

{
  foo: "bar",
  // Comment for good measure
  baz: "foo",
}

In the wild JSON is going to appear like this a lot.

make `jid -q` default

often, you want to be piping this to jq, and not as the final output

Swallows keystrokes when editing line, no error reporting

I'm exploring a json document consisting of an array of arrays grouped by a key. [[{"k":1,},{"k":1}],[{"k":2}]]. As I query > .[0] and try appending [*]. If I go back and try to add . just after the initial .[0] it overwrites the line before it. It seems that in order to edit an earlier part of the line, I need to always rewrite the line that follows.

The issue may be exacerbated by the behavior that filters are not evaluated until a whole new component is recognized, so it's hard to discover syntax because of a slower feedback loop.

Dot in keyname clashes with drilldown token

The drilldown token . can appear in the key name of the json object and will break drilling down

> echo '{ "a.a": 123 }' | jid

[Filter]> .
{    
  "a.a": 123
}

[Filter]> .a.a
 null

[Filter]> ."a.a"    
 null

The first prompt suggested the autocomplete .a.a which doesn't work.

How To Use Wildcard In JSON Path

How can I use wildcards in queries?

For example consider this.

echo '{"users":[{"name":"s1","id":1},{"name":"s2","id":2}]}'|jid 

I can query users[0].id or users[1].id.

How can I get all the ids? I tried users[*].id which didn’t work.

In Clojure I use specter for this which is able to handle wildcards using ALL.

Here is a use case for this. Suppose I get the JSON from AWS using awscli containing all the EC2 instances. I want to extract the public IP addresses of all the instances.

allow for `jid -q file.json`, instead of `cat file.json | jid -q`

Support for jq functions (eg `keys`)

Thanks for releasing this - I'm a big fan :)

Is there interest in covering off more of the jq builtins?

For instance:

echo '{"abc": 1, "abcd": 2, "Foo": 3}' | jq 'keys'
["Foo", "abc", "abcd"]

jid prevents me typing keys at all at the moment.

version string is old

To correct the version, I'm needing to do

    # Fix version
    inreplace "cmd/jid/jid.go", "VERSION = \"0.6.2\"",
                                "VERSION = \"#{version}\""

Is this something I should be passing via LDFLAGS or should it bumped in the code each time it's tagged? Thanks!

Doesn't work with empty string key

Completion or just input don't work for empty string key. With jq I would use this expression: .[""]

Here is an example JSON:

{
    "": "empty key: success!",
}

add keystroke to scroll to the beginning and to the end of the input json data (CTRL+*)

Doesn't work with keys containing periods.

Autocomplete and selection break down when the keys contain periods. Example use case:

{
  "hosts": {
    "192.168.0.1": "a",
    "192.168.1.1": "b"
  }
}

Support for CTRL + U

If jid support CTRL + U to clear the query, it will be more awesome.

Reading a file by passing it as first argument

Hi, just a matter to make it easier to use, if you can remove the "<" to open a file.
Also, if you jid file or jid then nothing happens but it hangs.

How to show the array length?

Could you support the feature to show the array length? Similar to jq .users | length.

Feature request: Raw search

What?

When reading big jsons, sometimes is nice to be able to just do a plain search, as you would do with less: Press / time string to search and enter in search mode.

For instance, to search for all cases with http://

Suggested behaviour

• Press / or ? to enter search mode.
• n/N to navigate next/prev occurrences
• Enter select current search occurrence and populate filter with the corresponding path
• Esc exit search path

Considerations:

• Search should be scoped to current selection in filter

Loaded JSON file is one-line form inside jid

Hello,

I tried jid and this tool looks really great, thx for that.
I have question about formatting. When I tried the example from readme I see all the time json in one-line format:

$ echo '{"aa":"2AA2","bb":{"aaa":[123,"cccc",[1,2]],"c":321}}' | jid
[Filter]> .
{"aa":"2AA2","bb":{"aaa":[123,"cccc",[1,2]],"c":321}}

I tried to use python -m json.tool but still the same result:

echo '{"aa":"2AA2","bb":{"aaa":[123,"cccc",[1,2]],"c":321}}'| python -m json.tool | jid

add leading dot

instead of silently ignoring everything except the dot character, just start with it inserted, or don't require it at all

ubuntu14.04(windows subsystem) panic: invalid argument

just copy and paste the simple json example2 in ubuntu14.04(windows subsystem)

al123xiaaaa@HASEE-PC:/jid$ ls -l
total 3584
-rwxrwxrwx 1 al123xiaaaa al123xiaaaa 3044060 Dec 6 23:17 jid_linux_amd64
al123xiaaaa@HASEE-PC:/jid$ echo '{"info":{"date":"2016-10-23","version":1.0},"users":[{"name":"simeji","uri":"https://github.com/simeji","id":1},{"name":"simeji2","uri":"https://example.com/simeji","id":2},{"name":"simeji3","uri":"https://example.com/simeji3","id":3}],"userCount":3}}'| ./jid_linux_amd64
panic: invalid argument

goroutine 1 [running]:
panic(0x514ee0, 0xc42000a7d0)
/usr/local/Cellar/go/1.7.1/libexec/src/runtime/panic.go:500 +0x1a1
github.com/simeji/jid.(*Engine).Run(0xc4200b6000, 0x0, 0x0)
/Users/ts/go/src/github.com/simeji/jid/engine.go:87 +0x7f3
main.run(0x5babe0, 0xc4200b6000, 0x534000, 0x1)
/Users/ts/go/src/github.com/simeji/jid/cmd/jid/jid.go:36 +0x49
main.main()
/Users/ts/go/src/github.com/simeji/jid/cmd/jid/jid.go:31 +0x10f

Add coloreful output like in jq

Colorful output is really missing.
This feature could be optional like in jq (-C colorize output, -M monochrome output). It's not vital, but would be a really nice to have.

Performance very slow on 16MB file (v0.7.6)

I've tried to use jid on a 16MB (minified) JSON file on an M1 Pro (32GB) and it's as slow or slower as described in #66

Interestingly, the slowness ends when what's filtered to is no longer in the MB range.

Feature request: Cursor navigation and node selection

What

Be able to enter in a cursor mode, that would allow to navigate the json in RAW using cursors or vim hjkl.

Behaviour

• Press some key (Ctrl+n) for cursor navigation
• Press esc to go back to filter mode
• Press Enter to focus the current json node where the cursor is, and automatically fill the filter mode (See #77)

Optionally: allow select some text pressing shift.

Performance is extremely slow with a 7MB JSON file

I have a 2016 MacBook Pro (pretty fast machine), but if I give jid as input a 7MB JSON file that is on my local SSD drive, performance is very slow; each character I type takes about 3 seconds to respond.

The JSON file consists of about 27,000 items. The file is structured like this:

{
  (item),
  (item),
  (item)
}

Each (item) looks like this:

"128708": {
   "id_1": 132499,
   "id_2": "1939",
   "lang": {
     "qc": 1
   },
   "items": [
     130556,
     126889,
     129103,
     131423,
     126300,
     131270,
     134335,
     134483,
     141317,
     136120
   ],
   "date": "7/7",
   "category": 133089,
   "cr_dt": "7/8/2001",
   "d_year": "2004",
   "some_id2": 132589,
   "b_year": "1/22",
   "l_en": {
     "qc": 3,
     "path": "jeffsmith",
     "name": "Jeff Smith"
   }
 }

I do:
jid < /tmp/some.json

With that as an input if I type "1" it takes about 3 seconds before jid reponds, then if I type "2" it takes another 3 seconds, and so on.

Is there a way this could be improved to be faster?

Feature Request: Shift+TAB to jump to previous matching key

We press TAB to navigate next matching field. It would be nice to use shift+tab to go to previous.

Key containing double quote doesn't produce a valid jq expression

To reach success in the following file, the jq expression is .["\""]. But jid produces ."

Input file:

{
    "\"": "key with double quote: success!",
}

Does NOT work inside tmux

Prints characters all over the place and shifts cursor when inside a tmux session (viz image)

When not inside tmux session, jid workds as expected in the same terminal.

Incorrect work, when json file have similar keys.

jid version v0.6.2

Test json file:

{ "a": 123, "aa": 124, "bb": 567 }

Identify a bug:

1. Start jid < test.json
2. Enter a

I expected 123, but saw full json file.

jq's .[] filter doesn't work

The .[] filter works fine in jq:

❯ echo '[{"id":1},{"id":2}]' | jq '.[]'
{
  "id": 1
}
{
  "id": 2
}

But returns null in jid.

Is there any reason for this? I couldn't spot any explanation in the documentation.

Thanks!

JSON