chrisdone-archive / jl

Functional sed for JSON

jl

jl ("JSON lambda") is a tiny functional language for querying and manipulating JSON.

Example:

$ jl 'map $ \o -> { sha: o.sha, ps: map _.sha o.parents }' x.json
[{"sha":"7b81a836c31500e685d043729259affa8b670a87","ps":["c538237f4e4c381d35f1c15497c...

Installing

Binary releases for Linux and OS X are available here.

Builds on Windows (see AppVeyor status), haven't added Windows binaries to the releases yet.

Installing from source:

1. Get stack
2. Run stack install in the repository directory.
3. Add ~/.local/bin/ to your PATH.

Core syntax

Literals:

123, 4.5, -6, "hi", null, true, false

Lambdas:

\x -> y

Function application

get "f" o

Arithmetic:

x * (4 + 3)

Objects:

{foo: 123, bar: 34.3, "a:b": "hi"}

Arrays:

[1, 4 * 5, id 5]

Conditionals:

if x then y else z

Short-hand for fields:

o.f  is sugar for         get "f" o
_.f  is sugar for  (\o -> get "f" o)

For arrays:

_[0] is sugar for   (\o -> get 0 o)

Or objects:

_[k]     is sugar for   (\o -> get k o)
_["foo"] is sugar for   (\o -> get "foo" o)

Function composition:

a | b | c is sugar for `\x -> c (b (a x))`

Mini tutorial

You do everything with usual functional programming functions.

Returning the same thing, aka identity. That's normal in functional programming:

jl 'id'

A sequence of JSON strings will be read in and processed individually:

E.g.

$ cat x.json | jl id
{"a":1}
{"a":2}
{"a":3}
{"a":4}

If you want to read the input in as an array, use --array:

$ cat x.json | jl --array 'map _.a'
[1,2,3,4]

After processing, sometimes you want to print each element of the array out line by line, for that use --lines:

$ cat x.json | jl --array --lines 'map _.a'
1
2
3
4

Taking the first element of something, using syntax that looks like regular array access. The _ is a short-hand so that you don't need a lambda:

jl '_[0]'

If you want to get what keys are available, you can run:

jl 'map keys | _[0]'
["sha","committer","url","comments_url","parents","author","html_url","commit"]

Taking the first element and then creating a record of some parts of it:

jl '_[0] | \o -> {msg: o.commit.message, n: o.commit.committer.name}'

Note the use of | to compose functions. Just like in the shell.

Applying a function to all elements in an array:

jl 'map _.commit.committer.name'

Note how you can nest property access easily.

Applying something more detailed, by constructing a record of our own

jl 'map $ \o -> {msg: o.commit.message, n: o.commit.committer.name}'

You can use $ to avoid using parentheses on the right. That's a trick from Haskell.

Applying functions to nested data structures:

jl '_[0] | \o -> {msg: o.commit.message, n: o.commit.committer.name, ps: map _.html_url o.parents }'

Notice the ps property comes by taking the html_url of all the parents.

Filtering is easy, simply write a function that returns true:

jl 'map (\o -> { sha: o.sha, ps: map _.sha o.parents }) | filter (\o -> length o.ps > 1)'

If you want to make an object with arbitrary keys that come at runtime, use set:

$ echo '"hello"' | jl '\x -> set x 123 {}'
{"hello":123}

This sets the key x in the empty object {} to "hello" with the value 123. You can use set repeatedly to construct more keys.

If you want to construct an object from a list of key/values, you can use fold:

$ echo '[{"k":"foo","v":123},{"k":"bar","v":456}]' | jl 'fold (\acc o -> set o.k o.v acc) {}'
{"foo":123,"bar":456}

Available functions

Record access

get :: JSON → JSON → JSON

Get the value at k from the object

set :: JSON → JSON → JSON → JSON

Set the value k to v in object

modify :: JSON → (JSON → JSON) → JSON → JSON

Modify the object at k with function f

keys :: JSON → JSON

Get all keys of the object

elems :: JSON → JSON

Get all elements of the object

Sequences

map :: (JSON → JSON) → JSON → JSON

Apply a function to every element in the sequence

filter :: (JSON → JSON) → JSON → JSON

Keep only items from the sequence for which p returns true

takeWhile :: (JSON → JSON) → JSON → JSON

Take elements from a sequence while given predicate is true

empty :: JSON → JSON

Is a sequence empty?

length :: JSON → JSON

Get the length of a sequence

reverse :: JSON → JSON

Reverse a sequence

drop :: JSON → JSON → JSON

Drop n items from the sequence

elem :: JSON → JSON → JSON

Is x an element of y?

concat :: JSON → JSON

Concatenate a list of sequences into one sequence

zipWith :: (JSON → JSON → JSON) → JSON → JSON → JSON

Zip two lists calling with each element to f x y

take :: JSON → JSON → JSON

Take n items from sequence

fold :: (JSON → JSON → JSON) → JSON → JSON → JSON

Fold over a structure with a state.

dropWhile :: (JSON → JSON) → JSON → JSON

Drop elements from a sequence while a predicate is true

any :: (JSON → JSON) → JSON → JSON

Does p return true for any of the elements?

all :: (JSON → JSON) → JSON → JSON

Does p return true for all of the elements?

nub :: JSON → JSON

Return the sequence with no duplicates; the nub of it

sort :: JSON → JSON

Return the sequence sorted

append :: JSON → JSON → JSON

Append the members of the second sequence to the first sequence

sum :: JSON → JSON

Get the sum of a sequence

product :: JSON → JSON

Get the product of a sequence

minimum :: JSON → JSON

Get the minimum of a sequence

maximum :: JSON → JSON

Get the maximum of a sequence

Strings

words :: JSON → JSON

Split the string into a list of words

unwords :: JSON → JSON

Join the list of strings into a string separated by spaces

lines :: JSON → JSON

Split the string into a list of lines

unlines :: JSON → JSON

Join the list of strings into a string separated by lines and terminated by a new line

Predicate operators

/= :: JSON → JSON → JSON

a /= b

= :: JSON → JSON → JSON

a = b

Boolean operators

&& :: JSON → JSON → JSON

a && b

|| :: JSON → JSON → JSON

a || b

not :: JSON → JSON

not b

Numeric operators

> :: JSON → JSON → JSON

a > b

< :: JSON → JSON → JSON

a < b

>= :: JSON → JSON → JSON

a >= b

<= :: JSON → JSON → JSON

a <= b

* :: JSON → JSON → JSON

a * b

+ :: JSON → JSON → JSON

a + b

- :: JSON → JSON → JSON

a - b

/ :: JSON → JSON → JSON

a / b

min :: JSON → JSON → JSON

a min b

max :: JSON → JSON → JSON

a max b

abs :: JSON → JSON

abs b

Function combinators

id :: JSON → JSON

Identity function, returns its input unchanged

compose :: (JSON → JSON) → (JSON → JSON) → JSON → JSON

Compose two functions

flip :: (JSON → JSON → JSON) → JSON → JSON → JSON

Flips the argument order of a function of two or more arguments