In order to make json schema references work, we need an interface for traversing json pointers in decoded go types. I've seen this pattern pop up in a few places, and would love to think about how to broaden the applicability of such an interface(s).
Problem
Let's say we decode this json to a generic map[string]interface{}
:
{
"foo" : ["bar"],
"references": {
"a": {"$ref": "#/foo/0"},
"b": {"$ref": "/foo/0"}
}
}
According to the json pointer spec, Both "a"
and "b"
in the above example should resolve to "bar"
. By decoding to map[string]interface{}
, this is no problem, and the current jsonpointer.Eval
can handle this for you using a type switch. This breaks, however, when anything that isn't map[string]interface{}
or []interface{}
are provided to jsonpointer.Eval
. This problem gets worse when we do exotic things with go types.
After experimenting a bunch with the "just marshal back to generic structs & traverse", it's just way too much overhead and complexity. So:
We need an interface that lets go types declare how to traverse json pointer paths
Suggested Solution:
Instead of having go structs be in charge of resolving full paths, I think each struct should only be in charge of resolving it's immediate children (if any). There are only two "container" types in JSON: objects and arrays. The thornier of the two types is an object. So I'm suggesting two interfaces.
The first interface provides fast lookup for traversing expected paths (eg, json pointers):
// JSONContainer returns any existing child value for a given JSON property string
type JSONContainer interface {
// JSONProp takes a string reference for a given JSON property.
// implementations must return any matching property of that name,
// nil if no such subproperty exists.
// Note that implementations on slice-types are expected to convert
// prop to an integer value
JSONProp(prop string) interface{}
}
The second interface provides comprehensive child listing for full-tree traversal:
// JSONParent is an interface that enables tree traversal by listing
// all immediate children of an object
type JSONParent interface {
// JSONChildren should return all immidiate children of this element
// with json property names as keys, go types as values
// Note that implementations on slice-types are expected to convert
// integers to string keys
JSONProps() map[string]interface{}
}
Looking up a path should be done by breaking up the pointer into tokens and evaluating the components, as the current jsonpointer.Eval()
does, but now with a check for the JSONContainer
interface, and reflection as a fallback.
The work of traversing the entire tree should be a generic function:
// WalkJSON calls visit on all elements in a tree of decoded json
jsonpointer.WalkJSON(tree interface{}, visit func(elem interface{}) error) error
This follows the spirit of encoding/json
and filepath.Walk
from the standard lib.
The filepath.Walk()
part allows users to BYO visit
function to call on each element in a tree.
Both functions would follow the spirit of the way encoding/json
handles mapping JSON to go types:
- Checks for implementation of the above interfaces, using those if present.
- Has fast-paths for
map[string]interface{}
and []interface{}
- Falls back reflection, examining the
json
tag when present for property naming, and case-insensitive matching on exported field names otherwise
My expectation is that most implementations on slice-types would not use the JSONContainer
or JSONParent
at all, and instead rely on reflection. Slice-types would still have the JSONContainer interface for odd circumstances or edge cases.
@Stebalien, I'd love to see if we can't make the distance between resolving json pointers and IPLD paths as short as possible, and would love your (or really anyone from the IPLD project's) input on this interface as we go forward. I'm assuming IPLD's needs will be a superset of these problems, but it'd be great if the two could play nicely.
I've come across this problem a bunch in the wild, and would love other's feedback on the subject. Thanks!