Comments (16)
@iddan Yes, I think we can close it. Most of it is implemented I believe. For everything else we have a set of more focused issues open.
from cayley.
this sorta seems, maybe, like an appropriate juncture to discuss what i'm trying to do in https://github.com/sdboyer/gogl, as the chief thing of value (i think) i've produced so far is some essential patterns around graph interfaces.
then again, i've not gotten to property graphs/triple stores at all yet. but if this is what you mean by "higher-level Go API", then i'm happy to bend my work in that direction.
from cayley.
@sdboyer -- Seems right, to a degree. Do you have a favorite example?
So iterators as ways of getting sets of things (in fact, exactly sets, if you drop the path data) seems like it plays a little into your story as well, though from my (albeit cursory) look, you're talking a little more about the extended library -- which does play in here, but is slightly orthogonal.
If I can think out loud a bit...
So, unless you know why and how building iterator trees work, it's not going to make much sense to the standard library writer. Query languages help, because they effectively do the same thing for you; they give you a way to start and work with your data.
So a similar thing might exist in Go-land, and it may (but doesn't have to) look a lot like Gremlin.
Thinking about it in terms of abstract data types....
Traversals are things that take Paths to Paths, Morphisms are just traversals as first-class objects, and the thing that people kind of need is a way to start with some Node or set or something and lift
that into a Path. These are the basic building blocks of queries.
Do correct me if I'm wrong, but I sense you're interested in, basically, other data types. Things like taking Paths and coming back with a topological list of Paths (via some pattern), or given two sets of nodes, the set of Paths that connect them (going from Node pair to Path, DFS style) or what have you.
Which is why it is related, but different at the same time. And you could imagine using the same building blocks behind your more complicated traversals.
from cayley.
So, copy and pasting from #58:
"In thinking about attacking #74, I think it may be useful to build a graph.Query object to wrap much of this. Then the gremlin and MQL parsers/output can share more code at least."
This makes sense. I think one of the things that may be worth doing is to write a RunQuery (or somesuch) file in graph/ -- perhaps with the aforementioned graph.Query-- that, for the purposes today, takes care of the for { _, ok := it.Next() } loop I have everywhere in one location, with some flags and options for how and what to query, whether to optimize, etc.
This also centralizes it for other such functions/new evaluations/etc, as per this thread. So this may spin off a new issue, if that sounds reasonable, but all the better.
from cayley.
Here's my brain-dump as well. Basically it'd be cool to have something like this:
type Query interface {
TreeWalk(walkFunc func(path []Iterator))
AddIterator(type string, options map, subIts ...Iterator) (Iterator, error)
Optimize() bool
Exec() error
Results() map
}
With TreeWalk I can display stats estimates without executing the query, so that I can manually refine it. Or perform other operations on the tree without too much pain.
AddIterator is akin to the current buildIterator methods in various forms. Basically allows you to dynamically create iterators without special-case code (would require a more augmented iterator registration system). I dont think this method prototype has enough info, but you get the idea.
Optimize() is what you think, Exec() is the RunQuery() @barakmich mentions with the ability to return errors instead of panic-ing, Results() returns data ready to be remapped to your own format.
from cayley.
Let's start with Optimize(), Exec() and Results(), which are pretty clearly going to need to happen anyway, and we'll come around to TreeWalk and AddIterator.
I might make Optimize a bool flag, that Exec() respects, along with the potential for tracing or limits or something, but that's the only thing in that part that I could think of. I take it then that the query languages would generate things with these interfaces and call an appropriate RunQuery(graph.Query) function?
from cayley.
Can you explain this a bit more?
Results() returns data ready to be remapped to your own format.
from cayley.
@barakmich TreeWalk is fairly trivial to implement and can be potentially
used to simplify things like Optimize. Also, Exec = your RunQuery, which
you really want as part of Query to fully encapsulate the Go API. Somewhere
in here we need to pass a TripleStore as well. Or is RunQuery part of
TripleStore?
@kortschak basically some sort of generic way to represent both Gremlin and
MQL outputs such that all you have to do is a basic transform on it. I
don't know if its possible or reasonable but its an idea =)
Can you explain this a bit more?
Results() returns data ready to be remapped to your own format.
—
Reply to this email directly or view it on GitHub
#64 (comment).
from cayley.
Have a look at database/sql to see how Brad did it. There Exec() returns a Result interface. This is probably the way to go - also naming to be consistent with that package would be nice where possible.
from cayley.
Might be closer to database/sql/driver since this is behind the query
language, but yeah that's a good idea.
On Jul 21, 2014 7:24 PM, "Dan Kortschak" [email protected] wrote:
Have a look at database/sql to see how Brad did it. There Exec() returns a
Result interface. This is probably the way to go - also naming to be
consistent with that package would be nice where possible.—
Reply to this email directly or view it on GitHub
#64 (comment).
from cayley.
Naw you're right, TripleStore is more like the driver interface. Query would be similar to a prepared statement I think, and the Result interface would likely be a thin wrapper on the IteratorTree, so I guess this does front-load the optimization and possible other options into Exec(). I'll think about it some more.
from cayley.
Forgot to mention the other benefit of AddIterator's dynamic nature - it means that internal data can be made more accessible. Which will be especially important for allowing triplestores to optimize them more. Right now a triplestore backend cannot access the direction(without typecast) and value in a LinksTo iterator, and the (currently used) Comparison iterator cannot export Operator and Value access as well. If we use a data map for these features, then a Postgres Comparison iterator and the base Comparison iterator can both use the same keys and be used interchangeably. This allows greater extensibility for more iterator types, and less boilerplate code compared to exported struct fields or interfaces.
Hope all that makes sense =)
from cayley.
Is that go api make cayley in some way like embeded mode?
from cayley.
@villadora i think it's safe to say that an embedded mode would be one benefits to having a higher-level Go API. i think another would simply be ease in moving between doing work on in-memory graphs and easily moving them into a persistence layer.
at least, that's the kind of thing i have in mind :)
from cayley.
I hope that's the right place to ask:
Where can I find some introductions for using cayley as a library? Is there a hello world example? And do you know a real world example of a go application that's using cayley (no matter which API is used) ?
from cayley.
Documentation for using Cayley as a lib can be found here: https://github.com/cayleygraph/cayley/blob/master/docs/Quickstart-As-Lib.md.
As far as I understand this feature was resolved and can be closed. @robertmeta @dennwc
from cayley.
Related Issues (20)
- 'go build ./cmd/cayley' breaks: missing go.sum entry for module providing package github.com/golang/protobuf/proto HOT 3
- Move cayley.io to Github Pages or Netlify or similar HOT 1
- Aggregations
- Code quality: NameOf and ValueOf should return errors HOT 2
- Relationships not displaying correctly in new cayley version HOT 1
- Cayley.io is down again HOT 10
- Seeking for a new maintainer HOT 13
- hidalgo: interfaces changed a bit in newer versions HOT 2
- Security issue with github.com/gogo/protobuf version < 1.3.2 HOT 2
- Repl Allows One Command to Execute and Subsequent Commands Fail HOT 1
- request a new release HOT 1
- Filter path by Label field of quads HOT 1
- Filter path by empty label field (FilterContext) HOT 1
- Plans for a new release? HOT 2
- Build Issue using cayley 0.7.7 HOT 1
- q InternalQuad ==> q *InternalQuad
- gio gui HOT 1
- Plans for a new release?
- Discourse forum down
- Is this project dead? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cayley.