Comments (7)
So I tracked this down further. The call to remove
in scc
seems to be the culprit. It keeps building lazy sequence on top of lazy sequence to filter out the nodes, and eventually reaches a point that overflows the stack. Perhaps using an ordered set would be better here?
from loom.
Could you please share the graph you're trying this on? Alternatively, could you try to see if the following fixes your problem:
(recur (doall (remove seen stack))
seen
(conj cc c))
on the following lines https://github.com/aysylu/loom/blob/master/src/loom/alg.clj#L312...L314?
Note the invocation of doall on the remove function.
from loom.
I did try that after reporting the issue, but it turned out to be only part of the problem (sorry for not coming back and reporting that!).
I needed to make the change you recommended, and also change a line in traverse-all
:
[(into seen ctrav) (doall (concat trav ctrav))])))
(notice the doall
around the concat
).
Between the two changes, it ran to completion but extremely slow. It took 4 hrs and 45 minutes to complete, while the same algorithm with a rudimentary implementation in Python ran in 1 minute and 20 seconds on the graph of 200,000 vertices and 200,000 edges.
I put the edge list for the graph here: https://gist.github.com/jszakmeister/7068034
It shouldn't be hard to read it in and build the digraph from it.
from loom.
Thanks for the follow-up and the test graph! It sounds like it's time for refactoring the algorithm for more optimized data structures. If you end up optimizing the function, your pull request will be very welcome.
from loom.
Thanks @aysylu. I hope to get some time to spend looking at this in the next couple of weeks. If I figure something out, I'll be sure to send you a pull request.
BTW, how do you feel about introducing another dependency? I'm looking at possibly using https://github.com/flatland/ordered, which has an implementation of ordered sets. I think that'll be useful for solving the stack overflow issue in a more performant manner, but I want to make sure you're okay with that first.
Thanks!
from loom.
Given the choice of having no other dependencies but Clojure and slow implementation of SCC or reasonably fast implementation of SCC and an extra dependency, I think it's fine to introduce another dependency. With the eventual goal of Loom becoming a Clojure contrib library, this will affect what we can include in the contrib version, but we can figure it out later.
Thank you, @jszakmeister!
from loom.
@aysylu @jszakmeister I tried to reproduce this error. I see that the computation never completes for post-traverse and pre-traverse as well. I hope to work on this issue this week, if all goes well.
from loom.
Related Issues (20)
- Implement Edge fully HOT 5
- Loom incompatible with CLJS >= 1.9.854 HOT 3
- Make 1.0.1 release ? HOT 3
- Possible Clojurescript [any Long] issue HOT 2
- Graph visualization in the browser HOT 1
- bipartite-color fails for digraphs with nodes containing no outgoing edges
- java.lang.NoClassDefFoundError: loom/graph/Graph HOT 2
- connected-components should return a seq of (sub)graphs
- Wrong typehint for clojurescript
- maximal-cliques silently breaks on digraphs
- Link to FAQ doesn't work HOT 1
- How does the ancestry cache work?
- Graph transpose does not work correctly in ClojureScript
- loom.io/view logic errors HOT 1
- remove-nodes* for BasicEditableDigraph throws null exception in cljs HOT 1
- Time for a new release? HOT 5
- Building a graph with an empty map throws a nullpointer exception
- hypergraphs using loom?
- loom.io/view calls render-to-bytes incorrectly HOT 1
- Add mincost flow algorithm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from loom.