thallium205 / bitcoinvisualizer Goto Github PK

Right now, every time the block chain is updated, the graph is traversed from the genesis block until the latest block is found. The index of the latest block found should be persisted so at next update, it starts at this location and not at the beginning.

Hi,

I'm trying to run visualizer on Windows 7

I installed Neo4j Enterprise 1.8.3 and Java 6 (for backwards compatibility)

I started the server Neo4jCoordinator.bat

I run BlockViewer.jar with the following command

Y:\Documents\bitcoin\BitcoinVisualizer\server\jar>java -jar -Xmx512m BlockViewer
.jar -dbPath Y:\Documents\bitcoin\neo4j-enterprise-1.8.3\data\graph.db\ -configP
ath Y:\Documents\bitcoin\neo4j-enterprise-1.8.3\conf\neo4j.properties -validate
false -client

I'm getting the following error:

Aug 22, 2013 5:21:45 PM bitcoinvisualizer.Main main
SEVERE: Unable to start Neo4j.
java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Unknown Source)
at java.lang.Integer.valueOf(Unknown Source)
at org.neo4j.graphdb.factory.GraphDatabaseSetting$IntegerSetting.valueOf
(GraphDatabaseSetting.java:212)
at org.neo4j.graphdb.factory.GraphDatabaseSetting$IntegerSetting.valueOf
(GraphDatabaseSetting.java:176)
at org.neo4j.kernel.configuration.Config.get(Config.java:113)
at org.neo4j.kernel.HighlyAvailableGraphDatabase.(HighlyAvailableG
raphDatabase.java:259)
at org.neo4j.kernel.HighlyAvailableGraphDatabase.(HighlyAvailableG
raphDatabase.java:209)
at bitcoinvisualizer.Main.main(Main.java:62)
Aug 22, 2013 5:21:45 PM bitcoinvisualizer.GraphBuilder StartDatabase
INFO: Starting database...
Exception in thread "main" java.lang.NullPointerException
at bitcoinvisualizer.GraphBuilder.StartDatabase(GraphBuilder.java:146)
at bitcoinvisualizer.Main.main(Main.java:94)

I took a look at the code, there seems to be some issue with the class HighlyAvailableGraphDatabase when initializing Neo4j. Unfortunately there is not a lot information around on the net about this class - it seems it may have been superseeded in Neo4j 1.9

I tried this on Windows & Linux, and got the same error.

Without a fix for this, the program is kinda unusable.

Cheers!

In GraphExporter, the Export function is synchronized at the moment which causes the API to process incoming web requests one at a time. This restriction was placed because the Neo4jImporter plugin is designed to only process the current workspace at a time (which is also synchronized). A new workspace should be passed into the exporter, modifying only that reference.

With upcoming plans to add more useful information to nodes within the ownership network, a side pane will be far moire appropriate than an onhover window.

Any owner that has had more than 1 transaction with another owner will only display the first transaction between the two. This is resulting in missing transactions and incorrect totals.

Hallo,
I want to use your BitcoinVisualizer for a small project. I will try to find some arbitrageurs in the blockchain. But I have got some problems using your tool. In your description, you write that the backend-user shall use java "-jar -Xmx2048m BlockViewer.jar...". All the parameters are ok, but I can't find the BlockViewer.jar . Can you tell me where I can find it or where my mistake is?

Kind regards and praise for your project!

The blockchain.info API was selected for expediency when this project was first started, but its time to start removing the API and swapping it with the actual bitcoin client's REST RPC.

More details of the plan here: http://toolongdidntread.com/uncategorized/the-future-of-blockviewer/

The Neo4jImporterImpl function was modified to prevent it from shutting down the database connection each time a graph was generated. This modification has resulted in a slight but persistent memory leak each time a graph is imported in GraphExporter.

When owners are getting created, they are not sufficiently checking to see if addresses withing the same_owner sub graph have already been claimed by another owner. Right now, only the toAddress.endNode() address node is being checked for an owner, and not any of the others. Read the STOP comment:

for (final Path toAddress : redeemedTraversal.traverse(block))
    {
        final Node address = toAddress.endNode();
        // Does an owner node already own this address?
        final Iterator<Relationship> owns = address.getRelationships(Direction.INCOMING, OwnerRelTypes.owns).iterator();
        if (!owns.hasNext())
        {
            // No.  We create a relationship from owner to all the addresses it owns

            // STOP.  This is our problem.  We need to check to see if ANY owner has previously redeedemd ANY of the addresses in this sub graph.  Simply checking the last address wont work.  If ANY of these addresses has an owner,
            // then we take that owner and and owns edges to the new addresses

            final Iterable<Node> addresses = address.traverse(org.neo4j.graphdb.Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL, AddressRelTypes.same_owner, Direction.BOTH);
            final Node owner = address.getGraphDatabase().createNode();
            for (final Node owned : addresses)
            {

                owner.createRelationshipTo(owned, OwnerRelTypes.owns);                  
            }               
        }           
    }

It is written that the following command will run the blockviewer:

java -jar -Xmx2048m BlockViewer.jar -dbPath ../graph.db/ -configPath ../neo4j.properties -validate false -low -high -scraper -exporter

but there is no BlockViewer.jar that I can find in the project I downloaded (BitcoinVisualizer-master.zip) and there is no manifest in the project either, that I can find. I tried making the entire project into a jar, but without a manifest it was not executable.

Hello I want to know if this still works?

After the addresses have been linked and owners built for the linked addresses, each edge from each affected owner needs to be updated. Currently, relinkOwners in GraphBuilder.java removes every edge, then rebuilds each one. This works well except for the super large nodes (mt.gox, satoshi dice, etc). Exploration into updating the values inside the edges instead of deleting them and recreating them might yield faster build times for nodes with high centrality.

After the graph is initially created, we need an algorithm that merely links existing entities to the high level graph instead of recreating it each time like the current implementation in GraphBuilder.BuildHighLevelGraph()

JSON is unable to fully map GEXF correctly, and has caused the client side code to become extremely unweildly. Refactor the code to consume GEXF and deprecate JSON support in the API.

When I run the following 3 commnds I get some log4j warning. I actually read the links from apache website where it's written that log4j.properties file if not found, then these errors will be shown. Can you help me out? And you haven't put any graph.db directory. And currently the neo4j-enterprise coordinator if tried to run, shows that currently you can run it by starting the neo4j itself,no dealing with neo4j coordinator. However, I am trying to build on a macintosh. So, I want to say the steps you have written in the details use section are little confusing.
So, what I have done is starting the neo4j in the port 7474 and then executing the following command but it gets stuck with that log4j. And I also have seen that you have modified the neo4j.properties file also. So I am using that modified file in your conf folder. And what happened to the blockviewer.com site? Would you please respond? I am quite new with neo4j.Thank you very much.

#export dbpath=/Users/zehadyzbdullahkhan/Documents/workspac e/MyGITHUB/neo4j-enterprise-1.9.3/data/graph.db/
#export configpath=../conf/neo4j.properties
#java -jar -Xmx2048m Blockviewer.jar -dbPath $dbpath -confi gPath $configpath -validate false -low -high -scraper -exp orter

usage: java -jar blockchainneo4j.jar -client Will only run the database service and not
attempt to build the blockchain.
-configPath The path to the neo4jconfig file. Ex:
/home/user/neo4j.properties
-dbPath The path to the neo4j graph.db directory. Ex:
/home/user/neo4j/graph.db/
-exporter Exports a time based analysis and owner
network of the entire block chain to mysql
-high Builds the high level data structure.
-low Builds the low level block chain structure.
-scraper Runs the scraper which attempts to associate
bitcoin addresses to real world entities.
-statistics Prints a CSV file in the current directory
with some statistics on the block chain.
-time The amount of time the program will wait
before rebuilding an updated version of the
graph again.
-validate <true/false> Toggle the verifier which checks if the local
json files form a complete blockchain.
Default: true. Recommended.
log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

There's some potential here but there are some serious performance and data management issues that needed to be fixed if this is ever to see the light of day again. I have successfully gotten it up to date with the new JSON fields, fixed problems that arise when you don't have the entire block chain, added features to arbitrarily download portions of block chain, and simplified a lot of things that cause it to choke down. I haven't uploaded my changes yet but hope to soon.

However, my recommendation is to learn what you can from this and design your own; and ditch Gephi and use later Neo4J... GLTA

The mysql library in node currently does not support streaming rows from a query. This causes nodejs to buffer the result set which could potentially cause node to run out of memory. The node application must only stream data.