nunoedgarhubsoftphotoflow / blockapi

A general framework for blockchain analytics

BlockAPI: Blockchain analytics API

A Scala framework for the development of general-purpose analytics on blockchains, maintained by Livio Pompianu and Stefano Lande of the Blockchain@Unica Lab at the University of Cagliari.

The framework allows to combine data within the blockchain with data from the outside (e.g. exchange rates and tags). The workflow consists in two steps:

1. construct a view of the blockchain and save it in a database;
2. analyse the view by using the query language of the database.

The blockchains currently supported are Bitcoin, Litecoin, and Ethereum. The DBMS currently supported are MongoDB, MySQL, PostgreSQL, and Fuseki.

The library is dicussed in A general framework for blockchain analytics, in proceeding of the SERIAL workshop 2017.

More material is available in the project page.

The Scaladoc is uploaded on this repository in the doc folder.

Install prerequisites

General prerequisites

Prerequisites depends on the target analyses. Generally speaking the framework needs at least one blockchain client and one DBMS.

1. Blockchain clients:
   • Bitcoin Core (extracts data from Bitcoin)
   • Parity (extracts data from Ethereum)
   • Litecoin Core (extracts data from Litecoin)
2. DBMS:
   • MongoDB (constructs a NoSQL view of the data)
   • MySQL (constructs a SQL view of the data)
   • PostgreSQL (constructs a SQL view of the data)
   • Apache Jena Fuseki (constructs a RDF view of the data)
3. Either
   • use an IDE for executing a Scala SBT project (we used IntelliJ IDEA) or
   • use the command line in place of an IDE (Install SBT)

Required libraries

This step is required to sucessfully compile the project. Before building the project, execute the following commands:

Bitcoinj

git clone https://github.com/bitbart/bitcoinj.git
cd bitcoinj
git checkout segwit
mvn install -DskipTests
cd core
mvn install -DskipTests

Litecoinj

git clone https://github.com/litecoinj-unica/litecoinj.git
cd litecoinj
mvn install -DskipTests
cd core
mvn install -DskipTests

Analysis dependent prerequisites

ICO Analytics

This instructions are needed only for performing analyses on ICO on Ethereum.

In order to use the ICO class (created to retrieve ICOs data) you must require the respective API keys:

1. EtherScan
   • It is sufficient to sign up and then go here
2. ICOBench
   • Sign Up, then require an API key here
3. Ethplorer
   • There is a default, free key: freekey. If you need more data or highload of service, you can to get personal API key.

After doing these steps, do the following:

1. Copy EtherScan key into EtherScanAPI.apiKey attribute
2. Copy ICOBench private key and public key into ICOBenchAPI.privateKey and ICOBenchAPI.publicKey attributes respectively
3. Copy Ethplorer key into EthplorerAPI.apiKey attribute

Install blockchain analytics API

1. Execute the blockchain client in order to obtain a local copy of the target blockchain (this process may take several hours)
2. Enable the client RPC calls.
   • In the Bitcoin Core case

     bitcoind -datadir=path/to/blockchain -server -rpcuser=user -rpcpassword=password -rpcserialversion=0

3. Run a localhost instance of the DBMS on the default port.
4. Clone this repository.
5. From your IntelliJ welcome screen: select "Import Project" and open the build.sbt file from your repository directory.

Running

1. Execute of one of the available examples. Either
   • open the file from your IDE and select run or
   • use the command line: from the root directory of the project, execute

     sbt "runMain tcs.examples.ClassName"

Our framework will build the selected database. Then you can query it for performing your analysis. For each available Scala script, we provide some default queries along with the resulting csv files.

Acknowledgments

The authors thank the following developers of the Department of Mathematics and Computer Science of the University of Cagliari for their valuable contributions.

Members of Blockchain@Unica lab:

• Nicola Atzei, Sergio Serusi - Improvements on software architecture and testing

Interns at Blockchain@Unica lab:

• Giulia Argiolas - Introduction of Litecoin blockchain
• Daniele Stefano Ferru - Introduction of Ethereum blockchain, analysis of ICOs
• Davide Curcio - Fuseki Database
• Antonio Sanna - Bitcoin mempool

Students of the Cybersecurity course:

• Andrea Corriga, Omar Desogus, Enrico Podda - Empty blocks on Ethereum
• Giacomo Corrias, Francesco Pisu - Empty blocks on Bitcoin
• Giancarlo Lelli - Bitcoin pools
• Fabrizio Chelo, Hicham Lafhouli, Antonello Meloni - Signature hash types
• Giovanni Laerte Frongia, Luca Pitzalis - Verified contracts from Etherscan.io
• Filippo Andrea Fanni, Martina Senis, Alessandro Tola - Ethereum pools
• Riccardo Mulas - Non-standard transactions
• Fabio Carta, Francesca Malloci, Flavia Murru - Duplicate Ethereum contracts
• Carlo Cabras, Federico Maria Cau, Mattia Samuel Mancosu - Balances of Ethereum addresses
• Andrea Demontis, Stefano Dessì - Balances of Bitcoin addresses
• Federica Gerina, Silvia Maria Massa, Francesca Moi - State of the DAPPS
• Giuseppina Lai, Federica Muceli, Federico Spiga - UTXO set
• Alessandro Bonini, Alberto Pes, Maurizio Porcu - Blockchain.info tags
• Stefano Raimondo Chessa, Marco Guria, Alessio Manai, Alessio Speroni - Ethereum tokens
• Paolo Orrù, Elisa Pau - Cross validation Ethereum
• Stefano Danese, Giuseppe Bellisano, Emanuela Ripoli - Bitcoin transactions ip-geolocalization
• Riccardo Casu, Carlo Cuccu, Vittoria Frau - Bitcoin exchange rates
• Federico Medda, Alberto Musa - Cross validation Bitcoin
• Simone Balloccu, Andrea Bellanti, Federica Stocchino - Transaction chains
• Daniele Sanna, Giovanni Usai - Cross validation Litecoin
• Soukaina Aassouss, Reda Belaiche, Khadija Zine - Counterparty Tokens