jinroh / kadoh

The Kademlia DHT in Javascript for Node.js and Browsers

KadOH - Javascript P2P framework

KadOH is a framework to build P2P applications for browsers and node.js. By implementing the basis of the Kademlia DHT, KadOH lets you build distributed web applications for mobile and desktop devices. With its flexible and extensible design, you can easily adapt KadOH to fit your needs. KadOH is available under the MIT License.

KadOH abstract many different transport protocols to provide P2P connections. In the browser we support XMPP over Bosh and Socket.io shipped with a node.js router, and you can go for UDP and native XMPP in a node.js application. We plan to support WebRTC soon !

See the wiki for more informations ! Also take look at our report and be aware that this document may be outdated.

---

Acknowledgments — We would like to thank Dr. Tudor Dumitraş, who gave us the honor to work with him during our project and always made available his support.

Alexandre Lachèze and Pierre Guilleminot

See it in action

Live demo based on XMPP transport running on Amazon EC2.

Run it yourself

First you can start a little DHT based on UDP transport using this command :

bin/dht udp.default

Then in an other terminal, start the web-based debug UI using the following Jake command and go to localhost:8080 :

jake run:udp

See the wiki for more informations on how to launch your own DHT from scratch and on different environments.

Installation

Install KadOH

The only required dependency is node.js (version >= 0.6, but you clould try below..). Take a look at the official manual to install node on your system.

Then run :

npm install kadoh
# or: git clone https://github.com/jinroh/kadoh.git
cd kadoh
npm install

You may also want to install the Jake module globally :

npm install -g jake

For Linux users, if the installation of KadOH's dependencies fails, you may need to install the libexpat-dev package. For Ubuntu/Debian users, run the following command :

sudo apt-get install libexpat-dev

Build and go

Browser

To build the source inside the dist folder run one of the following Jake command :

jake build

This will build two versions of KadOH supporting different transports :

• SimUDP using Socket.io
• XMPP using Strophe.js for XMPP over Bosh

Node.js

If you need to use KadOH in a node.js application, just add kadoh in your package dependencies and use :

var kadoh = require('kadoh');

Thanks

KadOH is built on top of many open-source libraries and projects:

• Strophe.js
• node-xmpp
• cube
• klass
• bncode
• socket.io
• express
• browserify

Tests

For the moment, tests are not running anymore.. But we hope that will be fixed as soon as we migrate to Mocha.

jake test:node
jake test:browser

kadoh's Issues

The XMPP protocol

Let's list here all the technologies we have seen that could help us to implement our system over XMPP

Migration from Jasmine to Mocha

http://visionmedia.github.com/mocha/

Making a system of bots to run tests ?

Bootstrap and Bot server

RESTFul..

License

Chosing a LICENSE for our project :

• MIT
• BSD
• GPL (v2 || v3)
• Apache
• ...

Front-end developement

Write a simple visualization for mobile.

Handle with dependencies in the browser

Du coup ? / So, well ?

Du coup, on en est où ?

SHA1 library for Javascript

Find a good SHA1 library for javascript

Adaptive time-out

Based on RTTs and stats of received response to an already completed query.

Migrate testing from Vows to Jasmine

Jasmine enable to write test that can be executed in Node or in browser.

Migration from Strophe to node-xmpp (BOSH)

Leverage SAX parser.

Organize as Classes

Organize client side code as Classes with appropriate namespaces..

Find the appropriate framework that works both on client side and Node (for testing and benchmarking).

avoidusingcomplexdesignpatternsinwhichweneverknowwhatisThis

Have an XMPP over BOSH bot

It's closer to reality for benchmarking.

Related to #41 since node-xmpp wil have an XMPP bosh for node.js and browser.

First version of our Kademlia implementation design

Finalize a draft of the UML and README for our Kademlia design

Implement the local functions of Kademlia

• Routing Table
• KBuckets
• Peers

value storing in mobile-browser/node : best solution ?

http://westcoastlogic.com/lawnchair/ ?

Fusion /router and /dht

Separate server-side than client-side code

In /lib create 2 folders : client and server

Handle with the namespace

Migrate SimUDP client Code into client/transport/SimUDP.js

Handle with the dependencies of Socket.io.client

Unify the build code

Migrate the readme into docs

Migrate example into example/SmUDP

WebRTC P2P Data Stream

http://dev.w3.org/2011/webrtc/editor/webrtc.html
http://dev.w3.org/2011/webrtc/editor/webrtc.html#peer-to-peer-connections
http://arstechnica.com/business/news/2012/01/hands-on-building-an-html5-photo-booth-with-chromes-new-webcam-api.ars
http://blog.chromium.org/2012/01/real-time-communications-in-chrome.html
http://www.webrtc.org/
https://www.w3.org/Bugs/Public/show_bug.cgi?id=15206

Writing the project's README.md

Describing the project in the README of the root folder.

Event emitter/listener in KadOH

Test Fail in Peer sometimes

Il m'est arrivé 1 ou 2 fois seulement en lançant les tests dans node d'avoir le fail suivant :

Peer
  it should be possible to instanciate a peer using a triple
  Error: Expected '90f6505db61ecc6021beb7fec7ee16793d83c33a' to equal '3164dfe811bf14f8b185b62c8da4d7a8164fc082'.
    at [object Object].<anonymous> (/Users/alex/Taf/3A Eurecom/SemesterProject/kadoh/spec/peer.spec.js:40:26)

Chelou ! Grave ?

### Nothing

Migration to Browserify

https://github.com/substack/node-browserify

Writing RPC functions

Write a package.json

Logic Routing Table : ping oldest..

Lorsqu'on veut ajouter un nouveau peer et que son KBucket est full et non splittable, on prend le plus vieux peer du KBucket, on le PING et s'il répond on laisse tomber le nouveau.

Ne penses tu pas qu'on devrait les pinger un par un dans l'ordre décroissant de vieillesse jusqu'à ce qu'il y en ait un qui ne réponde pas ?

C'est déjà vu quelque part ?

Gather the KadoH parts and the other in reactor

Tests in node fail..

SimUDP whoami doesn't work on iphone

Re-write connection/disconnection flow

Ideas :

• create an other state of the reactor : kind of pause state where RPCs are not sent for any reasons (transport disconnected).
• automatic re-connection after disconnection.. Related to #42

Using WebWorkers ?

Maybe we could use webworkers to run our node object since we don't need any DOM access and we would take advantage of multithreading.

stackoverflow

• John Resig (of jQuery fame) has a bunch of interesting examples of using web workers here - games, graphics, crypto.
• Another use is Web I/O - in other words, polling URLs in background. That way you don't block the UI waiting for polling results.
• Another practical use: in Bespin, they’re using Web Workers to do the syntax highlighting, which you wouldn’t want to block your code editing whilst you’re using the app.
• From Mozilla: One way workers are useful is to allow your code to perform processor-intensive calculations without blocking the user interface thread.
  As a practical example, think of an app which has a large table of #s (this is real world, BTW - taken from an app I programmed ~2 years ago). You can change one # in a table via inpput field and a bunch of other numbers in different columns get re-computed in a fairly intensive process.
  The old workflow was: Change the #. Go get coffee while JavaScript crunches through changes to other numbers and the web page is unresponsive for 3 minutes - after I optimized it to hell and back. Get Back with coffee. Change a second #. Repeat many times. Click SAVE button.
  The new workflow with the workers could be: Change the #. Get a status message that something is being recomputed but you can change other #s. Change more #s. When done changing, wait till status changes to "all calculations complete, you can now review the final #s and save".

General UI generator

Should we split the repository ?

We should split the repository in two ones using http://help.github.com/split-a-subpath-into-a-new-repo/ to maintain version numeration easy.

Add unit-testing in router

Unit testing of the router using zombie.js

Conflict between socket.io versions when using jasmine-runner

Values management

store_value and find_value

Value management : 2 successive store could raise error

To remind me.

If 2 stores on the same key, the second one defines supplementary timers.. Bad.

Unexpected XMPP disconnection on idle iPhone

Related to #45

Deploy on a server

Centralyzed logging server/DB

Create a REST server that receives the logging data of runnings instances (bots and non-bots) and store them in a SQL DB. There are then possibilities to gather interesting stistics information.

TODO

• what logging data do we need ?
• schema ?

Wish List..

• tests : migration from Jasmine to Mocha
• eventually migrate to Browserify
• do a complete BitTorrent Client
• have a nice documentation generator. Hope is here : doctor
• .. a report maybe ?

How to test one part of KadOH without making a general build ?

We have to deal with dependencies without affecting the code executed client-side ?

The response should be in Soket.io-client..

Escaping XSS in UI

Usable version of the Router

v0.2 of the router before the testing

Upgrade UI to Bootstrap 2.0

New : support for mobile screen.

App Example in Router

Write the page of the project

Beautiful gh_pages

EventEmitter to report activity

Bon alors comme on en avait discuter, je pense qu'il sera plus simple de faire un gros "EventManager" pour tout KadOH plutôt que chaque objet (RoutingTable , les KBuckets...) soient chacun un EventEmitter auquel il faudra s'abonner individuellement..

Avantages :

• on s'occupe d'ajouter des listeners qu'à un seul endroit, c'est à dire à l'endroit où on en a vraiment besoin : juste avant un console-log, à l'update d'une UI ou avant de l'envoyer dans socket.io pour un report à distance.
• pas d'extension de classe qui peut-être badante.

Petits inconvénients

• faudrait faire gaffe à faire un namespace des events "intelligent" et à donner des arguments "utiles" aux listeners, avec une bonne doc.

Petits plus

Y'a moyen de faire un systeme de namespace comme dans celui de EventEmitter2 qui permet de s'abonner à des events qui matchent un pattern (eg foo:bar)... Mais je trouve l'intêret finalement limité.

Démystification

Un systeme d'EventEmitter n'est en fait qu'un tableau de callback quoi qu'il arrive. Jette un coup au module event de Node.js, c'est déconcertant de simplicité.

Du coup..

Je vasi essayer de mettre en place ce truc rapidement, comme ça on rajoute des emitions d'event un peu partout rapidement..

Writing/generating documentation

Generate doc using dox.

in /docs folder

(http://dailyjs.com/2011/01/20/framework-part-47/)
(https://github.com/visionmedia/dox)