GithubHelp home page GithubHelp logo

consensys / wittgenstein Goto Github PK

View Code? Open in Web Editor NEW
45.0 51.0 7.0 16.37 MB

Simulator for some PoS or consensus algorithms. Includes dfinity, casper IMD and others

License: Apache License 2.0

Java 99.20% HTML 0.80%

wittgenstein's Introduction

CircleCI

Wittgenstein

A simulator for some PoS, consensus or just distributed algorithms. Includes dfinity, Ethereum's Casper IMD, San Fermin and others. A somehow outdated presentation can be found here.

It allows you to test protocols, and to generate large graph and videos of the protocol in action, such as this one (generated for Handel: aggregation at scale with Byzantine nodes (click on the image to see the video): Watch the video

Why this name?

Wittgenstein was a concert pianist. He commissioned Ravel's Piano Concerto for the Left Hand, but changed it: 'lines taken from the orchestral part and added to the solo, harmonies changed, parts added, bars cut and at the end a newly created series of great swirling arpeggios in the final cadenza.'

As it's often what happens with mock protocol implementations, it looked like the right name.

How to build it

You will need java 9+ and gradle installed.

To check everything is correct:

gradle clean test

You can build a jar with gradle:

gradle clean shadowJar

How to run it

Once built:

java -Xms6000m -Xmx12048m -classpath protocols/build/libs/wittgenstein-all.jar net.consensys.wittgenstein.protocols.GSFSignature

This command is typically for a 16GB machine. The memory is very important when you want to simulate tens of thousands of nodes. If you have less memory, use lower values for -Xms and -Xmx, and run the simulations with less nodes.

But you're actually supposed to write code to implement your specific scenarios today. An obvious improvement would be to be able to define scenarios reusable between protocols.

How to implement a new protocol

For a complete description go to the wiki.

Here is an example:

public class PingPong implements Protocol {
  /**
   * You need a network. Nodes are added to this network. Network latency can be set later.
   */
  private final Network<PingPongNode> network = new Network<>();

  /**
   * Nodes have positions. This position is chosen by the builder.
   */
  private final Node.NodeBuilder nb = new Node.NodeBuilderWithRandomPosition();

  /**
   * Messages, exchanged on the network, are specific to the protocol.
   */
  static class Ping extends Network.Message<PingPongNode> {
    @Override
    public void action(PingPongNode from, PingPongNode to) {
      to.onPing(from);
    }
  }

  static class Pong extends Network.Message<PingPongNode> {
    @Override
    public void action(PingPongNode from, PingPongNode to) {
      to.onPong();
    }
  }

 
  class PingPongNode extends Node {
    int pong;

    PingPongNode() {
      super(network.rd, nb);
    }

    void onPing(PingPongNode from) {
      network.send(new Pong(), this, from);
    }

    void onPong() {
      pong++;
    }
  }

  @Override
  public PingPong copy() {
    return new PingPong();
  }

  @Override
  public void init() {
    for (int i = 0; i < 1000; i++) {
      network.addNode(new PingPongNode());
    }
  }

  @Override
  public Network<PingPongNode> network() {
    return network;
  }


  public static void main(String... args) {
    PingPong p = new PingPong();
    
    p.network.setNetworkLatency(new NetworkLatency.NetworkLatencyByDistance());

    p.init();
    PingPongNode witness = p.network.getNodeById(0);
    p.network.sendAll(new Ping(), witness);
    for (int i = 0; i < 1000; i += 100) {
      System.out.println(i + " ms, pongs received " + witness.pong);
      p.network.runMs(100);
    }
  }
}

This will print:

0 ms, pongs received 0
100 ms, pongs received 38
200 ms, pongs received 184
300 ms, pongs received 420
400 ms, pongs received 765
500 ms, pongs received 969
600 ms, pongs received 998
700 ms, pongs received 1000
800 ms, pongs received 1000
900 ms, pongs received 1000

wittgenstein's People

Contributors

bkolad avatar nikkolasg avatar nkeywal avatar vanessabridge avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wittgenstein's Issues

Add a Flood routing protocol

See libp2p doc here: https://github.com/libp2p/specs/tree/master/pubsub/gossipsub
The basic protocol is quite simple:
"With flooding, routing is almost trivial: for each incoming message, forward to all known peers in the topic. There is a bit of logic, as the router maintains a timed cache of previous messages, so that seen messages are not further forwarded. It also never forwards a message back to the source or the peer that forwarded the message."

P2P Signature Aggregation: add the possibility to have passive nodes

It should be possible to have two types of nodes:

  • actual signers
  • peers that relay others' signatures but don't sign themselves.

In real life, there are multiple reasons for such deployments, including:

  • some signers may want to be difficult to identify as actual signers
  • some nodes may want to participate as simple observers
  • nodes participating may change often making more interesting to keep the same peers even if the actual signers changed

Support scenario

For example, run a scenario with a partition at a given moment and try it on different blockchain.

It's obviously complicated to implement.

POW Sim

Build a protocol that simulates pow algorithms, uses difficulty calculations and includes uncles:

For Uncle creation
Before adding as uncle:

  • Validate that block received has the same parent as existing block at same height
  • Check block by height in blockchain and add as uncle if same parent and hasn’t been included and there are less than 2 uncles at that height
  • Prevent uncles past height+6 to be added

Refactoring

  • Remove uncle reference from the node

Mining

  • Adjust difficulty calculation to include uncles
  • Validate calculations using real data from main net in a test
  • Create reward functions that allocates funds for successfully mining: 1) block + transaction fees 2) an uncle

** Data collection**

  • Generate stats for uncles and record them in a separate file (csv)
  • Select minimal parameters necessary to feed to RL agent
  • Set method that calls agent at decision points

Generate data and store it outside with minimum parameters necessary

Default network protocols

Hi,

I am planning on writing a Dandelion++ Simulation on top of a gossipsub network in order to test some ideas outlined in this issue.

As it currently stands, I would have to build 2-3 protocols, one for a gossipsub network, one for Dandelion++ and another for sharding.

As a potential future feature of this simulation library, would it make sense to have some default network protocols that simulators can choose from?

Gossipping ENR protocol

EIP for ENR itself it here: https://eips.ethereum.org/EIPS/eip-778

One of the proposal to communicate the ENDR, likely the simplest one actually, is that each node gossip periodically its ENR. There are at least three things to take care about:

  1. we don't want to overload the network -> we need to know how much data we send depending on the period time
  2. we don't want a new node to wait for ages before having access to the information it needs, not getting state data, even if the capabilities is rare.
  3. nodes will need to connect to peers with the capabilities they need. But this could lead to partition the network by capability which may be a bad idea.

The basic protocol would be:

  • we rely on a p2p flood protocol
  • a node has a set of capabilities (a set of string for readiness), some of them rarer than other. The distribution is a parameter; Pareto & gaussian are likely interesting.
  • for some (proportion being a parameter) nodes, these capabilities change regularly (representing a validator moving from one shard to another). The time between changes is a parameter of the protocol.
  • a node gossips its capabilities
    • when it joins the network
    • when it leaves it (sometimes it dies w/o this step, proportion another parameter)
    • when the capabilities change
    • every X seconds (another parameters)
  • nodes discard the information after a given amount of time if it didn't get an update (another parameter of the protocol).
  • new nodes enter regularly and look for given capabilities then leave the network after a given amount of time. Another parameter, may be we need it to be a random variable on a gaussian rather than an integer.

We will measure, for around 20K nodes:

  • the time it takes for a new node to get the capabilities it needs.

cc @carver

Overload network send to unique destination

The Network class should have an overloaded method send which takes an unique destination as argument instead of taking only a collection. That would relieve us to use the Collections.singleton() trick.

Add logarithmic scale graphs

Our current graphs don't allow us to appreciate what happens in the first rounds of the protocol. Having a log scale graph (i.e. plotting log_2(number of signatures aggregated) over time) would allow us to see what happens in the early stages of the protocol execution.

We might thus be able to see how vulnerable the initial phases of the protocol are wrt to byzantine actors etc...

Support dead nodes

We should be able to support dead nodes in the framework, including the fact that it's sometimes possible to detect dead nodes

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.