constellation-labs / constellation Goto Github PK

Cluster tests need to be run with many different parameter variations. Parameters of interest are number of nodes, size of mem pool selection, and emission / combination probabilities.

May have issue with dependency changes / etc.

Should be supplied on initialization through config. Ideally store a large number of keypairs and have deterministic process for nodes selecting the keypairs so there's no duplicate usages. Should also persist between pod restarts.

Need to finalize new schema -- we're going to migrate transaction to use the same edge format, and also add a data hash element to every edge.

After our initial implementation we're going to need to simulate a live network to find pitfalls and identify benchmarks for our users. This can/will be transformed into a continuous integration tool which will run a serious of integration tests every time we make a code change. I'm thinking Circle CI and hosting a small cluster on AWS.

This will be broken down as follows:

1. Formulate integration tests as unit tests using mocks
2. Write a script to launch a multi node cluster on AWS with at least one node processes running per machine
3. Turn those unit tests into actual integration tests
4. Automate with a tool such s Circle CI

Use the Mapbox API to stream node topological data and render in UI.

We require JSON for each point of the topology to create the graph that outputs Node data to the 'Horizon' UI.

Data rendered should include the following (I do not know what is available so help here would be appreciated):

Node IP address
Node throughput
- Current in tx/s
- Aggregate over time up
Node state and other data which would be useful

Horizon is to be located at https://horizon.constellationlabs.io

Eng to provide a backend schema for @giovaroma to use in the above front-end implementation.

Please add more details here.

CC @buckysballs @ryle-goehausen @prestonparris @tylerprete @brionhickey

Example front end vis is from NKM here:

http://testnetdemol.nkn.org/

@ryleg had a great idea about creating a cryptographically secure feedback mechanism for checking that the java opcodes of an dapp are available and deployed honestly.

1. Investigate how we can verify with some probabalistic margin of accuracy that code deployed somewhere actually is what it is.
2. Incorporate this into #4
3. How could we offer incentives for nodes to host this service?

As of commit f53316d

No logs / stacktraces are apparent, logs only indicate a restart happened. Kube doesn't even show a restart. Node is healthy afterwards.

Crash happened on
gke-constellation-te-constellation-po-610ff205-ttfb on pod constellation-app-ryle-3

When clicking through the peers list, the IP was incorrect for this node. Possibly due to DNS issue? Node was definitely healthy and participating in consensus up to the point of failure.

• have consensus read from the edge and mempool dao
• trigger consensus on edge event (for now 10 transaction in mem pool with 3 sigs)
• output new edge

• our current wiki does not contain much information. Need to take a first pass at populating this.

Skin JSON output data produced from localhost:9000 output that is generated from a local instance of a Constellation node connected to the test cluster.

UI found in:

constellation/ui/

Run a local node and compile using sbt from the ui directory

sbt fastOptJS

We need to figure out what features give us the best signal regarding node honesty while also minimizing computational overhead. I'm going to propose a simple heuristic initially but I would like to follow in the footsteps of REGRET. To clarify from here on out, a meme will represent the feature space of some reputation model.

With regards to computational overhead, we need to be cognizant of how memes are used. See issue #7 for a details, to which this will be heavily tied. This encompasses the data science portion of the efforts which will most likely require dimensionality reduction via PCA or something similar.

This is necessary for environment isolation updates.

Currently STS' are being deployed against same cluster. We need better branching isolation and automated testing for dev -> staging -> prod release cycle.

[info] ClusterTest:
[info] Cluster integration
[info] - should ping a cluster, check health, go through genesis flow *** FAILED ***
[info] org.json4s.package$MappingException: No usable value for metrics
[info] Expected object but got JNothing
[info] at org.json4s.reflect.package$.fail(package.scala:95)
[info] at org.json4s.Extraction$ClassInstanceBuilder.buildCtorArg(Extraction.scala:521)
[info] at org.json4s.Extraction$ClassInstanceBuilder.$anonfun$instantiate$4(Extraction.scala:541)
[info] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:234)
[info] at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59)
[info] at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52)
[info] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[info] at scala.collection.TraversableLike.map(TraversableLike.scala:234)
[info] at scala.collection.TraversableLike.map$(TraversableLike.scala:227)
[info] at scala.collection.AbstractTraversable.map(Traversable.scala:104)
[info] ...
[info] Cause: org.json4s.package$MappingException: Expected object but got JNothing
[info] at org.json4s.reflect.package$.fail(package.scala:95)
[info] at org.json4s.Extraction$.extract(Extraction.scala:374)
[info] at org.json4s.Extraction$ClassInstanceBuilder.buildCtorArg(Extraction.scala:509)
[info] at org.json4s.Extraction$ClassInstanceBuilder.$anonfun$instantiate$4(Extraction.scala:541)
[info] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:234)
[info] at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59)
[info] at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52)
[info] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[info] at scala.collection.TraversableLike.map(TraversableLike.scala:234)
[info] at scala.collection.TraversableLike.map$(TraversableLike.scala:227)
[info] ...
[info] ScalaTest

Re-running this test immediately afterwards worked. This happened during a deployment. Checking IP in browser manually seemed to work fine. Probably a timing issue.

Original UDP synchronization on ancestry lookups in chain bundles has some bugs under load and after long periods of time. REST endpoints for handling downloads need to be added (and should be supported long-term anyways.) Additionally need to speed up download logic.

Bug in kube where it's displaying the wrong node IP due to cluster update propagation speed. InetSocketAddress Java DNS resolver is getting the wrong IP. Easily fixed by just using the external string and bypassing InetSocketAddress.

Simple component prototype for selecting facilitators given the input of a window of chain state data (in the form of Set[Id]) and a tip hash, select facilitators by depth for next round.

We can swap out for more efficient like the Honeybadger BFT once we have the basic architecture implemented.

Deliverable: Create 200-500 word paragraph
Subject: Weekly update highlighting any news or progress in development over the past 2 week
Use case: Keep FUD at bay and engage with our existing community
Due date: May 3rd

Send content to: [email protected], [email protected]

Should keep track of all relevant peer info, i.e. any observed remote addresses, whether originating from REST / direct TCP / UDP / the string that the peer is originally added under. Should also contain sample data from ledger table for sync verification. Eventually should also incorporate more general sync info, such as SketchMap observations of their understanding of state of network.

Nodes should at regular intervals either query or push current list of peers to other nodes. They're currently keeping tracking of dead peers and automatically adding any that are unauthenticated, but it appears some kind of DNS change is interfering with reconnections. Cluster test showed a variable number of peers on each node, despite no restarts or other issues. For some reason nodes are not correctly maintaining set of all peers, but do appear to re-add sometimes. This may be related to iptables change?

Logs show a peer re-adding loop with DNS issue:

Intended for V2, whole process should act as a single stream and not require any heartbeat driver (apart possibly from a parity check)

In addition to regular stress tests we also need a series of control tests designed to lower memory as much as possible and increase throughput to force failures. These failures need to ensure that we get back proper stacktrace information and that no silent failure modes exist.

The main idea behind constellation is Hylochain, a self similar recursive extended trust chain of extended trust chains. After implementation of a basic star node, we need to find ways for star nodes to act as galaxies.

Galaxies are star nodes with two differences, they:

1. transact checkpoint blocks (hashed blocks of transactions) not individual transactions themselves and
2. keep track of star nodes that have 'turned on', or elected to begin gossiping and participate in consensus.

It seems very possible to enable this recursive structure using, namely, recursive data types. This would mean that a galaxy is merely a star where with modified parameters that indicate it's "scale" or whether it is gossiping transactions, checkpoint blocks or black holes. The number of "scale" layers will undoubtedly taper off, both as the size of the data gossiped hits a critical mass and due to the fact that we are bounded by a tradeoff between cryptographic security and the time it will take to perform a round of consensus (specifically, the fraction of delegated to number of nodes in a star cluster.)

The goal is to build an autoscaling service using recursive typing and investigate the best parameter combinations using a simulation environment.

For calculating most recent transactions for display, need to include time data for serialized transaction info. Also need to include ordering information on IDs that signed a particular transaction (and should be updated to only include the subset of IDs relative to the bundle for those that signed above the transaction.)

This is for me to keep track of my current work on #1. I want to reuse the scorex library's peer finder and so nodes can gossip over a live network. This will allow us to spin up star nodes on our local boxes and watch them gossip/perform consensus.

We can't depend on this library: https://github.com/Nycto/Hasher for every hash. Only 6 contributors and not widely reviewed / popular enough for cryptographic trust.

There's plenty of more popular libraries, i.e. apache / guava / etc. which have passed very strict reviews. See
https://stackoverflow.com/questions/5531455/how-to-hash-some-string-with-sha256-in-java

There may be some more exotic hashes we want that the popular libraries don't support -- so for those steps it's fine to use more niche libs. But those need to be reviewed extensively.

This won't matter for a while -- this is part of the cryptographic review epic which doesn't need to happen for a while, until project is out of initial testing phase.

Peer cleanup was temporarily disabled to debug other problems. Needs to be re-enabled and verify that it cannot accidentally remove live peers. Additionally we also need to pass more peer data during the handshake (which was turned off to enable clean testing.) On restart we need to add seed peers populated during deployment relative to the active cluster.

Sometimes if the nodes are restarted while under load the newly restarted instance will enter a crash loop. If /restart is called before this a few times, there's no problem.

TODO - Full explanation

Eventually we want all the existing crypto CLI / client code to be able to interact with constellation natively.

We should support JSON RPC protocol here (using RPC HTTP authentication as demonstrated by all the CLI clients.)

https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list

For now, we should at the very least try and support the majority of the methods on this api calls list through simple RPC procedural calls, and later add support to all of them matching the exact same input JSON.

I.e. {"method": "backupwallet" ... }

@RatulSaha is a pro at formal verification and is interested in spearheading this effort (woot!) It would be awesome if we could formulate a study investigating the following:

1. Given a distribution of numeric values that represent the byzantine-ness of a node participating in distributed consensus and a selective sampling technique for delegate selection, determine the probability that that nodes can collude in the form of a sybil-attack to censor transactions.
2. Prove that double signed transactions as made between two chains can prevent double spending, as demonstrated by CHECO and Hashgraph.

It would also be awesome to do some model checking or construct a proof in coq or TLA+.

http://www.scala-sbt.org/1.0/docs/Testing.html

Need to generate swagger docs from akka-http endpoints.

Should simplify some of the cleanup logic.

During each round of consensus, nodes decide both a new checkpoint block, but also on the set of delegates for the next round of consensus. This is possible by using a deterministic selective sampling method.

I proposed in the whitepaper to follow the approach of GURU which fits a probability density function to the distribution of reputation scores. Now in order to make this compatible with our underlying gossip protocol, the most scalable approach I've found is Temporal PageRank from P.Otte's work on "Sybil-resistant trust mechanisms in distributed systems." His experimental results show that scores tend to cluster in naturally occurring centroids, which is not necessarily ideal for fitting smooth probability densities proposed by GURU. However, to mitigate this we can deterministically sample from these centroids, where the proportion taken from each bucket follows the slope of the distribution. This isomorphic to the selection of a differential term in a numerical integration or the dx in a Reimann sum.

To break this down we need:

1. Implement deterministic discreet selective sampling for a probability density functions to begin testing/analysis for #4. The determinism part can be as simple as sorting the buckets with the previous block used for ordering.
2. Incorporate Temporal PageRank into consensus. This will likely require a multi block subprocess for information dissemination. We will need to factor this in to #6 as this is our main deviation from CHECO.
3. Implement and incorporate delegate selection using 1,2 into consensus.

Very simple. Need to refactor BlockChainApp to allow constructing an AppNode from alternative
configurations. (I.e. so it doesn't bind on the same ports.) This will have an accompanying unit test.

Should validate the internal state data and verify ledger and reputations are accurate. Should also compare with other peer data to determine network health.

We're gonna need a DHT for our autoscaling galaxy nodes. This is a solved program but would probably be a good first project for us to collaborate on once I've got a local network running.

Currently only sending combined bundles to peers that haven't yet signed it, but if the bundle is large enough to be accepted, we should be sending it to all peers so that they don't need to request the data later. Should also send TX data alongside instead of requiring requests.

Before was using a linear multiplier of bundle depth. Need to increase scores for higher depth bundles much more otherwise it will encourage stray chain formation.

Kubernetes /health endpoint should check if data is being updated, if node is healthy relative to peer information, if chain data has become corrupted / parity mismatch, etc.

Currently using b58 out of convenience since it's also used for addresses. Need to add b16 support for canonical key standard and remove the redundant key data prefix.

Currently our kube is setup to use a headless service and stateful sets.

We are running this inside of google compute engine https://cloud.google.com

We have an issue where the port we are exposing for akka remoting 2551 is not able to connect between pods.

Use the vagrant box instructions to compile and publish the docker image locally in vagrant

get an invite to gce

use gcloud init to setup connection to gce

run gcloud container clusters get-credentials constellation-test --zone us-central1-a --project esoteric-helix-197319

to setup the project on gcloud command line tools

run sudo docker tag constellationlabs/constellation gcr.io/esoteric-helix-197319/constellationlabs/constellation

to tag the docker container

run sudo gcloud docker -- push gcr.io/esoteric-helix-197319/constellationlabs/constellation

to push the latest version of your container to gcloud

kubectl get pods to view the current pods

kubectl logs pod-name to view logs of a specific pod

this is an example of a dns entry for a pod constellation-0.node-service.default.svc.cluster.local

make sure the kube proxy is running before trying the command below

sudo kubectl proxy --port=8001 &

this is an example of an add peer command that is failing which is what we would expect to work, seems to be timing out connecting to the other pod

curl -H "Content-Type: application/json" -X POST -d 'akka.tcp://BlockChain@constellation-0.node-service.default.svc.cluster.local:2551/user/constellation’ http://localhost:8001/api/v1/proxy/namespaces/default/pods/constellation-1/addPeer

to apply changes to the kube configs

cd into the constellation directory

run kubectl apply -R -f deploy/kubernetes/

This will update the configs but you need to destroy the current pods running and create new ones

the easiest way to do this is to run

kubectl delete statefulset constellation

and then run
kubectl apply -R -f deploy/kubernetes/

For unit tests we can just test wallet storage locally pretty easily. Same for 'simple' deploys of nodes.
We should probably support S3 for cluster integration tests for key storage.

There's a simple Java API for storage : https://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html

Or we could use the key encoder methods to dump them into Array[Byte] and handle ser/de ourselves.

Hi there,

I'm attempting to install the codebase on a Windows 10 box, using Vagrant and VIrtualBox.

FIrstly, will this configuration work?

1. I've attempted to fire it all up and have successfully deployed a box to VirtualBox (which Vagrant can 'talk' to, Vagrant has also prompted that I update the Ubuntu version, which I did.
2. Upon completion I was prompted to restart the machine, unfortunately when I attempt to Vagrant Up again, it times out at the SSH Auth Private Key stage. (The box in VirtualBox is starting up it appears) however Vagrant is not happy

Any ideas on what might be causing the issue and how I might be able to fix it?

Thanks

Formulating benchmarks for our users requires us to analyze data from these simulations and tests. Our simulation environment needs to store data in a format that can be parsed efficiently into a data analysis pipeline. The goal of this is to:

1. Construct a framework for storing data between code changes so we can easily index performance improvement
2. Build a data analysis pipeline
3. Construct analysis in a presentable format such as an ipython/jupyter notebook and connect this to our data store

Example is in SignTest. Causes interference with validation of signing data due to Kryo serializer changing the hostname that is serialized if it is called between successive JSON render calls. Need to initialize and kryo serialize a handshake upon node init to ensure consistent behavior in rendering.