jlongster / crdt-example-app

A full implementation of CRDTs using hybrid logical clocks and a demo app that uses it

This is a demo app used for my dotJS 2019 talk "CRDTs for Mortals"

Slides here: https://jlongster.com/s/dotjs-crdt-slides.pdf

View this app here: https://crdt.jlongster.com

It contains a full implementation of hybrid logical clocks to generate timestamp for causal ordering of messages. Using these timestamps, CRDTs can be easily used to change local data that also syncs to multiple devices. This also contains an implementation of a merkle tree to check consistency of the data to make sure all clients are in sync.

It provides a server to store and retrieve messages, so that clients don't have to connect peer-to-peer.

The entire implementation is tiny, but provides a robust mechanism for writing distributed apps:

• Server: 132 lines of JS
• Client: 639 lines of JS

(This does not include main.js in the client which is the implementation of the app. This is just showing the tiny size of everything needed to build an app)

Links:

• Actual: https://actualbudget.com/
• Hybrid logical clocks: https://cse.buffalo.edu/tech-reports/2014-04.pdf
• CRDTs: https://bit.ly/2DMk0AD
• Live app: https://crdt.jlongster.com/

crdt-example-app's Issues

Doesn't serve up from express app

The page isn't loading when visiting http://localhost:8006

Merkle tree size

First of all thank you for providing this insightful example.
One thing I cannot quite wrap my head around is the merkle tree used for data consistency. The way I understand it is that the tree contains hashes for all events from all clients ever. Wouldn't this lead to an ever growing data structure which is passed back and forth on each sync? Or am I missing something and the size of the tree is somehow bounded?

[question] Is it possible to subscribe dynamically to ToDo notes?

For example, Bob shares the link to subscribe on note22, Alice subscribes on note22. Now Alice gets all updates on note22.

Before:
Alice
-note11
-note12
-note13

Bob
-note21
-note22

After:
Alice
-note11
-note12
-note13
-note22

Bob
-note21
-note22

One of the approaches is that every user has own Merkle tree, and when Alice subscribes on note22, we insert timestamps of note22 to the Alice Merkle tree, so she will get an update on the next sync.

why does recv validate clock drift?

in timestamp.js, recv validates that the received message has not drifted too far from the clock. I'm trying to reimplement this from the paper and your implementation and am running into a problem with this in the following situation:

1. generate a timestamp in the present. The state of the clock is now up-to-date.
2. receive an timestamp some time in the past. I'm just artificially subtracting two minutes worth of milliseconds to get this.
3. it's a clock drift error!

It looks like this implementation will reject any remote messages further in the past than the maximum clock skew. If it doesn't, I don't actually see how! I also kinda see where they mention this check in the original paper but not why it has to be on receive.

So, am I missing something? Is this a bug, or a desired behavior for reasons I'm just not currently getting?

The order of the added messages is different in 2 application instances

The sync doesn't work correctly for this case:

• opened this application in 2 different browser tabs.
• typed and submitted h1 in the right tab - 1st snapshot
• typed and submitted h2 in the left tab - 1st snapshot (both messages nicely synced in the left tab)
• clicked on Sync in the right tab
• the order of the messages is incorrect in the right tab - 2nd snapshot

The sync works nicely if you explicitly press Sync button on every submission. To be more precise, you type message in one tab and press Sync button in the other tab before typing a new message in this other tab.

For what is operator `| 0`?

crdt-example-app/shared/merkle.js

Line 23 in 3acd310

let key = Number((timestamp.millis() / 1000 / 60) | 0).toString(3);

Database isn't initialized

Upon first running the app, the database isn't initialized.

I made a library from this code

It could be interesting for someone, so I'm leaving the link here.

https://github.com/evoluhq/evolu