Comments (11)
@dwblair oh wow nice! i just made public a thing i've been working on here and there called crepes (backronym: cabal replication diagnostics), which is very similar to what you've done!
https://github.com/cblgh/cabal-crepes
the structure: you have puppets (a special kind of client using a new cabal library i made) which connect to a cabal and also to a websockets server
it's intended as a kind of testbed so that you can individually control each client and see things such as puppet#1 sent a message at time t
but puppet#2 didn't receive it until time t+100
. this is possible thanks to sending all of the information to the websockets server. you can also do things like individually disconnect the puppets from the cabal swarm, but keep them posting to their local database, and then reconnect them.
the websockets server is also an http server, and i've currently been interfacing with it using curl and sending requests as curl -X POST <addr>/disconnect/0
. check out central.js
for the api
i've also been seeing similar stuff as you are wrt peer added and peer dropped, but i don't have a graph yet! i'd love to merge our efforts ^_^
as an end result, i want to both produce a log.txt (and a .csv, since that seems really easy to use for the analysis you've done) as well as visualize all of the information on a page served by the webserver, and allow all control of the various clients from the same webpage.
(i'll try to work a bit on crepes and document it later today (hopefully!))
(it's also a minimal kind of cabal client interface headless.js
which i intend to publish independently of the repo to npm, so that others can use in exactly these kinds of experiments ^_^)
from cabal-core.
@dwblair this has come up a few times previously and it's not a half-bad idea at all, especially not when framed from a trust perspective.
additionally, i quite like the idea of deriving it from the peer (given some kind of out-of-log message, we could e.g. do it by sending each peer's uptime, calculated from when they were started)
thanks so much for your well-thought out proposals and your determined energy, i'm always glad when i see something new from you and continually (positively) surprised that you've stuck around cabal for so long already :3
from cabal-core.
(And apologies for just making suggestions & not just test-implementing it myself -- I do hope to eventually learn enough JS to do that :))
from cabal-core.
from cabal-core.
"Peer-added" events for all peers in public cabal over last 24 hours
Hi All!
In order to start to explore the message propagation dynamics in a cabal network, I wanted to get a sense for the statistics of peers appearing online together. (I don't think I'm there, yet ...)
I tweaked the cabal-core's "cli.js" so that it would log a timestamp and peer key on every "peer-added" and "peer-dropped" event, to a logfile "log.txt"
My expectation was that I would see peers "added" to the swarm ... and then stay swarmed for a stretch of "online time" ... and then I would see them "dropped". And my job would be to try to graph out those "online time" stretches ...
Instead, I saw that for a given peer, there would be a long string of "peer-added" events, one right after the other ... and then maybe a "peer-dropped" here and there. I think I recall (vaguely) that the discovery-swarm code sort of "pings" the network frequently as part of its bag of hole-punching tricks (maybe?) -- so maybe this is why I'm seeing this? Curious ...
In any case, what I'm showing in the above graph are all of the "peer-added" events for all of the peers that happened to be in the public cabal during the last 24 hours or so. Each horizontal row is a different peer (denoted by a different color); the short vertical bars represent each "peer-added" event for that peer. The x-axis is the time in hours (measured from when I started logging).
I don't know if this plot of "peer-added" events captures the entire amount of time that peers are online -- again, as above, I would think that once a peer was "added", it would just "stay online". So the amount of time each peer was online is likely at least as long as indicated in the graph, but perhaps (much) longer.
Anyway ... the code (including the Jupyter analysis code) is here (very crude, and undocumented as yet, apologies!):
https://github.com/edgecollective/cabal-logger
If anyone has any suggestions as to how I might better use the "peer-added" and "peer-dropped" information to derive "peer online times", I already have this machinery put together, so I could probably implement it quickly -- lemme know!
Cheerio!
Don
p.s. It just occurred to me that I should also check to see which of the peers is 'me' (as the peer doing the logging), haha -- that might help me understand the relationship between "peer-added" events and online time ...
p.p.s. I also tried adding in the "dropped" events, denoted below by a black, vertical bar "|" symbol. Here's the resultant graph:
"Peer-added" and "peer-dropped" events for all peers in public cabal over last 24 hours
Looking at the graph and glancing at the log data, it seems to me that the pattern is tthat "peer-dropped" events happen almost immediately after "peer-added" events, and very frequently. Still chewing this one over ... :)
from cabal-core.
@dwblair i wrote up a small tutorial on the wip-wss branch
from cabal-core.
Fantastic -- this looks really clearly written -- can't wait to play with it!
(And quick update -- for whatever its significance, here's a graph of 'peer-added' events for the last 60 hours:
More soon -- cheers!
from cabal-core.
@cblgh Haha thank you! I'm just so amazed by what you are all putting together here -- it's all so far-sighted and inspiring! I'm hoping to run some urban resilience / rural farm workshops locally around this stuff very soon & it's been so exciting to see it all develop so rapidly and with such good vibes all around. (I'll fire off any materials I develop for that to get input from y'all before / during after -- should be fun!)
& yes -- peer uptime would be so cool -- and sort of democratic -- even just someone who happens to leave their cabal connected on their desktop all the time could end up being a very reliable 'relay', no fuss no muss!
Onward!
from cabal-core.
nick and kira just started doing that recently manually! i thought it looked nice :)
from cabal-core.
from cabal-core.
Oh wow cooool!! Haha it's always so much fun to see that you cabal folks are always 10E2 steps ahead :) This looks super useful, and opens up all sorts of network analyses. Can't wait to dig into it (sorry this is quick traveling now, will do tonight / tomorrow). Imagining all sorts of fun p2p networking analyses made possible with this tooling! (And indeed headless.js seems useful all on its own, too ... )
(It also makes we wonder whether -- if peers are willing to share data about their own peer connectivity histories) -- we could build up fun graph stats of the larger cabal network? Not even sure how useful that is, but I'm super curious to see neat visualizations around it :)) Whee!
from cabal-core.
Related Issues (20)
- hidden users can edit channel topic
- sodium-native undefined symbol causes startup crash
- Generate typescript definitions from JSDoc
- Status messages HOT 2
- ephemeral messages HOT 2
- Experiment with `bridge: true` property HOT 1
- Message ordering HOT 10
- Sync & discovery inconsistencies HOT 3
- Add destroy method HOT 3
- Add WebRTC Fallback? HOT 5
- Emit event for new feed being added HOT 1
- Export / import swarm nodes
- Add search to cabal HOT 1
- Unknown users are joining private cabals HOT 5
- host from URL is undefined in electron/browsers
- [moderation] Case where a mod can +mod or +admin others HOT 2
- Private cabals HOT 2
- Does cabal have any sort of flooding resistance? HOT 2
- Specify a default listening port HOT 2
- Suggestion: make it clear which is the main repo to track HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cabal-core.