Some user feedback from a test session about ipfs-cluster

[jbenet]

cc @whyrusleeping who took other notes, i think.

[jbenet]

@hsanjuan maybe close after you've read it / processed it as you want. I will try it one more time with my own machines soon and get you more feedback 👍

[hsanjuan]

wish it gave me more info on connectivity (who's connected to who?)

--enc json shows more info about the peerset as seen by each peer.

looks like right now consensus is clearing (committed and done) after the pin entry

Not sure, about this, ctl pin ls shows the consensus state and allocations. Adding to consensus is phase 1 and happens fast. ctl status shows local status aggregated from cluster peers which, after a successful consensus entry, pin the hash. That is phase 2.

[whyrusleeping]

My notes:

• cluster-ctl peers ls doesnt actually tell me which address was used to connect to a given peer
  • it would be nice if it showed which multiaddr my active connection was using
• Reconnecting peers seems to mess with consensus, did the following:
  • start node1
  • start node2, join to node1
  • node2 pins object, it pins to both machines
  • node3 joins
  • node1 pins object, it seems to work on all machines
  • node3 leaves
  • node2 leaves
  • node3 rejoins, fails with 'consensus start timing out'
• Pinning seems to be weird, I see pin state get updated, my node doesnt pin anything yet, then i see a log message saying that my node failed to verify it had the right pins or something (forgot to write down the exact message), and then immediately after that, a message saying my node pinned the thing sucessfully

[hsanjuan]

@whyrusleeping by joins and leaves, do you mean using --bootstrap or peer add? and did you use peer rm, or just ctrl-c the nodes?

Regarding the log messages, it's just confusing because it prints an error when it checks if ipfs has a pin before pinning (it won't pin if it has it already). When Ipfs does not have a pin it returns a 500 (which is indistinguishable from more serious errors though...). Anyway I'll make sure this gets fixed, it's been in the back of my head for a while.

[whyrusleeping]

@hsanjuan we were just ctrl+c'ing the nodes. For connects though we used peer add

[hsanjuan]

@whyrusleeping I don't manage to reproduce this. Not with 3 nodes at least. With 4 nodes yes. If you kill 3 of them and bring up one, it won't start because there is no majority to elect a raft leader (2 votes out of 4 peers).

By the way I have removed the weird pinning message you saw.