ssbc / ssb-replication-scheduler Goto Github PK

I had an idea of implementing sympathetic replication like this:

partialReplication: {
  0: [
    { purpose: 'main' },
    { purpose: 'git-ssb' },
    { purpose: 'index' }
  ],

  1: [
    { purpose: 'main' },
    { purpose: 'git-ssb', $certainty: 50 },
  ],

Note $certainty: 50, this means that with 50% probability I will replicate a friend's git-ssb feed. This wouldn't be random, because we need every sbot session to always replicate the same "lucky" friend git-ssb feed, so determinism.

I thought that one way to achieve determinism is to pluck the first nibble of the friend's subfeed, and replicate that subfeed only if it belongs to my set of "chosen lucky nibble". With 50%, suppose that my deterministic lucky nibbles are: 0, 2, 4, 6, 8, a, c, e. If I had chosen 100% certainty, then the lucky nibbles would be all the 16 possible: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f. The lucky nibbles can be selected "randomly" too.

I don't know if this is too complicated, maybe there is a simpler and easier way of deterministically choosing whether a friend's subfeed is "lucky" given my certainty parameter, but my point is that sympathetic replication overall seems like it could be achieved with this simple $certainty value.

Thoughts @arj03 ?

When adding a lot of messages for a feed, right now you might get a follow and a unfollow or block right after. Would be good if these were debounced a bit so that ebt doesn't start requesting these messages.

Seems like we should treat the else case here:

Say I follow both Alice and Bob, but Alice blocks Bob.

It's currently a feature in ssb-replication-scheduler that we won't forward Alice's (main) feed to Bob, and it's implemented here:

But that's only for main feeds. Bob is still capable of replicating Alice's metafeed tree.

We should support preventing Bob from getting Alice's metafeed tree (any of Alice's feeds).

But there is an exception to this: Bob can always get Alice's group leaf feed (and the whole branch above that) if both Alice and Bob are in the same group.

When i change this to carol.db.get, which seems to be intended, I get an error

✖ 20) Error: Msg %BYhQdJTDN52qRn8Rh1RJYa5icwgMnWmjPQxxjalxZns=.sha256 not found in leveldb index
      Expected error to be falsy
      At: async Test.<anonymous> (/test/integration/groups.js:607:18)

I was having trouble with (partial?) replication, and things started to work once I installed ssb-subset-rpc, but that's not mentioned in the readme here.

my setup before the fix

module.exports = function startSbot() {
  const stack = SecretStack({ caps: { shs } })
    .use(require("ssb-db2/core"))
    .use(require("ssb-classic"))
    .use(require("ssb-bendy-butt"))
    .use(require("ssb-meta-feeds"))
    .use(require("ssb-box2"))
    .use(require("ssb-db2/compat/feedstate"))
    .use(require("ssb-db2/compat/ebt"))
    .use(require("ssb-db2/compat/db")) // for legacy replicate
    .use(require("ssb-db2/compat/history-stream")) // for legacy replicate
    .use(require("ssb-friends"))
    .use(require("ssb-ebt"))
    .use(require("ssb-tribes2"))
    .use(require("ssb-lan"))
    //.use(require("ssb-subset-rpc")) // this (uncommented) is all i added to fix it
    .use(require("ssb-replication-scheduler"));

  const sbot = stack({
    path: dir,
    keys,
    ebt: {
      // logging: true,
    },
    db2: {
      flushDebounce: 10,
      writeTimeout: 10,
    },
    tribes2: {
      // timeoutLow: opts.timeoutLow,
      // timeoutHigh: opts.timeoutHigh,
    },
    friends: {
      hops: 1,
    },
    replicationScheduler: {
      //debouncePeriod: 1,
      partialReplication: {
        0: [{}],
        1: [{ purpose: "main" }, { purpose: "group/additions" }],
        group: [{ purpose: "$groupSecret" }],
      },
    },
  });

  sbot.name = "demo";
  sbot.ebt.registerFormat(bendyButtFormat);

  return sbot;
};

(Poor title for this issue, and I'm not sure which repo to put this in, but this repo seems to be the highest level)

Context

• ssbc/ssb-ebt#72
• ssbc/ssb-db2#395
• ssbc/ssb-db2#411
• https://gitlab.com/staltz/manyverse/-/issues/2161

Problems

Consider what happens to messages in the log (avoid duplicates, avoid ssb-ebt replication bugs and incorrect state) when:

1. Switching from partial replication to full replication
   • Due to hops change: e.g. when a peer used to be hops 2 for you (and you had configured indexed feed replication for hops 2) but they change to hops 1 (which configures to replicate the main feed) because you followed them
   • Due to subset replication: e.g. when you fetched and appended a metafeed/announce msg (from the main feed) via ssb-subset-rpc but after that replicated the full main feed for that peer
2. Switching from full replication to partial replication
   • Due to upgrading to metafeeds: e.g. when a peer adopts a new version of the JS stack that has metafeeds, your peer will detect that and switch replication from full to partial (e.g. using indexed feeds)
3. Indexed feeds plus full replication
   • If a pub wants to fully replicate main feed AND replicate indexed-feed msgs then ssb-ebt needs to be changed so that indexed replication DOESNT do addOOO.

cc @arj03 FYI

Tasks

• ssb-ebt: Add benchmarks for indexed feed replication ssbc/ssb-ebt#73
• ssb-db2: addOOO should not update Base index ssbc/ssb-db2#395
• ssb-db2: Update deleteFeed to support ooo feeds in a world where addOOO doesn't update Base index ssbc/ssb-db2#417
• ssb-replication-scheduler: delete the main feed when switching from full replication to metafeeds and indexed feeds replication
• ssb-replication-scheduler: delete the partially replicated feeds when switching to full replication
• ssb-replication-scheduler: tests for all the use cases mentioned in "Problems" above
• (For perf) ssb-db2: New API to synchronously get the state (base.getLatest() is not that, it does a leveldb read)
• (For perf) ssb-ebt: Update indexed.js to refuse payloads in case state exists for that feed, and this must improve benchmarks
• (For bandwidth perf) ssb-ebt: specify ranges of sequences ssbc/ssb-ebt#80
• (For private group removal) ssb-ebt: specify range ssbc/ssb-ebt#79