Ways forward for go-ssb? about go-ssb

[KyleMaas]

As far as how the program is architected, I'd have to agree that there are some major difficulties in debugging and maintaining parts of this project. It's clear that a lot of it was architecturally copied from the JS implementation where it's only run single-threaded and then had multithreading tacked onto it without thought for how this impacts Go operations. The whole Badger/Margaret/Luigi thing makes it nearly impossible to do multithreading properly, since none of them are really being used in a way that's friendly to multithreading. The goal of having indexes be able to process asynchronously is a great idea in theory, but there's so much functionality that relies on the indexes being up-to-date at all times that it's nearly a pipe dream. All in all, this project is a single-threaded project desperately trying to masquerade as a multithreaded one.

The biggest barrier I've found to actual production use of go-ssb is EBT. Not having that working is a deal breaker. But how that works and what's wrong with it is beyond my abilities to grok. I'm thankful and hopeful that the progress y'all have been making with figuring that out will be fruitful. Because if that was working, I think we're almost at a point where this project could be considered "early beta" quality instead of "alpha".

Metafeeds, private groups, and gabbygrove support seem like they were experiments that only ever got the half-baked stage. Which is not bad, but they do add a lot of complexity. They feel like they were trying to aim to copy functionality from the JS side that wasn't quite done, and then never finished. I don't know how you'd remove any of them, though, because they're rather intertwined in the codebase.

Nulling of feeds is one that I do think is a good idea to keep around. Particularly for any pubs that are open to the internet, you'd want to be able to have the functionality to eliminate a single malicious feed from the pub without wiping the entire database and starting over. It definitely adds complexity but the benefits IMHO are worth it.

Re: concentrating efforts - Personally, I'm of the opinion that more diversity in the implementation ecosystem is better. The more the merrier. And that means that in my mind a working go-ssb, scuttlego, and JS implementations are not a waste of time to work on separately.

Also, from what I understand, scuttlego uses some of the functionality from this project in more of a library context, which gives us another reason why the base "library" should at least achieve beta status at some point. Saying the official Go implementation is scuttlego but then dropping support for underlying functionality in go-ssb seems a little silly to me.

Re: JS interoperability - I don't see a whole lot of problems with using NodeJS for interoperability, since it means we can continually update the libraries it's pulling in so we can test against newer versions. netsim doesn't seem like it would be as complete of a test. And proper interoperability with the JS implementation is crucial for the success of this project. Now would be a good adjunct to the way it works now? Sure, I'd absolutely agree that more testing is better for interoperability's sake. But I think having tests run against the JS side directly is still a good idea.

This is a good discussion to have, though. Big picture planning is something that's usually lacking in open source projects in my experience, so it's good that's happening here.

[cryptix]

I don't want to rehash history to much but these days I'm even less convinced that trying to replicate flumedb was necessary. It might be if you need to embedded a database into your program but there are other ways to go about this, too and maintaining a custom database is more work than writing a ssb implementation should be. Some years ago I played with using Postgres as a storage backend and I'd probably do this again these days just to keep my sanity.

I also agree with @KyleMaas in that this Codebase needed rearchitecting years ago. Cutting down scope seems sensible to me, too. It would be nice if the mentioned incomplete prototypes could live ontop of that new thing and might be cool to sketch them as use-cases.

I don't care so much if that thing is scuttlego or something new but I'm all for sunseting the go-sbot as there is so much cruft and keeping the good pieces.

(My two cents and also being clear that I don't have much time right now. This might change if I manage to reduce my Dayjob hours a bit later this year but no promises. I also want to make music in my free time.. 😅)

[decentral1se]

Re: concentrating efforts - Personally, I'm of the opinion that more diversity in the implementation ecosystem is better. The more the merrier. And that means that in my mind a working go-ssb, scuttlego, and JS implementations are not a waste of time to work on separately.

In theory, yes. However, the reality of having few active Go SSB hackers atm needs consideration here imho. We have a wider network but not everyone is focused on working on this code base atm. If we're all solo hacking on different moving parts, we're dispersing our efforts.

We got really far by focusing on the test suite in a coordinated fashion. I imagine we'd have similar boosts if we continued in this way. I just don't know which way right now.

There is a lot of fatigure in the SSB developer ecosystem around trying to get some working reliably and new people find it very hard to get started. So, centralising efforts to get one thing working well would naturally open it up wider than ever afterwards because more people could use the tools and develop their own stuff.

That could be scuttlego, go-ssb or another thing at this point, it's not clear to me.

Also, from what I understand, scuttlego uses some of the functionality from this project in more of a library context, which gives us another reason why the base "library" should at least achieve beta status at some point. Saying the official Go implementation is scuttlego but then dropping support for underlying functionality in go-ssb seems a little silly to me.

scuttlego uses some internals but not all. More building blocks afaiu. So, go-ssb could go away and not affect it too much. Probably some code would need to be pulled out and modularised. Also, there is a plan to make scuttlego have pub magic powers.

[decentral1se]

https://github.com/ssbc/minibutt-spec

[decentral1se]

Some great thoughts, thanks all for weighing in.

Folks are welcome to pick up a direction from here. Will close for now.