Suggestion: Return 2 separate parameters in `onnotice()` or `onerror()` about nostr-relaypool-ts

[adamritter]

Sure, I didn't really need it in my code base, but feel free to fix it and I will merge it 😀 My next plan is to optimize batching thousands of subscriptions and fix bugs in event handling as I'm developing a client to show case the library.

…

On Sun, Jan 15, 2023, 14:49 Callum Macdonald ***@***.***> wrote: Looking at the API in README and the code, it looks like you concatenate the relay (presumably the relay URL) and error or notice params with a colon. I imagine that I want to know which relay generated which notices, so I can tell the user "this relay sent this X". Of course I could extract it from the string, but it's a bit more complicated to get just the error out of a string like this: wss://foo:error_with_colon:second_message. I'd imagine something like (relay: string, message: string) => void as a signature. — Reply to this email directly, view it on GitHub <#18>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN5SWACPJOSZBMMLIBDQ2HLWSQE65ANCNFSM6AAAAAAT345M3U> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[chmac]

Haha! I wanted to use your relaypool implementation, but the caching stuff was a bit too opaque for me to easily understand. I ended up writing a thin layer over redux instead. The filter deduplication stuff you've written is a nice idea, I had similar thoughts. I found the code impossible to understand, plus I'm a bit wary of let although I admit that's just a prejudice on my part! Anyway, all of that means I'm not using the codebase so I won't submit a PR. Just wanted to share the suggestion in case you think it's useful.

[adamritter]

Hey thanks for thinking about trying to use my code base. I see lots of clients popping up, and I feel like lots of coding talent is getting wasted. We are already a team of developers who if we work as a team and stick together, we can great traction, but if we keep separated, and write new code bases, nostr will still grow, but much slower. That was my main reason of trying to build a library that is just one extra thing every client must handle, and also in a non-frontent-framework specific way. I know that the code base is not as clean as it should be, but my primary goal was to keep it clean enough for me to get to a state where the API doesn't have to change that often, so clients can build upon it. I think right now for subscribing the API is in a very good shape I'm happy to clean the parts of the code you want to understand, also add more test cases for the parts you need to use the library. Can you tell me if I can do something for you / what can I do to start using the library? You can get specific about the code base and the parts that need to be cleaner.

…

On Mon, Jan 16, 2023 at 10:19 AM Callum Macdonald ***@***.***> wrote: Haha! I wanted to use your relaypool implementation, but the caching stuff was a bit too opaque for me to easily understand. I ended up writing a thin layer over redux instead. The filter deduplication stuff you've written is a nice idea, I had similar thoughts. I found the code impossible to understand, plus I'm a bit wary of let although I admit that's just a prejudice on my part! Anyway, all of that means I'm not using the codebase so I won't submit a PR. Just wanted to share the suggestion in case you think it's useful. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[chmac]

I appreciate you being open to feedback. It's not always the case!

I also hear you on everyone reinventing the wheel. It's helpful to have something like nostr-tools which seems to be a standard.

I'm happy to have a closer look at the code and open some issues if you like. For example I just looked at relay.ts and it seems like it's the same or similar to relay.ts in nostr-tools. Maybe there's a good reason for having it in there? An explanation as a comment at the top of the file might make that knowledge more accessible.

In general I'm a big fan of comments. Lots of them. I loved literate coffeescript where you write markdown files and the indented code blocks get executed, more comments than code!

Trying to read relay-pool, my mind is breaking! I think one of the biggest issues is the code formatting. Have you considered using prettier? I'm so used to prettier output by now, that personally, I struggle to read code properly without it. When I copy and paste the file into my editor and format it, I get a lot further.

There seems to be a ton of functionality in that one file. I wonder if there's a way to break it into smaller parts which are individually easier to understand?

The readme could also be a lot simpler to read. More headings, even just more formatting like code blocks on variable names and so on would make it easier to parse.

My sense is that the nostr space is pretty chaotic at the moment. I guess it'll take a bit of time for that to settle down. My advice for nostr-relaypool would be to work on the docs and simplifying the explanation of what it does. It seems like there's a ton of features in there. Like aggregating filters, sending subscriptions to specific relays, caching results, logic to handle different replaceable events, and so on. But I can't tell how much of which of those things works properly and overlaps. Like what happens if multiple relays return different nip33 events? I think I'd only be able to figure that out by testing, I don't think I'd be able to answer that by reading the code.

[adamritter]

Thanks, great comments, I'll try to answer each of them (and work on them as well): About relay.ts, I submitted a pull request to nostr-tools, but fiatjaf preferred not to put this change into nostr-tools: nbd-wtf/nostr-tools#73 As I wanted to build complex logic on top of a Relay implementation, I thought that handling WebSocket connection / reconnection has to be done by the Relay. As fiatjaf didn't agree with me, I had to fork the code (luckily I prefer MIT license just like him, I don't like the AGPL3 copyleft license direction that the community has taken, as it's not compatible with any other license). At the same time, it's very hard to read code, so I was afraid of touching it too much. I tried to refactor it, but it's a lot of work to do it well, and needs better tests first. I created an in memory relay implementation (another thing I notified fiatjaf about, but he didn't want to invest the time into putting it into nostr-tools). We need a heavy refactoring of Relay (for example to add auto reconnection with exponential backoff), but as it works fine, I didn't want to focus on it for some time.

…

---- About comments: I prefer unit tests and clean code to comments. I'm in the ,,if you don't understand the code by reading it, you need to clean it up'' camp. I loved Robert Martin's Clean code youtube lectures, and I'm happy that he's part of the nostr community: https://www.youtube.com/watch?v=7EmboKQH8lM

-- about prettier: sure, I'll try it and probably adopt it, as I'm doing a lot of formatting by hand and it's boring anyways.

--- ,,There seems to be a ton of functionality in that one file. I wonder if there's a way to break it into smaller parts which are individually easier to understand?'' You are totally right, I'll create a caching class, and filtering class for sure, probably a timer class as well, though that's right now not that complex code. Again, I was focusing on the API being right before doing these things, but it's great to have an external point of view.

--- about nip33: it's the first time I see it to tell you the truth (I don't know what it is for), but I can point you to how to implement it: There are two kind of caching, the ,,inside a subscription'' caching, and caching between subscriptions. doNotEmitOlder events function gets an onEvent callback, and filters events that get older for kinds 0 and 3. It's quite easy to modify it to handle NIP33 events. The other type of caching is similar to metadataByPubKey, so probably the best way to implement it by looking at how that variable is used. These two types of caching methods are implemented orthogonally and can be turned off separately as I wasn't sure about how they will be used on the higher layers. There are lots of caching strategies that can be implemented (should the cached result be returned instatly? Should the subscription be sent even if there's a cached result? Should it be sent only if there are other subscription requests sent anyways?) To answer these questions, and the options the library should provide, clients need to be built and experimented with.

On Mon, Jan 16, 2023 at 11:43 AM Callum Macdonald ***@***.***> wrote: I appreciate you being open to feedback. It's not always the case! I also hear you on everyone reinventing the wheel. It's helpful to have something like nostr-tools which seems to be a standard. I'm happy to have a closer look at the code and open some issues if you like. For example I just looked at relay.ts and it seems like it's the same or similar to relay.ts in nostr-tools. Maybe there's a good reason for having it in there? An explanation as a comment at the top of the file might make that knowledge more accessible. In general I'm a big fan of comments. Lots of them. I loved literate coffeescript where you write markdown files and the indented code blocks get executed, more comments than code! Trying to read relay-pool, my mind is breaking! I think one of the biggest issues is the code formatting. Have you considered using prettier? I'm so used to prettier output by now, that personally, I struggle to read code properly without it. When I copy and paste the file into my editor and format it, I get a lot further. There seems to be a ton of functionality in that one file. I wonder if there's a way to break it into smaller parts which are individually easier to understand? The readme could also be a lot simpler to read. More headings, even just more formatting like code blocks on variable names and so on would make it easier to parse. My sense is that the nostr space is pretty chaotic at the moment. I guess it'll take a bit of time for that to settle down. My advice for nostr-relaypool would be to work on the docs and simplifying the explanation of what it does. It seems like there's a ton of features in there. Like aggregating filters, sending subscriptions to specific relays, caching results, logic to handle different replaceable events, and so on. But I can't tell how much of which of those things works properly and overlaps. Like what happens if multiple relays return different nip33 events? I think I'd only be able to figure that out by testing, I don't think I'd be able to answer that by reading the code. — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN5SWADFDOCRTLDWIETTX7LWSUX7PANCNFSM6AAAAAAT345M3U> . You are receiving this because you commented.Message ID: ***@***.***>

[adamritter]

One more thing about the documentation: if you have ideas how to make it look nicer, you can just edit it and send a pull request, it would be great.

…

On Mon, Jan 16, 2023 at 12:32 PM Adam Ritter ***@***.***> wrote: Thanks, great comments, I'll try to answer each of them (and work on them as well): About relay.ts, I submitted a pull request to nostr-tools, but fiatjaf preferred not to put this change into nostr-tools: nbd-wtf/nostr-tools#73 As I wanted to build complex logic on top of a Relay implementation, I thought that handling WebSocket connection / reconnection has to be done by the Relay. As fiatjaf didn't agree with me, I had to fork the code (luckily I prefer MIT license just like him, I don't like the AGPL3 copyleft license direction that the community has taken, as it's not compatible with any other license). At the same time, it's very hard to read code, so I was afraid of touching it too much. I tried to refactor it, but it's a lot of work to do it well, and needs better tests first. I created an in memory relay implementation (another thing I notified fiatjaf about, but he didn't want to invest the time into putting it into nostr-tools). We need a heavy refactoring of Relay (for example to add auto reconnection with exponential backoff), but as it works fine, I didn't want to focus on it for some time. ---- About comments: I prefer unit tests and clean code to comments. I'm in the ,,if you don't understand the code by reading it, you need to clean it up'' camp. I loved Robert Martin's Clean code youtube lectures, and I'm happy that he's part of the nostr community: https://www.youtube.com/watch?v=7EmboKQH8lM -- about prettier: sure, I'll try it and probably adopt it, as I'm doing a lot of formatting by hand and it's boring anyways. --- ,,There seems to be a ton of functionality in that one file. I wonder if there's a way to break it into smaller parts which are individually easier to understand?'' You are totally right, I'll create a caching class, and filtering class for sure, probably a timer class as well, though that's right now not that complex code. Again, I was focusing on the API being right before doing these things, but it's great to have an external point of view. --- about nip33: it's the first time I see it to tell you the truth (I don't know what it is for), but I can point you to how to implement it: There are two kind of caching, the ,,inside a subscription'' caching, and caching between subscriptions. doNotEmitOlder events function gets an onEvent callback, and filters events that get older for kinds 0 and 3. It's quite easy to modify it to handle NIP33 events. The other type of caching is similar to metadataByPubKey, so probably the best way to implement it by looking at how that variable is used. These two types of caching methods are implemented orthogonally and can be turned off separately as I wasn't sure about how they will be used on the higher layers. There are lots of caching strategies that can be implemented (should the cached result be returned instatly? Should the subscription be sent even if there's a cached result? Should it be sent only if there are other subscription requests sent anyways?) To answer these questions, and the options the library should provide, clients need to be built and experimented with. On Mon, Jan 16, 2023 at 11:43 AM Callum Macdonald < ***@***.***> wrote: > I appreciate you being open to feedback. It's not always the case! > > I also hear you on everyone reinventing the wheel. It's helpful to have > something like nostr-tools which seems to be a standard. > > I'm happy to have a closer look at the code and open some issues if you > like. For example I just looked at relay.ts and it seems like it's the > same or similar to relay.ts in nostr-tools. Maybe there's a good reason > for having it in there? An explanation as a comment at the top of the file > might make that knowledge more accessible. > > In general I'm a big fan of comments. Lots of them. I loved literate > coffeescript where you write markdown files and the indented code blocks > get executed, more comments than code! > > Trying to read relay-pool, my mind is breaking! I think one of the > biggest issues is the code formatting. Have you considered using prettier? > I'm so used to prettier output by now, that personally, I struggle to read > code properly without it. When I copy and paste the file into my editor and > format it, I get a lot further. > > There seems to be a ton of functionality in that one file. I wonder if > there's a way to break it into smaller parts which are individually easier > to understand? > > The readme could also be a lot simpler to read. More headings, even just > more formatting like code blocks on variable names and so on would make > it easier to parse. > > My sense is that the nostr space is pretty chaotic at the moment. I guess > it'll take a bit of time for that to settle down. My advice for > nostr-relaypool would be to work on the docs and simplifying the > explanation of what it does. It seems like there's a ton of features in > there. Like aggregating filters, sending subscriptions to specific relays, > caching results, logic to handle different replaceable events, and so on. > But I can't tell how much of which of those things works properly and > overlaps. Like what happens if multiple relays return different nip33 > events? I think I'd only be able to figure that out by testing, I don't > think I'd be able to answer that by reading the code. > > — > Reply to this email directly, view it on GitHub > <#18 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AN5SWADFDOCRTLDWIETTX7LWSUX7PANCNFSM6AAAAAAT345M3U> > . > You are receiving this because you commented.Message ID: > ***@***.***> >

[adamritter]

Just as a follow up, I added prettier, and split off event caching and OnEvent callback filters into separate files, and also the functionality that preprocesses a subscription and prepares it for multiplexing subscriptions and demultiplexing relays. prettier + fast unit testing + typescript helps a lot in fast refactoring, I'm quite happy with how easy it went. Still, the main program is still over 200 lines of code, but I hope it's a little bit easier to understand. I can do more refactoring (and a bit of documenting) if you tell me where the biggest pain points are for understanding the code base.

…

On Mon, Jan 16, 2023 at 12:39 PM Adam Ritter ***@***.***> wrote: One more thing about the documentation: if you have ideas how to make it look nicer, you can just edit it and send a pull request, it would be great. On Mon, Jan 16, 2023 at 12:32 PM Adam Ritter ***@***.***> wrote: > Thanks, great comments, I'll try to answer each of them (and work on them > as well): > > About relay.ts, I submitted a pull request to nostr-tools, but fiatjaf > preferred not to put this change into nostr-tools: > > nbd-wtf/nostr-tools#73 > > As I wanted to build complex logic on top of a Relay implementation, I > thought that handling WebSocket connection / reconnection has to be done by > the Relay. As fiatjaf didn't agree with me, I had to fork the code (luckily > I prefer MIT license just like him, I don't like the AGPL3 copyleft license > direction that the community has taken, as it's not compatible with any > other license). > > At the same time, it's very hard to read code, so I was afraid of > touching it too much. I tried to refactor it, but it's a lot of work to do > it well, and needs better tests first. I created an in memory relay > implementation (another thing I notified fiatjaf about, but he didn't want > to invest the time into putting it into nostr-tools). > > We need a heavy refactoring of Relay (for example to add auto > reconnection with exponential backoff), but as it works fine, I didn't want > to focus on it for some time. > > ---- > > About comments: I prefer unit tests and clean code to comments. I'm in > the ,,if you don't understand the code by reading it, you need to clean it > up'' camp. > > I loved Robert Martin's Clean code youtube lectures, and I'm happy that > he's part of the nostr community: > https://www.youtube.com/watch?v=7EmboKQH8lM > > -- > > about prettier: sure, I'll try it and probably adopt it, as I'm doing a > lot of formatting by hand and it's boring anyways. > > --- > > ,,There seems to be a ton of functionality in that one file. I wonder if > there's a way to break it into smaller parts which are individually easier > to understand?'' > > You are totally right, I'll create a caching class, and filtering class > for sure, probably a timer class as well, though that's right now not that > complex code. > > Again, I was focusing on the API being right before doing these things, > but it's great to have an external point of view. > > --- > > about nip33: it's the first time I see it to tell you the truth (I don't > know what it is for), but I can point you to how to implement it: > > There are two kind of caching, the ,,inside a subscription'' caching, and > caching between subscriptions. > > doNotEmitOlder events function gets an onEvent callback, and filters > events that get older for kinds 0 and 3. > > It's quite easy to modify it to handle NIP33 events. > > The other type of caching is similar to metadataByPubKey, so probably > the best way to implement it by looking at how that variable is used. > > These two types of caching methods are implemented orthogonally and can > be turned off separately as I wasn't sure about how they will be used on > the higher layers. > > There are lots of caching strategies that can be implemented (should the > cached result be returned instatly? Should the subscription be sent even if > there's a cached result? Should it be sent only if there are other > subscription requests sent anyways?) > > To answer these questions, and the options the library should provide, > clients need to be built and experimented with. > > > > > > > On Mon, Jan 16, 2023 at 11:43 AM Callum Macdonald < > ***@***.***> wrote: > >> I appreciate you being open to feedback. It's not always the case! >> >> I also hear you on everyone reinventing the wheel. It's helpful to have >> something like nostr-tools which seems to be a standard. >> >> I'm happy to have a closer look at the code and open some issues if you >> like. For example I just looked at relay.ts and it seems like it's the >> same or similar to relay.ts in nostr-tools. Maybe there's a good reason >> for having it in there? An explanation as a comment at the top of the file >> might make that knowledge more accessible. >> >> In general I'm a big fan of comments. Lots of them. I loved literate >> coffeescript where you write markdown files and the indented code blocks >> get executed, more comments than code! >> >> Trying to read relay-pool, my mind is breaking! I think one of the >> biggest issues is the code formatting. Have you considered using prettier? >> I'm so used to prettier output by now, that personally, I struggle to read >> code properly without it. When I copy and paste the file into my editor and >> format it, I get a lot further. >> >> There seems to be a ton of functionality in that one file. I wonder if >> there's a way to break it into smaller parts which are individually easier >> to understand? >> >> The readme could also be a lot simpler to read. More headings, even just >> more formatting like code blocks on variable names and so on would make >> it easier to parse. >> >> My sense is that the nostr space is pretty chaotic at the moment. I >> guess it'll take a bit of time for that to settle down. My advice for >> nostr-relaypool would be to work on the docs and simplifying the >> explanation of what it does. It seems like there's a ton of features in >> there. Like aggregating filters, sending subscriptions to specific relays, >> caching results, logic to handle different replaceable events, and so on. >> But I can't tell how much of which of those things works properly and >> overlaps. Like what happens if multiple relays return different nip33 >> events? I think I'd only be able to figure that out by testing, I don't >> think I'd be able to answer that by reading the code. >> >> — >> Reply to this email directly, view it on GitHub >> <#18 (comment)>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/AN5SWADFDOCRTLDWIETTX7LWSUX7PANCNFSM6AAAAAAT345M3U> >> . >> You are receiving this because you commented.Message ID: >> ***@***.***> >> >

[adamritter]

I got under 200 lines in relay-pool.ts :)

…

Message ID: ***@***.*** >>> com> >>> >>

[adamritter]

@chmac Hi, I implemented this issue, still it would be nice to talk more, the feedback you provided was appreciated. You can reach me on Telegram as well @Adam8812

[chmac]

Cool, awesome to see so many updates. I'll ping you now on telegram, I'm also @chmac on there.

I'll look at the new version now and share any feedback.