Support for UDP transport about thousand_island

[njouanin]

I started working on an elixir implementation of the matter protocol which relies on UDP.
There's also a need for mDNS (UDP based) support.

[mtrudel]

UDP support is very much something that I'd like to land (and something I have a strict requirement for based on mtrudel/bandit#28), but I'm not sure yet that Thousand Island is the place for it. The UDP API is very different from TCP, and the ThousandIsland.Transport behaviour doesn't capture it well at all.

My very loose plan (that I haven't validated beyond very preliminary research) is Thousand Island will expose QUIC as a transport alongside TCP and TLS based on their similarities as connection-oriented protocols, and that Thousand Island's QUIC support would be based on UDP support implemented somewhere (I'm just not sure if that somewhere is within Thousand Island or its own project).

All that being said, I'm not planning on tackling this personally until after Bandit 0.8 lands. I would say that 90% of the difficulty in this workup would be building the right abstraction; the code-level work here is pretty close to trivial. What specific use cases were you looking at?

[mtrudel]

Interesting!

In terms of mDNS servers, I've tried all of them over the life of HAP, and have been left wanting by all of them for one reason or another. I've settled on mdns_lite (not least because @fhunleth is the most responsive maintainer on the planet). Looking at their implementation it seems that they've moved to using :socket directly (going even lower from :gen_udp, the opposite direction than an abstraction such as Thousand Island would provide).

[ryanwinchester]

I started working on an elixir implementation of the matter protocol which relies on UDP.

Nice! I was working on Z-Wave for Elixir until SmartRent released Grizzly (which is great, btw). Now, I'm not touching anything until I see how Matter pans out. 😅

[njouanin]

Nice! I was working on Z-Wave for Elixir until SmartRent released Grizzly (which is great, btw). Now, I'm not touching anything until I see how Matter pans out. 😅

This is still early work. I still fight with specification documents which is not always clear ...

[mtrudel]

Closing due to nothing being actionable on Thousand Island's side.

[hissssst]

@mtrudel @njouanin so are you implementing this feature in thousand island (or relative project)?

I need UDP ranch-like pool for socks5 proxy I am writing, and I think that instead of writing my own pool, I could implement it as a part of thousand_island. Do you need this implementation, or are you already writing one?

[mtrudel]

I haven't started anything yet, but I've spent a bit of time researching and I'm not really sure yet what the right abstraction is here. UDP is rather a bit different and I'm conscious of not putting a round peg in a square hole.

Do you have any implementation ideas? I'd love to chat them over!

[hissssst]

I thought about implementing ThousandIsland.Transport backed by gen_udp. Some callbacks like handshake or secure? could be simply ignored (or can return default values like {:error, :not_implemented} and false)

[hissssst]

Of course implementing QUIC is a different question. I am talking about vanilla UDP socket here

[hissssst]

Oh, I stumbled across some listener being incapable to handle UDP sockets, hmm

[mtrudel]

That's the thing, right? To restate the obvious:

• The TCP/SSL socket concept is a single pair of reliable byte streams between two peers
• The UDP datagram concept is standalone send / receive of discrete datagrams. There's enough meat to the abstraction to layer a simple 'receive/reply' semantic on top of this, but it's not persistent
• QUIC models any number of concurrent reliable byte streams between two peers

Thousand Island provides three fundamental things:

1. An easy to consume, GenServer-like abstraction of a reliable byte stream (with connection, send/receive, and shutdown as fundamental points of interaction)
2. Effective use of OTP processes to provide robust connection acceptance and a 'one process per connection' model
3. Reliable management & supervision of all of the above. If something crashes, we take care of it.

All three of these things make perfect sense for TCP/SSL. All three make some amount of sense for QUIC (point 2 may need to be extended to connections and streams as being handled in separate process, much as Bandit does with HTTP/2 today). But really, only the third point makes any sense for UDP (and it's a pretty anodyne point to begin with).

All of which is to say that I'm thinking less and less that Thousand Island is the right home for UDP. I'm not sure what is the right home is at the moment, and need to tinker adhoc a bit more before I'm going to feel confident that any particular approach is the right one.