Binary Model? about superpower

[BenjaminHolland]

https://github.com/BenjaminHolland/superpower/tree/feature/binary

Initial, minimal implementation.
I built it with Memory, though I'm not sure that was the right idea. It doesn't build yet due to the XML comment errors and the non-existant Core 2.1 library.

One idea here is actually to interoperate with the new pipelines library. That library handles the streaming and the asynchronous stuff, and exposes the data stream in smallish chunks that need to be polled, parsed, and dispatched separately, so the first point is really an issue.

It could also be used for image processing or other smallish data formats where you need to parse header, content, etc. Building a parsing language for an image, and having that language be able to say "Hey, this header is whack, yo". would, to me, seem pretty useful.

Speed is the one thing that I think is an issue. You're already trading off speed for expressiveness by using a combinator library in the first place.

[nblumhardt]

Hi! It's not at all an unreasonable idea; I think Pidgin (C#) and nom (Rust) are two examples I can think of that allow this.

The main considerations against adding it in Superpower are:

1. Binary parsers generally need to operate on streams, since files/network formats often need to deal with gigabytes of data at a time; Superpower is designed for small language sources where loading a string up-front is an acceptable trade-off for the simplicity it brings

2. The main benefit Superpower brings over other options is that for end-user-facing parsers it provides good error reporting; binary format parsers don't generally need to report errors back to users (I don't want to know what bytes were found/expected in a broken JPEG file ;-)) so its benefits are reduced, and there are probably faster options due to the reduction in error reporting machinery

Neither's an insurmountable barrier to considering this, but I think we'd need a really compelling use case to justify the addition.

Forking and stripping the library back to better suit these goals does sound like a fun project, though :-)

[nblumhardt]

That's awesome :-) !!!

[nblumhardt]

Closing this as I don't think it's a feature that will make it into Superpower, but I hope it's going well, @BenjaminHolland - keep us posted!