Boopickle still not playing nicely with BIG_ENDIAN about boopickle

[ochrons]

True, the pickling phase assumes little-endian ByteBuffers. Since new allocations can happen anywhere in the encoding, the provider must make sure the byte buffers are in correct order. This is a requirement for the provider (although it's not mentioned anywhere in the docs, so need to amend those).

Is there some specific reason why you'd want to provide big-endian BBs to BooPickle?

[slandelle]

I reuse ByteBuffers and have different code paths (w/ or w/o compression) and have to sometimes write outside BooPickle (eg compressed frame length).

With the current behavior, the most sane way is to use LE everywhere, which might be a good thing for performance, but can still be error prone as it's not the default. I sure can live with it.

Still, I don't really get why BooPickle has to assume LE. Could you please explain.

[ochrons]

In the browser typed arrays always use native endianess (which in practice means little-endian), so it's much easier/faster to work with. Although current Scala.js implementation goes through a Dataview which handles the endianess in both cases.

[slandelle]

I got that :)

the provider must make sure the byte buffers are in correct order

Sure, but why not memorize the original byte order, and restore it when pickling is done? And same thing when unpickling.

For example, if you try to send a Boopickle payload over HTTP with Netty, it will complain that byte orders are inconsistent: payload will use l-e while headers will use b-e. The workaround is to change the byte order before feeding Netty.

[ochrons]

Sure, but why not memorize the original byte order, and restore it when pickling is done? And same thing when unpickling.

This is under the ByteBufferProvider's responsibility so if you make a custom one, you can put the logic there. Likewise after the pickling is completed, you can flip the byte order to BE if needed.

On the unpickling side things are simpler as there is only one BB to work with and BooPickle already manages the byte order correctly.

[ochrons]

Documentation updated to emphasize the importance of correct ordering.

[slandelle]

Thanks. I'll ping if I come up with some elegant solution for my use case.