Comments (7)
True, the pickling phase assumes little-endian ByteBuffers
. Since new allocations can happen anywhere in the encoding, the provider must make sure the byte buffers are in correct order. This is a requirement for the provider (although it's not mentioned anywhere in the docs, so need to amend those).
Is there some specific reason why you'd want to provide big-endian BBs to BooPickle?
from boopickle.
I reuse ByteBuffers and have different code paths (w/ or w/o compression) and have to sometimes write outside BooPickle (eg compressed frame length).
With the current behavior, the most sane way is to use LE everywhere, which might be a good thing for performance, but can still be error prone as it's not the default. I sure can live with it.
Still, I don't really get why BooPickle has to assume LE. Could you please explain.
from boopickle.
In the browser typed arrays always use native endianess (which in practice means little-endian), so it's much easier/faster to work with. Although current Scala.js implementation goes through a Dataview which handles the endianess in both cases.
from boopickle.
I got that :)
the provider must make sure the byte buffers are in correct order
Sure, but why not memorize the original byte order, and restore it when pickling is done? And same thing when unpickling.
For example, if you try to send a Boopickle payload over HTTP with Netty, it will complain that byte orders are inconsistent: payload will use l-e while headers will use b-e. The workaround is to change the byte order before feeding Netty.
from boopickle.
Sure, but why not memorize the original byte order, and restore it when pickling is done? And same thing when unpickling.
This is under the ByteBufferProvider
's responsibility so if you make a custom one, you can put the logic there. Likewise after the pickling is completed, you can flip the byte order to BE if needed.
On the unpickling side things are simpler as there is only one BB to work with and BooPickle already manages the byte order correctly.
from boopickle.
Documentation updated to emphasize the importance of correct ordering.
from boopickle.
Thanks. I'll ping if I come up with some elegant solution for my use case.
from boopickle.
Related Issues (20)
- Publish for Scala 2.13 HOT 1
- Publish boopickle for Scala.js 1.0.0 HOT 14
- Maintenance HOT 7
- Release for scala.js 1.0 HOT 3
- Troubles with 1.3.2 release HOT 1
- Release 1.3.3 HOT 5
- Github actions release doesn't work HOT 2
- Support serializing scala 2.13 SeqMap
- Pickle Map with Any as value HOT 1
- Source path for Scala.js is wrong
- Dotty support HOT 1
- Support Pickler derivation with nested sealed traits for scala 3
- Scala 3 enums cause error at runtime HOT 1
- Scala 3 pickle format is not compatible with Scala 2 format HOT 5
- Is binary format сompatible with MessagePack? HOT 4
- Documentation / Mitigation regarding untrusted input HOT 2
- Split Pickler into reader and writer, for small Scala.JS bundle sizes
- Unable to access pickled bytes
- Unknown integer coding error HOT 5
- Could not find implicit value for nested parameter of same case class type (recursive reference)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from boopickle.