Comments (14)
I have made a basic working version with derive macros 😄 .
It's not ready, just some PoC.
#150
from borsh-rs.
Great, Probably i can work on it in the next week.
Thank you!
from borsh-rs.
@iho It is a nice-to-have, and we currently don't have an urgency for it, so let's wait for @hamidrezakp to experiment with it.
Sure, i will continue writing the whole thing and see if it works and benchmark it.
@iho if you have any idea to make my code better, i highly appreciate it.
My code is in this repo.
Just for context, there is an issue in serde where reader methods are much slower than
from_slice
which is another consideration point when we work on more features in that direction.
Good point, that's why we always support both reading from a Reader
and Slice
or writing to Writer
and Vec
.
This way we have the benefit of both of them.
from borsh-rs.
for reference, I'm following this method that Jon mentioned here
from borsh-rs.
@hamidrezakp I assigned this issue to you for now as I am on-boarding new contributors and I don't want them to work on this issue now. I don't expect you to deliver the PR, I just want to indicate that it is "taken" for now (let's say, 1 month). If someone in the future (1+ months from now) finds this issue, and wants to address it, feel free to reignite the conversation.
from borsh-rs.
@hamidrezakp There is nothing to make async, the whole process is CPU-bound, and fast. What is your use case?
from borsh-rs.
I meant we support reading from polllable sources like TCP/IP or sockets.
Right now we have support for Read
types which can not be used with Tokio TCP Streams
for example.
from borsh-rs.
I see, no there are no plans and no capacity to implement the support, but I am ready to review a PR if anyone will contribute the support with tests and documentation
from borsh-rs.
@hamidrezakp hi, are you going to finish it?
from borsh-rs.
Hey,
Sorry for not being active here.
I have tested the idea and it seems we need to change the BorshSerialize
and BorshDeserialize
trait and add some new bounds like Send + Sync
to the reader and writer the same.
This change is breaking and it makes a bad user experience, because with this approach we would have two set of traits, blocking and non-blocking, which is quite bad and no one is gonna use it.
Unless you or somebody else have a better workaround or idea to add async support, i think i may create a crate like tokio-serde
for borsh.
from borsh-rs.
Why do we need a new trait bound? Can you share your code, please? I am not very familiar with async Rust.
I guess we could just add an async crate feature and expose a new set of async functions something like from_reader_async
, and to_writer_async
.
@frol what do you think about this?
from borsh-rs.
Here is the code i came up for serialization:
#[async_trait::async_trait]
pub trait BorshSerialize {
async fn serialize<W: AsyncWrite + Send + Unpin>(&self, writer: &mut W) -> Result<()>;
/// Serialize this instance into a vector of bytes.
async fn try_to_vec(&self) -> Result<Vec<u8>> {
let mut result = Vec::with_capacity(DEFAULT_SERIALIZER_CAPACITY);
self.serialize(&mut result).await?;
Ok(result)
}
#[inline]
#[doc(hidden)]
fn u8_slice(slice: &[Self]) -> Option<&[u8]>
where
Self: Sized,
{
let _ = slice;
None
}
}
#[async_trait::async_trait]
impl BorshSerialize for u8 {
#[inline]
async fn serialize<W: AsyncWrite + Send + Unpin>(&self, writer: &mut W) -> Result<()> {
writer.write_all(core::slice::from_ref(self)).await
}
#[inline]
fn u8_slice(slice: &[Self]) -> Option<&[u8]> {
Some(slice)
}
}
from borsh-rs.
Send
and Unpin
is needed because in some types like Vec<T>
we may not be able to serialize the whole thing in one go and have to store Future
which contains the writer for later pooling.
from borsh-rs.
I was also thinking about introducing a separate trait for async methods and keep it under a feature-flag. This way it is not going to be a breaking change.
@iho It is a nice-to-have, and we currently don't have an urgency for it, so let's wait for @hamidrezakp to experiment with it. Just for context, there is an issue in serde where reader methods are much slower than from_slice
which is another consideration point when we work on more features in that direction.
from borsh-rs.
Related Issues (20)
- Security Policy violation SECURITY.md HOT 2
- Restriction on Serializing Zero-Sized Types Affects Marker Component Usage HOT 7
- derive BoshSerialize fails if the type already uses `W` generic name HOT 4
- BorshSerialize derive fails for structs with packed attribute HOT 4
- Should `std` feature imply `rc` feature and vice-versa? HOT 4
- Read/Write mutable reference in `serialize` and `deserialize_reader` is unnecessary HOT 8
- `derive` macros require `cargo` to be present HOT 4
- Old NEAR contracts won't compile with the new `borsh` re-exported from `near-sdk-rs` ("Could not find `borsh`") HOT 5
- equivalent of `#[serde(default)]` HOT 2
- Borsh 2.0? Not planned HOT 2
- Extract `alloc` feature (2.0 candidate) HOT 1
- `bytes::Bytes` and `bytes::BytesMut` can be complemented with `BorshSchema`
- `bson::oid::ObjectId` can be complemented with `BorshSchema`
- Add struct/enum attr to relax bounds
- feat(schema): for chars HOT 4
- feat(schema): support serde of HashMap with S hasher
- Support for adding new fields to previously serialized structs HOT 2
- Can't serialize a `Vec<ZST>`, but it appears that the attack is only on deserialization HOT 1
- blanket implementation of `BorshSchema` for `&T` (where `T: BorshSchema`)
- Replace `schema_map!` tests with `insta` snapshots
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from borsh-rs.