Comments (6)
Hi there!
I was recently looking at ReadOnlySequence<byte>
for 64 bit support scenarios. It gets a little awkward since data can be spread across a Span<byte>
boundary. I didn't make it as far as a full prototype (since I was investigating this as part of 64 bit support instead of on its own merits), but it does sound interesting. I suspect that performance will not be great since you'll need quite a bit of logic for dealing with Span
boundaries, but I don't see why it wouldn't be possible. The only awkward thing might be when FlatSharp wants to return a ReadOnlyMemory<byte>
for a [ ubyte ]
vector. That would likely require a copy, which would be unexpected for Lazy
/Progressive
vectors. The other option is to throw an exception, which I think is my preference in the interest of explicitness.
from flatsharp.
If you're interested in taking a shot at this, I'll be happy to do the PR. Otherwise, I might try it in the next week or so.
from flatsharp.
I've taken a look at this. Unfortunately, it will require an interface change and some changes to the codegen. The reason is that FlatSharp sometimes asks the IInputBuffer
for the entire Span<byte>
, which of course ReadOnlySequence<byte>
can't provide. We'd need to add a .GetSpan(int offset, int length)
API to IInputBuffer
, which would constitute a breaking change.
It might be a bit more than you're comfortable biting off, however.
from flatsharp.
Hi again -- quick update. I've prototyped this, and the performance isn't good. The issue boils down to reading data out of the ReadOnlySequence<byte>
. This structure isn't ideal for random access. My implementation relies on BufferExtensions.CopyTo
for the heavy lifting:
// IInputBuffer.ReadInt
public int ReadInt(int offset)
{
ReadOnlySequence<byte> seq = this.sequence;
// Reasonable overhead
Span<byte> temp = stackalloc byte[4];
// This method call results in a 257 byte method as emitted by the JIT. This won't be fast.
seq.Slice(offset, sizeof(int)).CopyTo(temp);
// Fast
return BinaryPrimitives.ReadInt32LittleEndian(temp);
}
You can see this here
ReadOnlySequence<byte>
is really intended to be read left-to-right, not randomly. However, reading a FlatBuffer results in non-sequential jumps throughout the buffer, both forward and back. While I can build an IInputBuffer
based on ReadOnlySequence
, I'm leaning against including this in the library because I want the API to steer people to do the right thing.
In this case, it will almost certainly be faster to copy data from the sequence into a pooled array, and then hand that array to FlatSharp.
Do you have any thoughts here?
from flatsharp.
I've been continuing to tinker with ReadOnlySequence<byte>
(outside the context of FlatSharp) and came to the same conclusion. Your suggested solution is fast and idiomatic.
I do really appreciate you taking the time to investigate this. Also, thank you for this awesome library. 😊
from flatsharp.
Also, thank you for this awesome library
Hey, thanks for the kind words!
from flatsharp.
Related Issues (20)
- Compile fails to add 'required' to net7.0 generated code HOT 5
- Duplicate Clone Functions v 7.0.2 HOT 4
- Serializer.Parse stream API? HOT 4
- Some questions HOT 2
- Generated code uses nullable annotations HOT 2
- Arithmetic operation resulted in an overflow HOT 2
- Pipeline-Build-Error at executing the flatsharp compiler HOT 3
- Generated defaults fail to handle enums with negative int & large ulong HOT 2
- FlatSharp Support for 64 Bit FlatBuffers
- Codestripping with IL2CPP HOT 7
- Compilation target fails when executable path (i.e. nuget cache path) contains spaces
- Namespace not found HOT 1
- Share table HOT 2
- [Bug] Using the latest "Standalone" `FlatSharp` nuget in a .net6 project; generates a file that requires `IPoolableObject` HOT 2
- Reducing compiler generated file HOT 15
- Duplicate fbs filenames in different folders break compilation HOT 2
- Getting TypeLoadException when trying to access FlatSharp types HOT 9
- Shared fbs between multiple projects HOT 11
- [Bug] Flatsharp can't compile schemas on seperate drives HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flatsharp.