Comments (2)
I closed #68 as this provides more adequate API with a lot more predictable memory profile.
from js-car.
Quoting relevant comment from #68
It wouldn't really even need a byteLength argument because it's going to be holding on to a list of Uint8Arrays anyway and it just needs to create the header when it gets the roots and spit that out first. Nothing about the bytes that it buffers is impacted by the length of the header. You just get a one-shot addRoots() call to open the flood gates.
This is interesting point! In our use cases we generally know how many roots we'll have ahead of time even though we do not know what they're going to be. So it's fairly easy to tell how much memory needs to be allocated to fit the roots. And doing so would allow writing into buffer without having to hold onto blocks freeing memory sooner.
However if number of roots is unknown ahead of writes not having to allocate space is definitely going to be a better option. Given known use cases, I would still suggest going with proposed API one with unknown roots could easily be build as layer on top when/if desired.
from js-car.
Related Issues (17)
- Can't resolve import when using @ipld/car from jest HOT 2
- Premature exit of reader when consuming an iterable stream with empty blocks HOT 2
- CarWriter.fromIterable(roots, blockIterator) HOT 1
- Propsoal: New API to create a writer with unknown root HOT 2
- Proposal: make CarReader API sync HOT 5
- reading from a file object HOT 6
- CarReader.fromIterable for large files HOT 1
- Add support for WebStreams HOT 6
- Option to validate CID matches block HOT 4
- `CarBufferWriter` should be exported by index-browser HOT 3
- No "exports" main defined HOT 4
- What is the proper way to get CAR content without the `carHeader` HOT 2
- Very strange issue with `writer.close` (silently closes entire node process) HOT 7
- dag-json support for CarReader.fromBytes HOT 1
- tsize is too large? HOT 1
- Export a version from the writer HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from js-car.