Comments (4)
Hm, it could have be interplay of GC and locking issue. I have to test something.
from zstd.
Hi @luben .
zstd streaming interface is supposed to be thread safe.
It only uses information from local context, so as long as you have one context per thread, there is no race-condition to worry about.
from zstd.
Yes, I looked over the code and there is no global state. I suspect that the JVM is not resprecing the buffer pinning and it shows under GC pressure only - it moves them while compression/decompression is in progress. I have to test this and possibly allocate off-heap memory for the buffers or use the buffered version that you provide already.
from zstd.
Yes, it is a problem with JVM not pinning the buffers, I have confirmed. I still have to find a proper solution. Closing it now as it is not a problem of the library.
from zstd.
Related Issues (20)
- is there a pre-trained version for "english" dictionary for web use? HOT 1
- patch-from in single-thread mode performs worse than multithreaded HOT 7
- Consider new release, with #4019 applied
- [reproducibility] Zstd varies compressed output based on the presence of SSE / NEON in some cases HOT 5
- Building libzstd.a using NDK is broken HOT 8
- [Question] Decompression is ~3x slower than compression? HOT 7
- Crash on benchmark using recursive flag HOT 1
- There is a memory leak defect at line 3466 in the file /zstd/lib/compress/zstd_compress.c.
- Newer releases of the zstd-dev package for Alpine Linux do not have Cmake files, while older versions do. This causes build problems for Alpine projects depending on zstd. HOT 2
- Could you please share the policy or algorithm to generate the dictionary? HOT 2
- segfault when a file should be written to a nonexistent directory HOT 1
- ZSTD-22 is 30 times slower than it should be
- [Help Wanted, Questions] Improving Dictionary Training Process
- Questions related to benchmark results in README file
- [Question] Do you use GPLv2 or GPLv2+?
- Build Warning with -Wunused-return on gcc
- ZSTDm (special mode of zstd optimized for memory compression, see paper inside)
- [Question] The Future of the Block API HOT 1
- Allow using dictionaries in benchmark HOT 1
- Please do not remove Block level API
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zstd.