Comments (8)
Bonjour Christophe
Is the default level still intended to be level 1?
Yes.
I have (unfulfilled) long term plans for even faster modes,
that's what negative levels will be used for.
or is the whole zstd vs zstd_hc a thing of the past?
Well, I realized that having 2 separate functions, with zstd meaning the same as zstd_hc with a fixed level 1, did not helped API readability.
So decided to merge everything into a single ZSTD_compress()
What about level 0? The comment in zstd_static.h says that level 0 is "never used", but quick tests show that using compression level 0 works fine (and compress about the same as level 1)
All levels <= 0 are simply remapped to 1 internally. for now.
Are there plans to have standardized names for some compression levels, like "fast", "high", "ultra" and so on?
Nope.
Could change in the future if there is reasons to. But every wrapper should be able to select whichever value it wants.
For example, folly
selected fast : 1
, default : 1
, best : 19
Right now max level is 20 but it seems that it can change. If yes, do you plan to have some way to, at runtime, probe for the range of supported compression levels?
Yes.
The value ZSTD_MAX_CLEVEL
within zstd_static.h
is meant for this usage.
from zstd.
For example, folly selected fast : 1 , default : 1 , best : 19
Do you mean this? https://github.com/facebook/folly/blob/master/folly/io/Compression.h#L152
The idea of having values -1
, -2
, -3
remapped to whatever the min/default/max points in the range supported by the loaded library seems nice, but if you intend to indroduce negative levels, this would need be moved to some other values...
The value ZSTD_MAX_CLEVEL within zstd_static.h is meant for this usage.
It is currently a #define
which is not visible when you are consuming a dll
(at least from a managed language such as .NET). Or maybe there is a way but I'm not familiar with it. Currently, I have a constant in my .NET code which I synchronize with the latest version of the .h
when I rebuild the dll internally, but this is brittle. I don't have this issue with ZSTD_VERSION_...
because there is a ZSTD_versionNumber()
method.
from zstd.
It is currently a
#define
which is not visible when you are consuming adll
Good point.
from zstd.
Within latest "dev" branch update d608088, there is a new method ZSTD_maxCLevel()
to retrieve that value.
from zstd.
I have changed my code to query for ZSTD_maxCLevel()
on start, and I'm currently using an enum with 4 levels:
Default
= 0, but is converted to 1 internally before calling zstd.Fastest
= 1 (but could change?)Medium
= (1 + MAX) / 2Highest
= MAX
This gives Medium = 10 and Highest = 20 in the current version.
Having 0 for the default level helps in that it is also the default value for uninitialized variables and optional parameters in .NET, which works well in the API. This could work if you NEVER use 0 as a compression level (even if you start using negative values). If not then I could simply use a nullable integer in the API, which would also be acceptable.
If the minimum compression level can be less than 1 in the future, we would need a ZSTD_minCLevel()
method to query the value. Or maybe a single method that returns the bounds?
from zstd.
Having 0 for the default level helps
Agreed. This is the current convention, and is expected to remain like this. LZ4
uses the same convention.
If the minimum compression level can be less than 1 in the future, we would need a ZSTD_minCLevel() method to query the value.
Agreed, although this is just a "long term intention", with no precise plan.
Even if I introduce negative compression levels later on (which is not yet guaranteed), this won't change anything for "normal" existing compression levels, hence interface break.
So we'll have time to handle this issue if it ever becomes a reality.
from zstd.
Is there anything else to add, or could we close the topic ?
from zstd.
I think it's OK to close.
To sum up:
- I'm using 0 to mean "the default" which currently is 1
- I'm calling ZSTD_maxCLevel to get the max supported by the version of dll loaded at runtime.
- I provide a set of Fastest/Medium/High/Highest helpers that can be used optionally, but with guidance in the comments that app code should rely on testing for the best level and use the numerical value instead... or use default if they don't really care.
- I'm probably going to validate levels at the wrapper level, and throw if they are above this, instead of capping it silently at the MAX, because it's more inline with what a .NET developer would expect (most API validate enum-like parameters and throw).
- In the future if compression level can get negative, then maybe a ZSTD_minCLevel could be added to get the lower bound (but sill keep 0 as the default).
from zstd.
Related Issues (20)
- Provide Linux & Darwin (macOS) builds via GitHub Releases
- Disable auto vectorization of xxhash64, when AVX512 is present. HOT 5
- No check if Reserved of Symbol_Compression_Modes is 0 HOT 8
- Spec cleanup: Should fixup behavior when repeat1-1==0 be specified or changed to an error? HOT 3
- Strange tags make automation crazy HOT 1
- Modernize macros to use `do { } while (0)` instead of `{ }` HOT 9
- [question] Seek for insights on the suitable case for zstd dictionary compression HOT 5
- zstd not buildable with PAC/BTI becauseof `huf_decompress_amd64.S` HOT 4
- get a core dump on use ZSTD_compressCCtx (Unhandled exception access violation) HOT 3
- Adding a library to https://facebook.github.io/zstd/#other-languages HOT 1
- Allow files bigger than 2GiB for --patch-from option HOT 1
- Allow files bigger than 2GiB for --patch-from option HOT 2
- [Question] How to force single literal streams during compression HOT 1
- Export "selected" CMake target for zstd HOT 5
- [Question] Understanding of compression level with external sequence producer HOT 2
- zstd fails to process some filenames on Windows [we need a hero] HOT 3
- We need a ZStd JavaScript library HOT 1
- Compiler warnings present when integrated with Swift Package Manager
- will zstd get nvcomp acceleration or a gpu support like g-brotli? HOT 1
- How can I change the window size? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zstd.