Comments (9)
@kontura I know you didn't want to go quite this far but I thought there was going to be some functionality deprecated with warnings?
from createrepo_c.
Unfortunately I didn't find the time to do any additional work on createrepo_c and we needed to get the other changes released at least a bit in advance in case there are some problems.
Therefore this was postponed.
from createrepo_c.
Chiming in the discussion, as docs are not really clear (at least, for me): am I right to understand that --xz
option is just a shorthand for ----general-compress-type=xz
? Or is it a shorthand for --compress-type
?
from createrepo_c.
It is a shorthand for just --compress-type=xz
(so it doesn't affect "primary", "filelists", and "other" xml metadata).
When we will be unifying the compression options we should also remove the --xz
option.
from createrepo_c.
I guess --general-compress-type
and --compress-type
was motivated by a microoptimization when some (group) files are too small to be effectively compressed. I.e. a compressed file is larger than an uncompressed one. Or their decompression takes disproportional time. Or it was simply a hack when a package manager did not understood a compression at all files.
It's basically a hack that we cannot specify exactly what files should be compressed and how should be compress. I'm not actually sure it's important to implement such feature. I would simply merge the two options into one and instead allow placing output files in multiple compression formats, including no compression.
Would you like this interface?:
--compress-type COMPRESSION_TYPE
Which compression type to use. Supported compressions are: none, bz2, gz, zck, zstd, xz.
"none" means no compression. You can use this option multiple times to output in multiple compress types.
If this option is not used, "--compress-type none" is implied".
from createrepo_c.
I guess --general-compress-type and --compress-type was motivated by a microoptimization when some (group) files are too small to be effectively compressed. I.e. a compressed file is larger than an uncompressed one. Or their decompression takes disproportional time. Or it was simply a hack when a package manager did not understood a compression at all files.
Probably the last one. comps.xml is so tiny compared to the others that compression is basically irrelevant.
The API is fine but I disagree strongly about "none" being a default compression type. Without compression, those files are massive. I have a copy of Fedora 38 "release" metadata on my disk, without compression filelists.xml is 747 megabytes, with compression it's 42 megabytes (zstd-compressed, gzip is a bit bigger but not that much bigger). other.xml is 110mb vs 5.5mb, primary.xml is 167mb vs 15mb
zstd is a good default. It has good compression ratios, it's fast to compress and decompress, and by this point it is widely supported on {Open}SUSE and Fedora / RH based distros (at least, I haven't checked the others. But anything that uses libsolv either has it enabled already or can enable it with a simple flag).
from createrepo_c.
That's your use case. My use case is many small repositories. A good default is very subjective. Do you think zstd will be the best in 10 years? Is zstd supported on RHEL 7? A good default does not last in time. No compression has the advantage that it avoids a risk of breaking a compatibility. Or we can make the default build-time configurable, so it's not our = upstream problem anymore.
from createrepo_c.
No compression has the advantage that it avoids a risk of breaking a compatibility.
"none" is not currently even an option in createrepo_c
. Neither does legacy createrepo
seem to have provided that option. (adding metadata manually with modifyrepo_c
will allow it, but that's different, and still not the default).
The maximally compatible choice would be to use gzip (the default prior to 1.0) forevermore, which I would still vastly prefer over defaulting to no compression at all.
I do agree that a "none" choice should exist, but it should not be the default. It has no compatibility advantages whatsoever.
Do you think zstd will be the best in 10 years? Is zstd supported on RHEL 7? A good default does not last in time.
Does it really matter if it's "the best" in 10 years? It's meaningfully better across relevant metrics and widely supported by both Red Hat and SUSE derived distributions for several years now. Support was added to libsolv
in 2018 and is inherited by both dnf
and zypper
.
But no, RHEL 7 and yum
generally don't have support, that is true. That leads to a bit of frustration for the limited number of people / projects who are managing repos for the oldest distributions using the very newest versions of Fedora, but it's not a common case and will be only a short-term problem.
from createrepo_c.
Submitted #411 for --compatibility
defaulting to gzip
from createrepo_c.
Related Issues (20)
- [FIX] Build from 'master' branch is broken HOT 3
- `_XOPEN_SOURCE` define in `src/misc.c` seems extraneous
- Drop `--database` and `--no-database`, split?/drop `sqliterepo_c` HOT 5
- Sending SIGTERM to "createrepo_c --workers 2" sometimes leads to a crash HOT 6
- `--pkglist` can't be used with non-regular files
- Parsing primary.xml error: Start tag expected, '<' not found HOT 4
- heap buffer overflow and stack buffer overflow in test suite HOT 3
- Intermittent crash in `ci-dnf-stack/dnf-behave-tests/createrepo_c/zchunk.feature` HOT 2
- Python bindings fail to add the default version for sqlite records
- Has `--deltas` option been removed? HOT 9
- Brainstorm ways to shrink RPM metadata HOT 5
- Fix the building process to drop documentation for disabled features
- Newer createrepo_c doesn't generate comps readable EL7 HOT 27
- sqlite3_enable_shared_cache HOT 1
- `modifyrepo_c` and `mergerepo_c` generate `--no-pretty` metadata by default
- createrepo_c zstd compression doesn't fill in the content size, in the frame header. Python API problems. HOT 4
- Allow parsing packages metadata without filelists HOT 2
- cr_xml_dump_int() should point to a forbidden character HOT 2
- how does src rpm by pass sub packages in conditons HOT 2
- Removing older versions from the repo. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from createrepo_c.