szeweq / mc-repack Goto Github PK

I have a suggestion which would be useful for me and perhaps for other creators:

• Create a GitHub Action.

I am currently having to use the following combination:

- name: Install tools
  uses: baptiste0928/[email protected]
  with:
    crate: mc-repack
- name: Clean files
  run: mc-repack "/home/runner/work/[repository-name]/[repository-name]/build/libs/" --out "/home/runner/work/[repository-name]/[repository-name]/build/libs/clean/"

Note:

1. I have to include full path.
2. I can't use a environment variable.

If there is anything I can do to help, let me know.

Hi,
just found your project and this looks rather interesting.

Here's a couple of improvements/changes that would be amazing if they could be added:

• Explode nested jars and recursively repack them. however, in those nested jars only ever STORE files. then add DEFLATE compression at the final level. Separate issue: #3
• Sort files by their extension. When files of the same type are beside each other in a nested jar file, it will be able to be better compressed. (based on some rudementary testing, this combined with the previous item, can drastically improve the compression at times) Separate issue: #3
• Ability to specify oxipng level
• Recompress nbt files (they are just gzip compressed, so uncompress then attempt to recompress at level 9), then STORE them. Separate issue: #4
• Don't include zip entries for the directories. fun fact: you don't need zip entries for directories!

Based on a quick little command I ran, I recorded the frequencies of different file types present in jar files.
The command used was

mkdir ./tmp
# copy a bunch of jars into this temporary directory
for i in *.jar; do
    unzd "$i"
done

find . -type f \( -iname '*.jar' -o -iname '*.zip' \) -print0 \
    | xargs -0 -n1 unzip -qqql \
    | perl -0777 -C -pe 's/.*?\/?(.*\.(.*))?/\2/g' \
    | sed '/^$/d' \
    | sort \
    | uniq -c \
    | sort -n \
    | awk '{s+=$1; print $0} END {print s}'
# note: the last line is the total count of all files. this isn't by any means perfect, but whatever.

Here are the results from that:

Based on this, I think it might be reasonable to consider adding optimization processes for the following files:

• .ogg files. These can possibly be optimized using ffmpeg. (you could possibly consider decreasing the quality to increase compression. a cli arg for this would be nice.) (there seem to be a couple of crates that provide ffmpeg support, so those could be considered)
• .nbt files. They are just gzip compressed binary data, so uncompress then recompress with level 9, then STORE them.
• .properties files. Blank lines can be stripped from these. (it doesn't make up a large amount of the files, but it's an easy addition)
• .accesswidener files. Blank lines can be stripped from these, as well as any lines starting with a hash (#) are comments. (it doesn't make up a large amount of the files, but it's an easy addition)

It's probably not worth it to consider additional compressors for files not in that table as they appear so infrequently it just won't make much of a difference.

Do note however, that this is not based off of the size of the files in the jar, rather just their count. may make something basic to calculate this using the size later.

NBT files (from Notch's design) have its data compressed using Gzip. MC-Repack should have a special minifier for checking and recompressing this file format. (Suggestion from #2)

It should use either gzip or zopfli.

MC-Repack should have a minifier for handling JAR files stored inside another JAR file. (Suggestion from #2)

• This implementation must not break the JAR files stored in a file system,
• This minifier should be user-configured by CLI options,
• Add an option to make jar-in-jar file entries be stored or deflated and jar-in-jar itself be compressed.