olle / lz77-kit

Some LZ77 compression utilities for the handy hands-on developer in search for a few less bytes of I/O.

LZ77 Kit

The lz77-kit is aimed at being another tool in the web-persons tool belt, enabling a simple and very pragmatic compression utility for various languages. It's perhaps not the sharpest compression-knife for heavy-duty cutting, but may suite some craft-peoples needs.

This could be a simple way for you to throttle down on some of that bandwidth usage, at least a few bytes or so. And a byte saved is a byte earned!

Currently features LZ77 implementations for the following languages:

• JavaScript
• PHP
• Python
• Java
• Ruby
• Erlang (ahem... maybe not quite there yet)

Language wish-list

• Go
• OCaml
• Haskell
• Clojure
• Swift
• ObjectiveC
• Rust
• C

Are you missing a language? Please get involved and contribute!

Getting started

1. Get the project source code
2. Go to the project directory
3. Test and build using Ant on the command line.

To build and test all the current LZ77 implementation use the default build target:

shell$> ant

For more build information and project build targets, you may run:

shell$> ant -p

Look in the ./output folder for the built artifacts and copy/paste what you need. Please keep the license and credits in any generated source code or scripts.

Thank you!

A note on requirements

Being a project with such a broad programming language base, the requirements are more or less dependent on your target build platform. It's of course only possible to do a complete build if all programming languages are installed on that system.

I've lately had success building the project using the following:

• Ant 1.9.4
• PHP 5.4.30
• Java 1.8.0
• Python 2.7.5
• Ruby 2.0.0p481
• Erlang/OTP 17

Oh, and by the way, my shell have slashes that slant "the right way" (/).

Contributing

The most source code have unit tests and the project is pretty independent, or free from third party dependencies, so it should be easy for any developer, with a decent environment, to clone and and start working with the code.

Guidelines

• Try to keep the self-dependent structure, add libs and dependencies into the project but be very sparse.

• Write tests and wire them to work with the Ant build.xml buildfile.

• Keep implementations simple and open to copy/paste (e.g. no packages or namespaces if possible).

• Code with joy, not in anger!

This is all done with the hope of being of use to someone out there.

lz77-kit's Issues

Settings parameter is not documented

You can define some input parameters for algorithm (at least for javascript implementation https://github.com/olle/lz77-kit/blob/master/src/main/js/lz77.js#L37) but they are not documented

1. referenceIntBase
2. minStringLength
3. defaultWindowLength

How changing this will influence the workflow.

1. As I see referenceIntBase. which is 96, defines the last used character above 'space' (with code 32), so last character will be 127. which is ASCII 'DEL'. Is it ok to change value from 96 to 95 to allow only printable characters in encoded string? Can it have any side effects? Also javascript implementation produces UTF-16 characters(with String.fromCharCode), but while you are using this 96 value, you are in ASCII range. Can it be set to larger value to produce real UTF-16 string? Any use of it?
2. How changing minStringLength will change algo behaviour?
3. As I understand it can be changed but may be too large. How changing it will change algo behaviour?

Main method not found

Hi,
I get this error when I try to run it

Error: Main method not found in class LZ77, please define the main method as:
public static void main(String[] args)
or a JavaFX application class must extend javafx.application.Application

More programming language examples

Girls and guys!

I known many of you are referencing this repo as a source for compression assignments in CS/EDU, which is fine by me. But the example implementations are sort of not very up-to-date with regards to programming languages.

It would be great to have implementations in: Go, Kotlin, Clojure, C, C++, Swift, R or any other more recent and modern langauge.

Come on and contribute!

decompressing error

Hi, I was doing some tests, and I encountered an error compressing and decompressing 64 kb of text with the maximum window size of 9220
I did this test:

String msg = new String(Files.readAllBytes(Paths.get("C:\\path\\to\\file", "test_data.txt"))); // input
String x = LZ77.compressStr(msg);
String y = LZ77.decompressStr(x);
System.out.println(x.equalsIgnoreCase(y));

I get

Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -10
    at java.lang.String.substring(String.java:1960)
    at lz77.LZ77.decompress(LZ77.java:216)
    at lz77.LZ77.decompressStr(LZ77.java:82)
    at Main.main(Main.java:29)

Which is the same error as the one in the c# impl.
Data is just a random text, here's the link: http://pastebin.com/fywqdyi3

I also tried to compress the same data with the max window length (9220) using this online tool: http://www.geocities.ws/diogok_br/lz77/demo.html. The compression works, but decompressing loses data. Example using the online tool and the same 64kb input data: http://pastebin.com/7VYcT7t6
As you can see in the decompressed text there are some scattered weird chars, and the size is 65125 bytes instead of the original 65536 chars of the input.

As I don't know the algorithm, could you fix this?

PS: that online tool uses your js version, here's their source code: http://www.geocities.ws/diogok_br/lz77/lz77.js
so the problem is for all versions

A Go-lang version, people!

Dig in, and make a pull-request. This project needs a go-lang version.