Consider moving images into a submodule about lepton

[danielrh]

It's an attractive suggestion... normally I shy from submodules because many git clients don't grab them by default--and I want the thing to build out of the box...
but I could probably add something to make check that warns the user that the files are elsewhere...
unfortunately it has already polluted the .git directory-- so it doesn't really help for a first time git clone

[zmwangx]

it doesn't really help for a first time git clone

Actually it also helps with shallow clones, but pushing from a shallow clone could sometimes be hairy, and presumably most people either don't know or don't bother, so the value in this regard is indeed limited.

[UnitedMarsupials-zz]

And, likewise, please, make download of dependencies/ optional as well -- as discussed in #18 .

Thanks!

[danielrh]

The problem with images as a submodule is that it doubles the checkout time of the repo, since it has to clone both the main repo with the images and the sub repo with the images. Is git smart about sharing objects it has downloaded across submodules?

[zmwangx]

That's indeed a big problem...

Is git smart about sharing objects it has downloaded across submodules?

I'm happy to be proven wrong, but I'm afraid not.

EDIT: I just tested and indeed, the data would be cloned twice despite being the same files.

Maybe we have to discard this proposal.

An alternative strategy is to leave the repo as is, but for each release upload a tarball without the images directory (in addition to the auto-generated source code tarball). Then fetch the full archive only when make check is run. Basically, say we have the project root path in PROJECT_SOURCE_DIR and git revision in VERSION, then this can be part of make check:

#!/bin/sh
[ -d "${PROJECT_SOURCE_DIR}/images" ] && exit
download_url="https://github.com/dropbox/lepton/archive/${VERSION}.tar.gz"
tarball_path="$(mktemp /tmp/lepton.tar.gz.XXXXXXX)"
[ -n "$tarball_path" ] || exit 1
trap 'rm -f "$tarball_path"' EXIT
curl -Lo "$tarball_path" "$download_url" || exit 1
tar xf "$tarball_path" -C "$PROJECT_SOURCE_DIR" --strip-components 1 "lepton-${VERSION}/images" || exit 1

By the way, this would require the git revision written somewhere in release tarballs. IMHO that's a problem worth solving in its own right — currently lepton -revision is empty when built from the release tarball.

[danielrh]

That sounds complicated...what if your build machines are behind firewalls, etc...
I think I'm going to just close this for now..
it's too bad we didn't think of this when making the test suite--but we had no idea, at the time, that this would go anywhere--- it was still making JPEGs 10x bigger when the test suite was completed

[zmwangx]

No problem, that's understandable.