Make it possible to skip a sprite if no files have changed about glue

[jorgebastida]

This sounds like a great idea. The problem is... Where should glue store this tmp/cache/hash of files?

Anyway, (afaik) after 0.2.5 glue output should be deterministic - same input, same output, so if your source images don't change you should not get different outputs sprites/css.

Is this the case? What version of glue are you using?

[dominicbarnes]

I'm on 0.2.4, which is what is available via easy_install.

Perhaps adding the hash as an option appended to a sprite.conf file in the sprite directory would work? Or perhaps jush a hidden .glue.hash in the directory?

[dominicbarnes]

Just upgraded to 0.2.5 via pip.

[jorgebastida]

Hello @dominicbarnes
The last version available on PyPI is 0.2.5 http://pypi.python.org/pypi/glue/

About the file yes, but how big/many sprites are you trying to build to make it a slow process?

[dominicbarnes]

The project I'm working on currently has 24 sprites, several of which contain many small icons.

Yeah, it's a bit excessive in terms of how many sprites, but it's largely legacy code that's being migrated through a serious overhaul anyways. As of now, it takes glue over 20 seconds to run, and was just hoping to cut that down.

Being deterministic definitely helps, as the rest of the build won't be tainted by unchanged sprites, will make sure to have my team members upgrade glue asap.

[jorgebastida]

After glue 0.2.6 (Planned for this week if nothing goes wrong) the next logic step is make glue faster.

Until this version I've been completely focused on developing new features, but as far as the current version is fairly complete (in terms of features) I'll try now to make it faster as well as more efficient ( in terms of image packaging using the square algorithm).

I'll love to ear any other suggestion you have about how to make it run quick :D

[dominicbarnes]

Ok, so I take it this particular feature won't be added?

[jorgebastida]

No, no - I'll keep this open and ponder what's the best approach :)

[jvsummeren]

I think this is a great idea, since I'm always optimizing my sprites with ImageOptim and ImageAlpha. Those results are way better that Glue can achieve. Which is fine by me, cause the main business of Glue should be generating sprites, not optimizing them to the max.

I only optimize my sprites, which takes some work, when I'm at the end of a project, so it would be could if we can figure something out so those optimized sprites do not get overwritten either.

[jorgebastida]

Hi guys,

Probably this branch feature/metadata would be of your interest :)

After thinking about how to solve this problem... I got an idea. What about EXIF instead of creating more files...

Basically the idea is to store a hash of the generated sprite inside the sprite's metadata and on the next run (before overriding the file) check if the new sprite's hash is going to be the same one. It's quite tricky... because we should generate the hash before allocating the images in the sprite (in order to make this fast) so the hash is created using only the input files and glue's settings.

As this information is not related with the sprite pixel distribution, we can optimize the image using (i.e optipng) change the image size, pixels, color deep etc... without any problem, and in the next run and while the metadata is present glue will skip this sprite.

And what about CSS?
We can't add metadata to a CSS file, so the only solution I've found is to add a small comment on top like
/* glue: 0.2.8 hash: 1bcb178c08 */. This is harmless and the comment would be removed during the CSS minification.

That's all, any change on the source images (the image itself or name) or any change on the settings (more padding, more margin, etc...) will make glue generate two different hashs and as consequence rebuild the sprite.

Thoughts?

[dominicbarnes]

That's an excellent idea! I wish I had thought of it lol.

[jorgebastida]

First speed tests...

50 x famfamfam folders using --project => 50.050 source images => 50 sprites.

First Run:

real      0m56.905s
user      0m49.612s
sys       0m4.747s

Second Run:

real      0m14.962s
user      0m12.011s
sys       0m2.613s

I'll try to make it better but it's encouraging :)

Note: OS X 10.8.2 - 2.2Ghz Core i7

[dominicbarnes]

Can't wait to see this land! :)

[jorgebastida]

After some refactoring I've archive some exciting results :)

First Run:

real      0m47.415s
user      0m45.599s
sys       0m1.286s

Second Run:

real      0m4.107s
user      0m2.959s
sys       0m0.817s

That's a 1.3x faster run for a cold run and a 14x for the second one.

[jvsummeren]

Will the metadata be stored in the CSS file (since ImageOptim will most likely strip all exif headers out)?

[jorgebastida]

Hi!

OptiPNG doesn't remove these metadata fields, but it seems that one of the libraries ImageOptim use (AdvPNG) yes.
I've read about and it seems that there is not any way to disable this behavior from AdvPNG.

The rest of the libraries ImageOptim use (PNGout PNGCrush and OptiPNG) play well with the metadata.

Your suggestion about only reading the metadata from the css file doesn't sounds very well to me... What about if you choose a --css location with files with the correct comment and an empty --img directory? or perhaps with images... but the wrong ones...

I think this feature is overall a benefit and in the worst case (where the PNG is not present) glue will behave like today.

I'll investigate how to make AdvPNG don't remove these data.

[jvsummeren]

And what about putting this metadata in both places and look if you can find either one of them? Does this sound plausible to you?

[jorgebastida]

Just released this fix as part of glue 0.2.8.
http://pypi.python.org/pypi/glue/
https://glue.readthedocs.org/en/latest/changelog.html#id1

[jorgebastida]

@jvsummeren The problem about only looking for the hash on either of them is that if for some reason the generated sprite or css file change and the other doesn't, glue will not rebuild them.

I think this approach is good enough for v1. Let's try this on production and see how could we improve it for the next version :)

Thank you guys for all your feedback and ideas!

[dominicbarnes]

Awesome! This new release is great! Thanks @jorgebastida