When you set a WATCHDIR in s3sync it still tries to sync from S3 to /sync on startup about docker_base_images

[vladgh]

You're welcome. If you see any improvements please feel free to open a PR.
This discussion triggered some changes though, that I will apply later today, and they might not be backward compatible.

[vladgh]

Hey @ryanwalls, that makes perfect sense.
However, this is the intended behaviour. This script is actually downloading the files first FROM (in both modes: cron or inotify), and only after this initial step, it will start uploading files TO S3 (periodically or when files change in the 'watchdir').
Probably this is not very clear in the way I documented the code, but I'm willing to change it, to maybe accommodate both cases. One in which it just syncs to S3, and one in which it syncs from S3 first and then syncs to S3.

[vladgh]

S3Sync

[ryanwalls]

@vladgh Ah that makes sense. Yeah, it would be nice to have the enhancement of syncing only from host to s3 without the initial download.

[vladgh]

Instead of adding another environment variable, what about a simple check if WATCHDIR is empty or not? If it is empty do the initial download, if it has files in it then skip it.

[ryanwalls]

In my case I want the WATCHDIR to control the sync in all cases. So I would want the ability to disable the initial sync whether WATCHDIR is empty or not.

[ryanwalls]

One minor point about the inotify setup: seems we have 3 locations specified S3PATH, WATCHDIR, and DESTINATION, when we are really syncing between just 2 locations. Not sure best way to fix that or if that is intentional. Perhaps just use WATCHDIR as the destination if it is set?

[vladgh]

That DESTINATION var is redundant anyway, I will get rid of it.

[ryanwalls]

Just realized we actually don't need to have the WATCHDIR control the sync. Have another way to solve the problem. So I would put this as a lower priority. :)

Thanks for the tool. Looked at a lot of s3 syncs and this was the best setup.

[vladgh]

Is this working better for you?

[ryanwalls]

@vladgh So far so good. Thanks!