Comments (5)
Shouldn't be the cached file invalidated only when the version of a packages is newer?
from pip-accel.
Shouldn't be the cached file invalidated only when the version of a packages is newer?
Before I added that invalidation I actually expected the same thing, so your assumption is very reasonable! However a lot of people run CI builds on all changes to a repository, when no version bump has been performed in a Python package under source control, so unfortunately this is not a good solution (personally I avoid these problems by always bumping versions, if need be based on a "local revision number" from a version control repository).
Ideally I can get pip-accel to "do the right thing" in all situations, but I'm not yet sure how that should work. Several ideas come to mind though:
- Get the "last modified time" from the contents of the source and binary distributions. If possible this should be fairly simple to implement.
- Keep a map of SHA1 digests of all source and binary distributions and base the cache invalidation on a digest that changes. This is more work but the result would be platform independent.
- Add a configuration option that basically tells pip-accel to ignore last modified times and trust the user that version numbers will be bumped.
Without further input I'll probably investigate option one and fall back to option two if one doesn't work out (I don't like opt-in configuration options and would rather have the defaults work for the majority of users).
from pip-accel.
- I think that appveyor does not preserve mtime when caching files between builds and when it restores the pip-accel cache it just randomly copies files from the cache location.
- This is the appveyor doc for caching files: http://www.appveyor.com/docs/build-cache
from pip-accel.
@xolox I think for the majority of users would work this scenario:
- use SHA1 digest if it was created and is available for cached items
- fallback to "last modified time"
from pip-accel.
Hi Martin,
Because pip-accel's default cache invalidation scheme has always worked fine it seemed wasteful to unconditionally switch to the alternate scheme of calculating SHA1 hashes all the time (for what is essentially a quirk of AppVeyor CI, not Windows). Instead I've decided to make the cache invalidation scheme configurable.
In the newest version of pip-accel (0.36.2) the alternate cache invalidation scheme can be enabled by setting trust_mod_times to False
(this is the default on AppVeyor CI).
I hope this helps to avoid useless rebuilds. If you encounter problems feel free to reopen this issue or open a new one.
from pip-accel.
Related Issues (20)
- pip-accel not updating package HOT 3
- Doesn't work with pip 7+ HOT 2
- Add option for terse logging. HOT 5
- is_wheel fails when source_directory is None HOT 6
- Not able to use pip-accel both globally and inside a virtual environment HOT 4
- Support custom pip command HOT 7
- Upgrade to pip 8 HOT 4
- Disabling S3CacheBackend because it failed HOT 2
- pip-accel does not support running as a module HOT 6
- Is there any way to use pip-accel without installing to virtualenv?
- pip-accel is not compatible with `-e` editable installs of packages that use `setuptools_scm` HOT 2
- Why does `pip-accel` make an sdist (?) archive when I `pip-accel install -e .`? HOT 3
- pip-accel doesn't support the newest pip? HOT 4
- Not compatible with alternative install locations, e.g. `--user`? HOT 2
- [need advice] Do not cache development versions
- Failed to install some packages
- pip-accel should honor settings in pip.conf
- Is this project still alive? HOT 2
- Trying to install pip-accel, it insists on downgrading pip to 2.7. How can I get it to respect the current version, 21.1.2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pip-accel.