Upgrade to pip 8 about pip-accel

[iFreilicht]

We're actually at pip 9 now and pip 8 isn't supported yet. Are there any breaking changes from 8 to 9?

[tehfink]

Any ETA on this? By the way, thanks for a great project! It's great for developing while on the road.

[tehfink]

Ping

[xolox]

Sorry for not updating this issue earlier @tehfink and others who are interested. I started working on this upgrade quite a while ago (in May 2016) and initially it didn't seem that involved, but that impression turned out to be wrong and since then things have stalled.

The last time I tried to pick up this issue (when @tehfink asked for a pong) I actually published my intermediate work in the pip-8.1-upgrade branch intending to post a positive update here, but then lots of unanticipated Travis CI test failures popped up and I got seriously demotivated.

So unfortunately the new branch is not ready to be released yet. It's been a while since I had time to look at this in depth (my personal life and 40+ open source projects are filling up my to-do list faster than I can finish tasks) but I'll try to summarize the current status below (both for myself and others).

1. Minimal required changes

Making the minimal changes required to get most of the test suite running against pip 8.1.2 on my personal laptop didn't seem that hard at first, in fact the complete test suite was passing! However it did spew out deprecation warnings.

2. Switching to pip download

Because pip 8 deprecated the pip install --download option I decided to switch to the new pip download subcommand instead of rushing the pip 8 upgrade (reasoning that in the mean time pip-accel based on pip 7 would continue to work fine). These changes are available in 8af2069.

The switch to pip download got more complex than I would have liked because download doesn't support the same command line options as install. I want pip-accel to stay as backwards compatible as possible so I resorted to rewriting the command line arguments to remove options that are supported by pip install but not by pip download.

I don't like having to rewrite the command line options at all, but to be honest I don't see any other options if I want pip-accel to stay backwards compatible without using deprecated ways of calling pip.

Due to the rewriting of the command line arguments I need to make a conscious decision about every single pip install command line option: whether to support it, reject it (raise an error) or just silently drop it. In previous versions of pip-accel certain options would be silently accepted but not actually honored, which is (IMHO) worse than refusing those options, but still this raises some questions about backwards compatibility.

3. Test failures on Travis CI

After pushing the pip-8.1-upgrade branch to GitHub (the last time I looked at this issue) the new code was first tested on Travis CI and … the test suite failed on all Python versions with different tests failing on different versions - this discouraged me quite a bit because locally the test suite was running just fine on multiple Python versions! Here's an attempt to summarize the test failures:

	Python 2.6	Python 2.7	Python 3.4	Python 3.5	PyPy
test_editable_install			135686251	135686252
test_package_downgrade	135686249	135686250	135686251	135686252	135686253
test_setup_requires_caching			135686251	135686252

4. Finally making progress again

I already looked into these test failures a bit the first time I saw them, though not very detailed. My initial impression was that the test_editable_install and/or test_setup_requires_caching failures might be caused by a bug outside pip-accel, which was even more frustrating because I could find lots of very old bug reports, every time claiming that the issue was long since fixed.

Tonight I pushed 947810d to validate that theory and try out a fix and indeed, now only test_package_downgrade is failing, see Travis CI build #148472898. I guess I'll go and look into the one remaining test failure now :-) (which by the way is also passing for me locally, rather obscure :-S).

Once that is fixed I won't be ready to release the new version just yet, I'm considering asking people to try it out before it's officially released. As explained the switch from pip install --download to pip download required quite a bit more complexity than I'd initially hoped, and it's kind of hard for me to correctly judge the full impact on various (sometimes unanticipated) use cases of pip-accel.