erikrose / peep

A "pip install" that is cryptographically guaranteed repeatable

Note

Peep is deprecated, as we have merged its functionality into pip 8. This brings myriad improvements, including support for caching, detection of omitted dependencies, and better handling of errors and corner cases. To switch to pip 8's hash-checking without hitting any race conditions...

1. Upgrade to peep 3.0 (which exists mainly as a stopgap to support race-free upgrades like this).
2. Upgrade to pip 8.
3. Atomically, switch the format of your requirements files using peep port (described below), and start calling pip instead of peep.
4. Delete peep from your project.

Thank you for using peep! Your early support helped establish hash verification as a feature worth uplifting, and now the package ecosystem is safer for everyone.

Here are some more detailed upgrade instructions in case you need them.

Peep

Deploying Python projects has long been a source of frustration for the security-conscious: a compromise of PyPI or its third-party CDN could get you a package different from the one you signed up for. To guarantee known-good dependencies for your deployments, you had to run a local package index, manually uploading packages as you vetted them, maintaining a set of ACLs for that server, and trying to somehow keep an audit trail of who did what. Alternatively, you could check everything into a vendor library, but that meant a lot of fooling around with your VCS (or maintaining custom tooling) to do upgrades.

Peep fixes all that.

Vet your packages, and put hashes of the PyPI-sourced tarballs into requirements.txt, like this:

# sha256: L9XU_-gfdi3So-WEctaQoNu6N2Z3ZQYAOu4-16qor-8
Flask==0.9

# sha256: qF4YU3XbdcEJ-Z7N49VUFfA15waKgiUs9PFsZnrDj0k
Jinja2==2.6

Then, use peep install instead of pip install, and let the crypto do the rest. If a downloaded package doesn't match the expected hash, Peep will freak out, and installation will go no further.

There are no servers to maintain, no enormous vendor libs to wrestle, and no need to trust a package author's key management practices. With the addition of a few hashes to your requirements file, you can know that your chain of trust is safely rooted in your own source tree.

Switching to Peep

1. Install Peep:

   pip install peep
   

   (Or, better, embed peep.py into your codebase as described in the Embedding section below. That eliminates having to trust an unauthenticated PyPI download, assuming you manually vet peep.py itself the first time.)

2. Use Peep to install your project once:

   cd yourproject
   peep install -r requirements.txt
   

   You'll get output like this:

   <a bunch of pip output>
   
   The following packages had no hashes specified in the requirements file,
   which leaves them open to tampering. Vet these packages to your
   satisfaction, then add these "sha256" lines like so:
   
   # sha256: L9XU_-gfdi3So-WEctaQoNu6N2Z3ZQYAOu4-16qor-8
   Flask==0.9
   
   # sha256: qF4YU3XbdcEJ-Z7N49VUFfA15waKgiUs9PFsZnrDj0k
   Jinja2==2.6
   
   # sha256: u_8C3DCeUoRt2WPSlIOnKV_MAhYkc40zNZxDlxCA-as
   Pygments==1.4
   
   # sha256: A1gwhyCNozcxug18_9RjJTmJQa1rctOt-AnP7_yR0PM
   https://github.com/jsocol/commonware/archive/b5544185b2d24adc1eb512735990752400ce9cbd.zip#egg=commonware
   
   -------------------------------
   Not proceeding to installation.
   

3. Vet the packages coming off PyPI in whatever way you typically do. For instance, read them, or compare them with known-good local copies.

4. Add the recommended hash lines to your requirements.txt, each one directly above the requirement it applies to. (The hashes are of the original, compressed tarballs from PyPI.)

5. In the future, always use peep install to install your requirements. You are now cryptographically safe!

Warning

Be careful not to nullify all your work when you install your actual project. If you use python setup.py install, setuptools will happily go out and download, unchecked, any requirements you missed in requirements.txt (and it's easy to miss some as your project evolves). One way to be safe is to pack up your project and then install that using pip and --no-deps:

python setup.py sdist
pip install --no-deps dist/yourproject-1.0.tar.gz

The Fearsome Warning

If, during installation, a hash doesn't match, Peep will say something like this:

THE FOLLOWING PACKAGES DIDN'T MATCH THE HASHES SPECIFIED IN THE
REQUIREMENTS FILE. If you have updated the package versions, update the
hashes. If not, freak out, because someone has tampered with the packages.

    requests: expected FWvz7Ce6nsfgz4--AoCHGAmdIY3kA-tkpxTXO6GimrE
                   got YhddA1kUpMLVODNbhIgHfQn88vioPHLwayTyqwOJEgY

It will then exit with a status of 1. Freak out appropriately.

Other Features

• Peep implicitly turns on pip's --no-deps option so unverified dependencies of your requirements can't sneak through.

• All non-install commands just fall through to pip, so you can use Peep all the time if you want. This comes in handy for existing scripts that have a big $PIP=/path/to/pip at the top.

• Peep-compatible requirements files remain entirely usable with pip, because the hashes are just comments, after all.

• Have a manually downloaded package you've vetted? Run peep hash on its tarball (the original, from PyPI--be sure to keep it around) to get its hash line:

  % peep hash nose-1.3.0.tar.gz
  # sha256: TmPMMyXedc-Y_61AvnL6aXU96CRpUXMXj3TANP5PUmA
  

• If a package is already present--which might be the case if you're installing into a non-empty virtualenv--Peep doesn't bother downloading or building it again. It assumes you installed it with Peep in a previous invocation and thus trusts it. The only exception to this is for URL-specified requirements where the URL contains a SHA-like filename (eg https://github.com/foo/bar/archive/<SHA>.zip), since the package version number is typically not incremented for every commit, so Peep cannot be sure the contents have not changed. Note: Re-using a virtualenv during deployment can really speed things up, but you will need to manually remove dependencies that are no longer in the requirements file.

• peep port converts a peep-savvy requirements file to one compatible with pip 8's new hashing functionality:

  % peep port requirements.txt
  certifi==2015.04.28 \
      --hash=sha256:268fa00c27de756d71663dd61f73a4a8d8727569bb1b474b2ce6020553826872 \
      --hash=sha256:99785e6cf715cdcde59dee05a676e99f04835a71e7ced201ca317401c322ba96
  click==4.0 --hash=sha256:9ab1d313f99b209f8f71a629f36833030c8d7c72282cf7756834baf567dca662
  

  Note that comments and URLs don't make it through, but the hard part—hash format conversion—is taken care of for you.

Embedding

Peep was designed for unsupervised continuous deployment scenarios. In such scenarios, manual ahead-of-time preparation on the deployment machine is a liability: one more thing to go wrong. To relieve you of having to install (and upgrade) Peep by hand on your server or build box, we've made Peep embeddable. You can copy the peep.py file directly into your project's source tree and call it from there in your deployment script. This also gives you an obvious starting point for your chain of trust: however you trust your source code is how you trust your copy of Peep, and Peep verifies everything else via hashes. (Equivalent would be if your OS provided Peep as a package--presumably you trust your OS packages already--but this is not yet common.)

Security and Insecurity

Here's what you get for free with Peep--and what you don't.

You get repeatability. If you peep install package Foo==1.2.3, every subsequent install of Foo==1.2.3 will be the same as the original (or Peep will complain).

Peep does not magically vet your packages. Peep is not a substitute for combing through your packages for malicious code or comparing them with known-good versions. If you don't vet them, they are not vetted.

Peep does not make authors or indices trustworthy. All Peep does is guarantee that subsequent downloads of Foo==1.2.3 are the same as the first one. It doesn't guarantee the author of that package is trustworthy. It doesn't guarantee that the author of that package is the one who released that package. It doesn't guarantee that the package index is trustworthy.

Troubleshooting

Multiple Hashes: Architecture-Specific Packages and Old Versions of PyPI

Are you suddenly getting the Fearsome Warning? Maybe you're really in trouble, but maybe something more innocuous is happening.

If your packages install from wheels or other potentially architecture-specific sources, their hashes will obviously differ across platforms. If you deploy on more than one, you'll need more than one hash.

Also, some packages offer downloads in multiple formats: for example, zips and tarballs, or zips and wheels. Which version gets downloaded can vary based on your version of pip, meaning some packages may effectively have more than one valid hash.

To support these scenarios, you can stack up multiple known-good hashes above a requirement, as long as they are within a contiguous block of commented lines:

# Tarball:
# sha256: lvpN706AIAvoJ8P1EUfdez-ohzuSB-MyXUe6Rb8ppcE
#
# And the zip file:
# sha256: 6QTt-5DahBKcBiUs06BfkLTuvBu1uF7pblb_bPaUONU
mock==0.8.0

If you don't want to wait until you're bitten by this surprise, use the peep hash command to find hashes of each equivalent archive for a package. I like to vet one of them (say, the tarball), then download the others and use a file comparison tool to verify that they have identical contents. Then I run peep hash over both original archives, like so, and add the result to my requirements.txt:

% peep hash mock-0.8.0.tar.gz mock-0.8.0.zip
# sha256: lvpN706AIAvoJ8P1EUfdez-ohzuSB-MyXUe6Rb8ppcE
# sha256: 6QTt-5DahBKcBiUs06BfkLTuvBu1uF7pblb_bPaUONU

Upgrading Wheels with Old Versions of pip

If you're reusing a virtualenv and using Peep with pip <6.0, then you should avoid using wheels. Otherwise, the old version of a package will not be entirely removed before the new one is installed, due to pypa/pip#1825.

If you're using pip 1.4, don't pass the --use-wheel argument.

If you're using pip 1.5, pass the --no-use-wheel argument.

Version History

3.1.2

• Fix compatibility with pip 8.1.2. (abbeyj)

3.1.1

• The "peep had a problem" traceback is no longer output for several cases of pip installation errors that were not peep's fault: for instance, the specified package version or requirements file not existing.
• peep port now emits URLs for URL-based requirements, if you're using pip 6.1.0 or greater. (jotes)

3.1

• Print the name each new requirements file we encounter during peep port. This helps untangle the mess if your files use includes. (pmac)
• Always put hashes on their own lines, even if there's only one. (pmac)

3.0

• Add support for pip 8.x.
• Drop support for the --allow-external, --allow-unverified and --allow-all-external arguments (for compatibility with pip 8).
• Drop support for Python 3.1/3.2.

2.5

• Support pip 7.x, through the currently latest 7.1.2, working around its buggy line counting. (kouk)
• Add peep port command to facilitate the transition to pip 8's hashing.
• Fix bug in which the right way to call parse_requirements() would not be autodetected.

2.4.1

• Tolerate pip.__version__ being missing, which can apparently happen in arcane situations during error handling, obscuring informative tracebacks.
• Fix flake8 warnings again, and add flake8 to Travis runs.

2.4

• Add support for flags in the requirements file, pip-style, such as specifying alternative indices with -i.
• Remove a duplicate #egg= segment from an error message.

2.3

• Copy the operative portion of the MIT license into peep.py so embedding it doesn't break the license.
• Fix flake8 linter warnings.
• Make peep compatible with pip v6.1.0+.
• Add tests against pip 6.0.8, 6.1.0, and 6.1.1 to the tox config.
• Run full set of tox tests on Travis.

2.2

• Add progress indication while downloading. Used with pip 6.0 and above, we show a nice progress bar. Before that, we just mention the packages as we download them.
• Remove extra skipped lines from the output.
• Add tests against pip 6.0.7 to the tox config.

2.1.2

• Get rid of repetition of explanatory messages at the end of a run when one applies to multiple packages.

2.1.1

• Fix bug in which peep would not upgrade a package expressed in terms of a GitHub-dwelling zip file if its version had not changed.
• Add tests against pip 6.0.4, 6.0.5, and 6.0.6 to the tox config.

2.1

• Support pip 6.x.
• Make error reporting friendly, emitting a bug reporting URL and environment info along with the traceback.

2.0

• Fix major security hole in which a package's setup.py would be executed after download, regardless of whether the package's archive matched a hash. Specifically, stop relying on pip for downloading packages, as it likes to run setup.py to extract metadata. Implement our own downloading using what's available everywhere: urllib2. As a result, HTTP proxies, basic auth, and --download-cache are unsupported at the moment.
• Refactor significantly for comprehensibility.
• Drastically improve test coverage.
• Note that HTTPS certs are no longer checked. This shouldn't matter, given our hash checks.

1.4

• Allow partial-line comments.
• Add the beginnings of a test suite.
• Treat package names in requirements files as case-insensitive, like pip.

1.3

• Pass through most args to the invocation of pip install that actually installs the downloaded archive. This means you can use things like --install-options fruitfully.
• Add Python 3.4 support by correcting an import.
• Install a second peep script named after the active Python version, e.g. peep-2.7. This is a convenience for those using multiple versions of Python and not using virtualenvs.

1.2

• Support GitHub-style tarballs (that is, ones whose filenames don't contain the distro name or version and whose version numbers aren't reliable) in requirements files. (Will Kahn-Greene)
• Warn when a URL-based requirement lacks #egg=. (Chris Adams)

1.1

• Support Python 3. (Keryn Knight)

1.0.2

• Add support for .tar.bz2 archives. (Paul McLanahan)

1.0.1

• Fix error (which failed safe) installing packages whose distro names contain underscores. (Chris Ladd)

1.0

• Add wheel support. Peep will now work fine when pip decides to download a wheel file. (Paul McLanahan)

0.9.1

• Don't crash when trying to report a missing hash on a package that's already installed.

0.9

• Put the operative parts of peep into a single module rather than a package, and make it directly executable. (Brian Warner)

0.8

• Support installing into non-empty virtualenvs, for speed. We do this by trusting any already-installed package which satisfies a requirement. This means you no longer have to rebuild lxml, for instance, each time you deploy.
• Wrap text output to 80 columns for nicer word wrap.

0.7

Make some practical tweaks for projects which bootstrap their trust chains by checking a tarball of peep into their source trees.

• Start supporting versions of pip back to 0.6.2 (released in January 2010). This way, you can deploy trustworthily on old versions of RHEL just by checking a tarball of peep into your source tree and pip-installing it; you don't have to check in pip itself or go to the bother of unpacking the peep tarball and running python setup.py install from your deploy script.
• Remove the explicit dependency on pip. This is so a blithe call to pip install peep.tar.gz without --no-deps doesn't go out and pull an untrusted package from PyPI. Instead, we scream at runtime if pip is absent or too old. Fail safe.

0.6

• Add peep hash subcommand.
• Require pip>=1.2, as lower versions have a bug that causes a crash on peep install.

0.5

• Allow multiple acceptable hashes for a package. This works around PyPI's non-stable handling of packages like mock, which provide equivalent zips and tarballs: https://bitbucket.org/pypa/pypi/issue/64/order-of-archives-on-index-page-is-not.

0.4

• Rework how peep downloads files and determines versions so we can tolerate PEP-386-noncompliant package version numbers. This amounted to a minor rewrite.
• Remove indentation from hash output so you don't have to dedent it after pasting it into requirements.txt.

0.3

• Support Windows and other non-Unix OSes.
• The hash output now includes the actual version numbers of packages, so you can just paste it straight into your requirements.txt.

0.2.1

• Add a shebang line so you can actually run peep after doing pip install peep. Sorry, folks, I was doing setup.py develop on my own box.

0.2

• Fix repeated-logging bug.
• Fix spurious error message about not having any requirements files.
• Pass pip's exit code through to the outside for calls to non-install subcommands.
• Improve spacing in the final output.

0.1

• Proof of concept. Does all the crypto stuff. Should be secure. Some rough edges in the UI.

peep's Issues

update tox.ini to add pip 6.x test runs

We need to update tox.ini to add pip 6.x test runs. At the time of this writing, it's at least 6.0, 6.0.1 and 6.0.2.

Support for '--extra-index-url/--index-url' in requirements.txt

pip supports specifying the --extra-index-url/--index-url options in a requirements.txt file, e.g.

--index-url=https://our-internal-package-server.com/pypi/
# sha256: f11aEB33vl31Rw53uFAN_govGddZgkWBScQHEfg9Q4w
custom-package==1.0.1

However, attempting to install this with peep fails:

$ peep install -r requirements.txt
No handlers could be found for logger "pip.index"

Traceback (most recent call last):
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/bin/peep", line 9, in <module>
    load_entry_point('peep==2.1', 'console_scripts', 'peep')()
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 811, in main
    return commands[argv[1]](argv[2:])
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 771, in peep_install
    for path in req_paths))
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 771, in <genexpr>
    for path in req_paths))
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 747, in downloaded_reqs_from_path
    session=PipSession())]
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 340, in __init__
    self.__class__ = self._class()
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 615, in _class
    if self._actual_hash() not in self._expected_hashes():
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 252, in memoizer
    self._cache[func.__name__] = func(self)
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 580, in _actual_hash
    return hash_of_file(join(self._temp_path, self._downloaded_filename()))
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 252, in memoizer
    self._cache[func.__name__] = func(self)
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/peep.py", line 535, in _downloaded_filename
    if self._req.url is None
  File "/Users/rouge8/.virtualenvs/tmp-f31839f7ea06217/lib/python2.7/site-packages/pip/index.py", line 397, in find_requirement
    'No distributions at all found for %s' % req
pip.exceptions.DistributionNotFound: No distributions at all found for custom-package==1.0.1
 (from -r requirements.txt (line 9))

Option to keep downloaded tarballs

I just had two hash failures in a peep-install and would have loved to see what's different in the tarballs, but I don't see any promising option to not delete the tempdir; does one exist already?

If not, hopefully I can come up with a patch.

Crash with install -r on a tarball source

I ran peep install -r on a large project; it eventually bombed out with this unhelpful error message:

Cleaning up...
Traceback (most recent call last):
  File "/Users/cadams/Library/Python/2.7/bin/peep", line 8, in <module>
    load_entry_point('peep==0.8', 'console_scripts', 'peep')()
  File "/Users/cadams/Library/Python/2.7/lib/python/site-packages/peep/__init__.py", line 366, in main
    return commands[argv[1]](argv[2:])
  File "/Users/cadams/Library/Python/2.7/lib/python/site-packages/peep/__init__.py", line 304, in peep_install
    name = req.req.project_name
AttributeError: 'NoneType' object has no attribute 'project_name'

A quick python -m pdb which peep peep install -r requirements.pip later and I had the culprit, which turned out to be a tarball install line:

http://github.com/acdha/django-sugar/tarball/fc193995aa012b9cb9d4f24e19ddfbc76a488b01

It's not immediately clear whether this should have worked or not – the setup.py file does have a name but the InstallRequirement instance has name = None – but it does also suggest both that this error could be cleaner and that InstallRequirement's str / repr methods could be more helpful.

`--install-options` not kept for the pip install

If you do peep install --install-options="..." -r requirements.txt it'll download the package, compare it and if all is well proceed with the (pip) install. Doing that it forgets your you --install-options stuff you sent the first time.

TODO in code here

KeyError raised if specified module already installed

It seems --ignore-installed is not passed through to pip:

$ ./peep.py install --user --ignore-installed -r requirements.txt 

The following packages had no hashes specified in the requirements file, which
leaves them open to tampering. Vet these packages to your satisfaction, then
add these "sha256" lines like so:

Traceback (most recent call last):
  File "./peep.py", line 378, in <module>
    exit(main())
  File "./peep.py", line 370, in main
    return commands[argv[1]](argv[2:])
  File "./peep.py", line 343, in peep_install
    print '# sha256: %s' % downloaded_hashes[package_name]
KeyError: 'nose'

$ python
Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nose
>>> nose
<module 'nose' from '/usr/lib/python2.7/dist-packages/nose/__init__.pyc'>
>>> 

setuptools==0.6 can cause DistributionNotFound error while using peep.

It shows the 'You don't have pip>=0.6.8' error, but it's caused by the DistributionNotFound error that's thrown.

There are a couple solutions:

1. Add a requirements file with setuptools>=0.9.8
2. Anytime setuptools < 0.9.8 is tried to be installed, throw an error and let the user know why.

I say setuptools>=0.9.8 because according to the docs[1] it's supposed to be backwards compatible with all previous setuptools versions.

[1] https://pypi.python.org/pypi/setuptools

During successful install, log messages happen 7 times.

Running setup.py install for Werkzeug
Running setup.py install for Werkzeug

Running setup.py install for Werkzeug


Running setup.py install for Werkzeug



Running setup.py install for Werkzeug




Running setup.py install for Werkzeug





Running setup.py install for Werkzeug

Stop that.

Support Python 3.x

Peep doesn't play well with tox

Simply using install_command = peep.py install {opts} {packages} won't work as is, because by default tox uses this same install command to install the distribution it created for the project under test.

There's two ways around ths:

• use the usedevelop = True tox setting: this way, it does a pip install -e .': this brings us nowhere, because peep doesn't accept the-e` parameter
• use the skipsdist = True tox setting, and install the project in the commands using python setup.py develop

The second option works "out of the box", but needs an extra step, and feels a bit hacky. The first option looks nice, but needs peep to accept the -e parameter (and then maybe just forward that to pip?).

The workaround for the time being is to simply not use the install_command nor the deps, but install the dependencies in the commands:

commands = 
    peep.py install -r requirements.txt
    # test command

Peep can't get version from Wheel archive file

Not sure what the exact situation is. Here's the deal though:

Running peep.py from current master (00ec6c1).

The requrements.txt file

Django==1.6.2
django-appconf==0.6
django-compressor==1.3
redis==2.9.1
six==1.6.1

The CLI output

(mrburns) ± bin/peep.py install -r requirements.txt
Downloading/unpacking Django==1.6.2
  Using download cache from /Users/pmclanahan/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fany%2FD%2FDjango%2FDjango-1.6.2-py2.py3-none-any.whl
  Saved /var/folders/z6/4r3j0xc54yg70twp3nvbm5x80000gn/T/peep-hFKwxA/Django-1.6.2-py2.py3-none-any.whl
Successfully downloaded Django
Cleaning up...
Downloading/unpacking django-appconf==0.6
  Using download cache from /Users/pmclanahan/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2F2.7%2Fd%2Fdjango-appconf%2Fdjango_appconf-0.6-py2.py3-none-any.whl
  Saved /var/folders/z6/4r3j0xc54yg70twp3nvbm5x80000gn/T/peep-hFKwxA/django_appconf-0.6-py2.py3-none-any.whl
Successfully downloaded django-appconf
Cleaning up...
Traceback (most recent call last):
  File "bin/peep.py", line 382, in <module>
    exit(main())
  File "bin/peep.py", line 374, in main
    return commands[argv[1]](argv[2:])
  File "bin/peep.py", line 311, in peep_install
    downloaded_versions[name] = version_of_archive(archive_filename, name)
  File "bin/peep.py", line 166, in version_of_archive
    raise RuntimeError("The archive '%s' didn't start with the package name '%s', so I couldn't figure out the version number. My bad; improve me.")
RuntimeError: The archive '%s' didn't start with the package name '%s', so I couldn't figure out the version number. My bad; improve me.

Should archive/package comparison be case-insensitive?

In a requirements.txt file I had, there was PyOpenSSL==0.14

Although pip will install this, peep fails at:

RuntimeError: The archive 'pyOpenSSL-0.14' didn't start with the package name 'PyOpenSSL', so I couldn't figure out the version number. My bad; improve me.

In this case, I can simply change the requirements.txt to be "pyOpenSSL==0.14" but if pip merrily installs this anyway, perhaps it should just be a warning in peep?

peep crashes when deps are already in the virtual env

I had a requirements file with a single requirement. The package was already installed in my virtual environment, but I wanted peep to give me the sha to put in the file.

It appears that peep errors when the package is already installed into the virtual env because it never downloads files to hash.

console log of the problem with description:
https://gist.github.com/lonnen/c5dc13deb12589f6de04

Support offline use

Another use case, besides trustability, for a local PyPI mirror is resilience to network failures. It's nice to be able to deploy even when PyPI is down or inaccessible. At the moment, peep installing with --download-cache still insists on hitting PyPI, failing out if it can't. That seems unnecessary. Find the right set of flags to pass to pip to make that not happen, or at least understand why it needs to happen.

"KeyError: None" when there's no req.name

With this requirements file:

http://people.mozilla.org/~wkahngreene/pep8/pep8-1.6.0a0-willkg.1.tar.gz

I get this:

<pep-test> saturn2 ~/tmp/pep8-test> ./peep install -r reqs.txt 

The following packages had no hashes specified in the requirements file, which
leaves them open to tampering. Vet these packages to your satisfaction, then
add these "sha256" lines like so:

Traceback (most recent call last):
  File "./peep", line 486, in <module>
    exit(main())
  File "./peep", line 478, in main
    return commands[argv[1]](argv[2:])
  File "./peep", line 442, in peep_install
    print('# sha256: %s' % downloaded_hashes[req.name])
KeyError: None

That's with Python 2.7, pip 1.5.6 and peep 1.3.0.

I'm not entirely sure what's going on, but I suspect req.name is junk or there's nothing in downloaded_hashes.

Allow comments on hash lines.

Especially when there are multiple hashes for one package (think wheel vs tarball, windows vs linux vs osx build, etc), it would be nice to give comments to each hash to identify where it came from. The only way to do that now is to add comments between the hashes (either above or below) to explain what they are for. I think this is less readable and takes up a lot of space for a few words about what is being installed.

I think a syntax like this would be ok

# sha256: cVnbFB5A1yz_4Wk_xDAjIf7WfMwBU4vyXrNurfi4n3E # wheel
# sha256: NWZcEPVSRwd27M_RWSLTdI4w18AQLrd_3kfOKX2WuyM # tarball
djangorestframework==2.3.14

I don't know if the comments above are actually correct, but I would have known the correct comment to write when I added those hashes to to the file.

I also thought about:

• Prepending the comment to the line, like

  # tarball sha256sum: NWZcEPVSRwd27M_RWSLTdI4w18AQLrd_3kfOKX2WuyM
  

• Appending the comment to the end without an extra character (space delimited) like

  # sha256sum: NWZcEPVSRwd27M_RWSLTdI4w18AQLrd_3kfOKX2WuyM tarball
  

Both of these looked strange to me, and I think they would make the file harder for humans to visually parse.

"ImportError: No module named log" when using peep with newer versions of pip

The pip.log module was removed in favor of the stdlib logger in pypa/pip#2008 but it is imported in https://github.com/erikrose/peep/blob/master/peep.py#L74 in order to clear out the registrations in the logger singleton in https://github.com/erikrose/peep/blob/master/peep.py#L130. I'll submit a PR that adds error handling to the import shortly.

Maybe mess with freeze output.

Maybe make peep freeze lay out the hashes. I came up with that, and then somebody asked about it independently.

Install as peep-${major}.${minor}

I have python version 2.7 and 3.3 on my boxes, but peep installs as /usr/local/bin/peep no matter which flavour of pip I use. It would be nice to have peep install as peep-${major}.${minor} — i.e., as peep-2.7 and peep-3.3 in my case — so that I can easily invoke the right flavour.

Cheers.

urlparse is now in the urllib library in Python 3.4

Had to change the line
from urlparse import urlparse

to

import urllib.parse as urlparse

Junk leftover when installing a new version after installing github tarball.

1. Make and activate a virtualenv.

2. Install something with a github url.

   $ cat github.txt
   # elasticutils: tags/v0.9.1^0
   # sha256: OTZ4mjjiLzXDRK7mbZ5hzOSR7v62NYpUYS_FYNKBoPo
   https://github.com/mozilla/elasticutils/archive/641ccade9cfba1415809077fad674b41b436d174.tar.gz#egg=elasticutils
   $ peep install -r github.txt
   

3. Install the same library, but a new version, from pypi

   $ cat pypi.txt
   # sha256: CnOPMnGcdnU38yYRjBBcIZ5nfna3hUfthHxqqvvKDGU
   elasticutils==0.10.1
   $ peep install -r pypi.txt

4. Observe that the metadata for the both the first and second installation of the package exist at once.

   $ ls venv/lib/python2.6/site-packages/
   elasticutils/                       pip-1.5.6.dist-info/       easy_install.pyc
   elasticutils-0.9.1-py2.6.egg-info/  setuptools/                peep.py
   elasticutils-0.10.1.dist-info/      setuptools-3.6.dist-info/  pkg_resources.py
   peep-1.3-py2.6.egg-info/            _markerlib/                pkg_resources.pyc
   pip/

5. Be sad.

I'm not sure if the contents of the actual package (venv/lib/python2.6/site-packages/elasticutils) is entirely the new version, or if it is a mash-up of the old version and the new version. I don't know elasticutils well enough to tell right now.

When installing from pypi, peep/pip opted to download a wheel file. I'm not sure if this is relavent.

Pip does not have this behavior. It's logs contain a few lines that indicate it is removing old versions before installing the new versions. Peep's logs do not contain any mention of this.

Installing collected packages: elasticutils, elasticsearch, six
  Found existing installation: elasticutils 0.9.1
    Uninstalling elasticutils:
      Successfully uninstalled elasticutils
  Found existing installation: elasticsearch 0.4.5
    Uninstalling elasticsearch:
      Successfully uninstalled elasticsearch
Successfully installed elasticutils elasticsearch six

Set pip executable path from envvar

The embeddable peep.py (#20) seems to be using the system pip, and not sure how to make it use ours (specific version of pip vendored in project) with gratipay/inside.gratipay.com#30

Is this something that would make sense to implement?

Print actual lines from requirements file in the advisory output

Rather than...

# sha256: L9XU_-gfdi3So-WEctaQoNu6N2Z3ZQYAOu4-16qor-8
Flask==x.y.z

...print the actual Flask version number, etc. Preserve comments, if you want to be really nice.

Improve feedback when there are missing dependencies

Chris Calloway says...

Here's what I got on Win7/64 with Python 2.7.5 via Anaconda: I did a dummy requirements file with with my favorite dummy packages: pep8, pyflakes, and pylint, to peep into a virtualenv (or rather, a conda env). Without the sha256 lines, it worked as expected ("Not proceeding to installation"). With the proper sha256 lines, it appeared to install everything. But because of the implicit --no-deps, pylint's dependencies were not installed silently and i'm left with a broken pylint. Yeah, that doesn't let unverified dependencies through. But it's probably not a great default behavior, either. One of pip's great advances over easy_install was that it not only figured out all of the nested dependencies like easy_install, but it also made sure to download them all first before attempting to install anything. A more consistent behavoir might be to raise the "Not proceeding to installation" exception for unverified deps.

Maybe we could run pip --download without --no-deps and then freak out if there are things in the download folder we didn't explicitly ask for. (Or maybe there's a way to do it without network IO (but that's less important, because it's valuable to verify the hashes anyway).) When we run pip install, we can include --no-deps but install only the requirements that appear by name in the reqs file.

Would peep get along with download caches?

Building a new venv from scratch with each deployment takes awhile: you have to download the packages and compile the compiley ones.

We can solve the first problem using a pip download cache. Would peep take advantage of one of those? Could we make it do so?

Avoid recompiling compiley packages

It might be nice to avoid C recompiles. We could certainly avoid rebuilding a package that's already installed in the venv. We could assume that, if you used peep to install stuff in the venv in the first place, the hashes were verified. How would we go about making sure anything not in the requirements.txt any longer gets removed? Out of scope for peep? It would be pretty trivial to implement as a layer around pip freeze and pip uninstall, with a diff against requirements.txt: uninstall what shouldn't be there, then do the peep install.

Peep doesn't support -rrequirements.txt

The usual way to use a different install command for dependencies in tox is to use the install_command setting.

Here's an example (it needs the peep.pyscript available):

[testenv:tests]
install_command = {toxinidir}/bin/peep.py install {opts} {packages}
deps =
    -rrequirements.txt

However, peep doesn't accept passing the requirements file this way (it needs a space between the two: -r requirements.txt. Pip does accept this though.

If you were to add this space, to have peep accept it, then it would be tox's turn to fail, calling the following command: (notice there is only one parameter for '-r requirements.txt', instead of two like '-r', 'requirements.txt'):
cmdargs=['/Users/mathieu/test_sugardough/bin/peep.py', 'install', '-r requirements.txt']

From a discussion in irc with @erikrose and @jezdez, we found out that a custom arg parser was coded for peep (due to optparse's decision to explode when it encounters unfamiliar options), but that might not be needed if using the same trick as the following code: https://github.com/django/django/blob/stable/1.4.x/django/core/management/__init__.py#L152-L204

The workaround for the time being is to simply not use the install_command nor the deps, but install the dependencies in the commands:

commands = 
    peep.py install -r requirements.txt
    # test command

This is only partially the problem with using peep together with tox, please see #62

Checksum always different on git URL installs

My requirements file:

$ cat requirement.txt
-e git+https://github.com/pyflakes/pyflakes.git@06c713990a87d3c10cd02749ee7e0ee44220ea4c#egg=pyflakes

This installs just fine on doing pip install -r requirement.txt
Here's what happens with peep:

(venv)peterbe@mpb:/private/tmp/venv$ peep install -r requirement.txt
Obtaining pyflakes from git+https://github.com/pyflakes/pyflakes.git@06c713990a87d3c10cd02749ee7e0ee44220ea4c#egg=pyflakes
  Cloning https://github.com/pyflakes/pyflakes.git (to 06c713990a87d3c10cd02749ee7e0ee44220ea4c) to /var/folders/l7/9ffkzwg965gf0zl261g34n_m0000gp/T/pip-aIuKUX-export
  Could not find a tag or branch '06c713990a87d3c10cd02749ee7e0ee44220ea4c', assuming commit.
  Running setup.py (path:/private/tmp/venv/src/pyflakes/setup.py) egg_info for package pyflakes

Saved /var/folders/l7/9ffkzwg965gf0zl261g34n_m0000gp/T/peep-KzRYTs/pyflakes-0.8.zip
Installing extra requirements: 'egg'
Successfully downloaded pyflakes
Cleaning up...

The following packages had no hashes specified in the requirements file, which
leaves them open to tampering. Vet these packages to your satisfaction, then
add these "sha256" lines like so:

# sha256: 4YRvF7Xhvsd0AfHPhJW3GvZOXARI-uaSF-H4PWXODoQ
pyflakes==0.8

-------------------------------
Not proceeding to installation.

a second time

(venv)peterbe@mpb:/private/tmp/venv$ peep install -r requirement.txt
Obtaining pyflakes from git+https://github.com/pyflakes/pyflakes.git@06c713990a87d3c10cd02749ee7e0ee44220ea4c#egg=pyflakes
  Cloning https://github.com/pyflakes/pyflakes.git (to 06c713990a87d3c10cd02749ee7e0ee44220ea4c) to /var/folders/l7/9ffkzwg965gf0zl261g34n_m0000gp/T/pip-N4xbCr-export
  Could not find a tag or branch '06c713990a87d3c10cd02749ee7e0ee44220ea4c', assuming commit.
  Running setup.py (path:/private/tmp/venv/src/pyflakes/setup.py) egg_info for package pyflakes

Saved /var/folders/l7/9ffkzwg965gf0zl261g34n_m0000gp/T/peep-RSOrIc/pyflakes-0.8.zip
Installing extra requirements: 'egg'
Successfully downloaded pyflakes
Cleaning up...

The following packages had no hashes specified in the requirements file, which
leaves them open to tampering. Vet these packages to your satisfaction, then
add these "sha256" lines like so:

# sha256: 95em-8NHpy_6amMr6cSH_djdEp__FyU-B2uzwZb4Yq4
pyflakes==0.8

-------------------------------
Not proceeding to installation.

Problem is that the checksum is always different repeatedly.

Confirm compatibility with various pip versions

I'm using 1.2.1 locally. @peterbe and @selenamarie have reported trouble with other (newer!) versions:

(socorro)peterbe@mpb:~/dev/MOZILLA/SOCORRO/socorro/webapp-django (bug-912011-colors-in-front-page-graph-dont-match-headers-below %)$ pip -V
pip 1.4.1 from /Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages (python 2.6)
(socorro)peterbe@mpb:~/dev/MOZILLA/SOCORRO/socorro/webapp-django (bug-912011-colors-in-front-page-graph-dont-match-headers-below %)$ peep install -r requirements/dev.txt
Traceback (most recent call last):
  File "/Users/peterbe/virtualenvs/socorro/bin/peep", line 8, in <module>
    load_entry_point('peep==0.4', 'console_scripts', 'peep')()
  File "/Users/peterbe/virtualenvs/socorro/lib/python2.6/site-packages/peep/__init__.py", line 218, in main
    path in req_paths)))
  File "/usr/local/Cellar/python2.6/2.6.6/lib/python2.6/site-packages/pip-1.1-py2.6.egg/pip/req.py", line 1240, in parse_requirements
    skip_regex = options.skip_requirements_regex
AttributeError: 'NoneType' object has no attribute 'skip_requirements_regex'
(socorro)peterbe@mpb:~/dev/MOZILLA/SOCORRO/socorro/webapp-django (bug-912011-colors-in-front-page-graph-dont-match-headers-below %)$ deactivate
peterbe@mpb:~/dev/MOZILLA/SOCORRO/socorro/webapp-django (bug-912011-colors-in-front-page-graph-dont-match-headers-below %)$ pip -V
pip 1.4.1 from /Library/Python/2.7/site-packages (python 2.7)

Consider console_scripts entrypoint rather than scripts= in setup.py

Using something like this…

entry_points={
    'console_scripts': ['shiva = shiva_deployer.commandline:main']
    },

…instead of scripts= should make us more compatible with Windows, which doesn't do shebang lines.

Allow peep to work with pip-accel

https://github.com/paylogic/pip-accel

Not sure if this best belongs in this issue queue or theirs, so let me know :)

cc: @xolox @paylogic

Peep installing commits from Github fails

Peep doesn't seem to work well when installing from a commit on GH.

$ peep install -r requirements/dev.txt
Downloading/unpacking configman from git+git://github.com/mozilla/configman@c2179c80be74b1ecb4a6f9c837993d2ce8d4926b#egg=configman (from -r requirements/prod.txt (line 2))
  Cloning git://github.com/mozilla/configman (to c2179c80be74b1ecb4a6f9c837993d2ce8d4926b) to /var/folders/mg/yfz5pk0x66d57hvh1pxg_49w0000gn/T/pip-vl8U3Y-export
  Could not find a tag or branch 'c2179c80be74b1ecb4a6f9c837993d2ce8d4926b', assuming commit.
  Running setup.py egg_info for package configman
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
    IOError: [Errno 2] No such file or directory: '/var/folders/mg/yfz5pk0x66d57hvh1pxg_49w0000gn/T/pip-build/configman/setup.py'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

IOError: [Errno 2] No such file or directory: '/var/folders/mg/yfz5pk0x66d57hvh1pxg_49w0000gn/T/pip-build/configman/setup.py'

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /var/folders/mg/yfz5pk0x66d57hvh1pxg_49w0000gn/T/pip-build/configman
Storing complete log in /var/folders/mg/yfz5pk0x66d57hvh1pxg_49w0000gn/T/tmp4B5xFH

dependencies being downloaded anyways?

While testing out peep-0.8, I tried installing a package from PyPI (PyNaCl-0.2.3), via a requirements.txt that contained the hash of the pynacl tarball. I noticed that peep install was downloading and installing several packages that weren't listed in the requirements.txt (including cffi and pycparser). Is peep missing something, or is PyNaCl doing something too weird for peep to handle?

% cat reqs.txt
# sha256: N8MuTzCIbny5_KU3v6yvkqathrS_masfPazzMOHsv5Y
PyNaCl==0.2.3
% ./bin/peep install -r reqs.txt
Downloading/unpacking PyNaCl==0.2.3
Saved /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/peep-E6EIhS/PyNaCl-0.2.3.tar.gz
Running setup.py egg_info for package PyNaCl
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  clang: warning: argument unused during compilation: '-mno-fused-madd'
  In file included from c/_cffi_backend.c:210:
  c/misc_thread.h:22:13: warning: implicit conversion loses integer precision: 'intptr_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      errno = value;
            ~ ^~~~~
  c/_cffi_backend.c:248:25: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      int base_name_len = strlen(ct_base->ct_name);
          ~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:249:26: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      int extra_name_len = strlen(extra_text);
          ~~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:858:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = read_raw_signed_data(data, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:867:50: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = read_raw_unsigned_data(data, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:877:58: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
              double value = read_raw_float_data(data, ct->ct_size);
                             ~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:913:71: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = (unsigned PY_LONG_LONG)read_raw_signed_data(data, ct->ct_size);
                                         ~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:927:50: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = read_raw_unsigned_data(data, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:1150:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(buf, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~             ~~~~^~~~~~~
  c/_cffi_backend.c:1151:52: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          if (value != read_raw_signed_data(buf, ct->ct_size))
                       ~~~~~~~~~~~~~~~~~~~~      ~~~~^~~~~~~
  c/_cffi_backend.c:1153:49: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(data, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~              ~~~~^~~~~~~
  c/_cffi_backend.c:1163:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(buf, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~             ~~~~^~~~~~~
  c/_cffi_backend.c:1164:54: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          if (value != read_raw_unsigned_data(buf, ct->ct_size))
                       ~~~~~~~~~~~~~~~~~~~~~~      ~~~~^~~~~~~
  c/_cffi_backend.c:1166:49: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(data, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~              ~~~~^~~~~~~
  c/_cffi_backend.c:1185:51: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
              write_raw_float_data(data, value, ct->ct_size);
              ~~~~~~~~~~~~~~~~~~~~              ~~~~^~~~~~~
  c/_cffi_backend.c:1329:53: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      rawfielddata = read_raw_unsigned_data(data, ct->ct_size);
                     ~~~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:1331:52: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      write_raw_integer_data(data, rawfielddata, ct->ct_size);
      ~~~~~~~~~~~~~~~~~~~~~~                     ~~~~^~~~~~~
  c/_cffi_backend.c:1351:21: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          align = ct->ct_length;
                ~ ~~~~^~~~~~~~~
  c/_cffi_backend.c:1604:68: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = (long)read_raw_signed_data(cd->c_data, cd->c_type->ct_size);
                        ~~~~~~~~~~~~~~~~~~~~             ~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:1653:65: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
              value = read_raw_float_data(cd->c_data, cd->c_type->ct_size);
                      ~~~~~~~~~~~~~~~~~~~             ~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:2269:16: warning: implicit conversion loses integer precision: 'long' to 'ffi_abi' (aka 'enum ffi_abi') [-Wshorten-64-to-32]
          fabi = PyInt_AS_LONG(PyTuple_GET_ITEM(signature, 0));
               ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/intobject.h:51:51: note: expanded from macro 'PyInt_AS_LONG'
  #define PyInt_AS_LONG(op) (((PyIntObject *)(op))->ob_ival)
                             ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:2768:56: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
                                             cd->c_type->ct_size) != 0.0;
                                             ~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:2862:55: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(cd->c_data, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~                    ~~~~^~~~~~~
  c/_cffi_backend.c:2965:61: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
                  write_raw_float_data(cd->c_data, value, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~                    ~~~~^~~~~~~
  c/_cffi_backend.c:3286:38: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
      name_size = strlen(ptypes->name) + 1;
                ~ ~~~~~~~~~~~~~~~~~~~~~^~~
  c/_cffi_backend.c:3304:28: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      td->ct_name_position = strlen(td->ct_name);
                           ~ ^~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:3423:19: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      int namelen = strlen(name);
          ~~~~~~~   ^~~~~~~~~~~~
  c/_cffi_backend.c:3706:53: warning: implicit conversion loses integer precision: 'long' to 'int' [-Wshorten-64-to-32]
                      bits_already_occupied = boffset - (field_offset_bytes * 8);
                                            ~ ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:3739:53: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
                          prev_bitfield_size = ftype->ct_size;
                                             ~ ~~~~~~~^~~~~~~
  c/_cffi_backend.c:4023:37: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          fb->fct->ct_name_position = i;
                                    ~ ^
  c/_cffi_backend.c:4038:40: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
          fb_cat_name(fb, farg->ct_name, strlen(farg->ct_name));
          ~~~~~~~~~~~                    ^~~~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:4051:70: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
                  strlen(fresult->ct_name) - fresult->ct_name_position + 1);
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
  c/_cffi_backend.c:4071:30: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      fct = ctypedescr_new(fb->nb_bytes);
            ~~~~~~~~~~~~~~ ~~~~^~~~~~~~
  c/_cffi_backend.c:4121:56: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'unsigned int' [-Wshorten-64-to-32]
      if (ffi_prep_cif(&cif_descr->cif, fabi, funcbuffer.nargs,
          ~~~~~~~~~~~~                        ~~~~~~~~~~~^~~~~
  c/_cffi_backend.c:4517:31: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
      name_size = strlen(ename) + 1;
                ~ ~~~~~~~~~~~~~~^~~
  35 warnings generated.

  Installed /private/tmp/t/build/PyNaCl/cffi-0.7.2-py2.7-macosx-10.8-intel.egg
  Searching for pycparser
  Reading http://pypi.python.org/simple/pycparser/
  Best match: pycparser 2.10
  Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f
  Processing pycparser-2.10.tar.gz
  Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/easy_install-zfYHNm/pycparser-2.10/egg-dist-tmp-PA9odl
  zip_safe flag not set; analyzing archive contents...

  Installed /private/tmp/t/build/PyNaCl/pycparser-2.10-py2.7.egg

  warning: no previously-included files matching '__pycache__/*' found anywhere in distribution
Successfully downloaded PyNaCl
Cleaning up...
Unpacking /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/peep-E6EIhS/PyNaCl-0.2.3.tar.gz
Running setup.py egg_info for package from file:///var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/peep-E6EIhS/PyNaCl-0.2.3.tar.gz
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  clang: warning: argument unused during compilation: '-mno-fused-madd'
  In file included from c/_cffi_backend.c:210:
  c/misc_thread.h:22:13: warning: implicit conversion loses integer precision: 'intptr_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      errno = value;
            ~ ^~~~~
  c/_cffi_backend.c:248:25: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      int base_name_len = strlen(ct_base->ct_name);
          ~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:249:26: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      int extra_name_len = strlen(extra_text);
          ~~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:858:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = read_raw_signed_data(data, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:867:50: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = read_raw_unsigned_data(data, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:877:58: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
              double value = read_raw_float_data(data, ct->ct_size);
                             ~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:913:71: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = (unsigned PY_LONG_LONG)read_raw_signed_data(data, ct->ct_size);
                                         ~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:927:50: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = read_raw_unsigned_data(data, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:1150:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(buf, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~             ~~~~^~~~~~~
  c/_cffi_backend.c:1151:52: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          if (value != read_raw_signed_data(buf, ct->ct_size))
                       ~~~~~~~~~~~~~~~~~~~~      ~~~~^~~~~~~
  c/_cffi_backend.c:1153:49: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(data, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~              ~~~~^~~~~~~
  c/_cffi_backend.c:1163:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(buf, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~             ~~~~^~~~~~~
  c/_cffi_backend.c:1164:54: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          if (value != read_raw_unsigned_data(buf, ct->ct_size))
                       ~~~~~~~~~~~~~~~~~~~~~~      ~~~~^~~~~~~
  c/_cffi_backend.c:1166:49: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(data, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~              ~~~~^~~~~~~
  c/_cffi_backend.c:1185:51: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
              write_raw_float_data(data, value, ct->ct_size);
              ~~~~~~~~~~~~~~~~~~~~              ~~~~^~~~~~~
  c/_cffi_backend.c:1329:53: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      rawfielddata = read_raw_unsigned_data(data, ct->ct_size);
                     ~~~~~~~~~~~~~~~~~~~~~~       ~~~~^~~~~~~
  c/_cffi_backend.c:1331:52: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      write_raw_integer_data(data, rawfielddata, ct->ct_size);
      ~~~~~~~~~~~~~~~~~~~~~~                     ~~~~^~~~~~~
  c/_cffi_backend.c:1351:21: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          align = ct->ct_length;
                ~ ~~~~^~~~~~~~~
  c/_cffi_backend.c:1604:68: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          value = (long)read_raw_signed_data(cd->c_data, cd->c_type->ct_size);
                        ~~~~~~~~~~~~~~~~~~~~             ~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:1653:65: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
              value = read_raw_float_data(cd->c_data, cd->c_type->ct_size);
                      ~~~~~~~~~~~~~~~~~~~             ~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:2269:16: warning: implicit conversion loses integer precision: 'long' to 'ffi_abi' (aka 'enum ffi_abi') [-Wshorten-64-to-32]
          fabi = PyInt_AS_LONG(PyTuple_GET_ITEM(signature, 0));
               ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/intobject.h:51:51: note: expanded from macro 'PyInt_AS_LONG'
  #define PyInt_AS_LONG(op) (((PyIntObject *)(op))->ob_ival)
                             ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:2768:56: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
                                             cd->c_type->ct_size) != 0.0;
                                             ~~~~~~~~~~~~^~~~~~~
  c/_cffi_backend.c:2862:55: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          write_raw_integer_data(cd->c_data, value, ct->ct_size);
          ~~~~~~~~~~~~~~~~~~~~~~                    ~~~~^~~~~~~
  c/_cffi_backend.c:2965:61: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
                  write_raw_float_data(cd->c_data, value, ct->ct_size);
                  ~~~~~~~~~~~~~~~~~~~~                    ~~~~^~~~~~~
  c/_cffi_backend.c:3286:38: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
      name_size = strlen(ptypes->name) + 1;
                ~ ~~~~~~~~~~~~~~~~~~~~~^~~
  c/_cffi_backend.c:3304:28: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      td->ct_name_position = strlen(td->ct_name);
                           ~ ^~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:3423:19: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
      int namelen = strlen(name);
          ~~~~~~~   ^~~~~~~~~~~~
  c/_cffi_backend.c:3706:53: warning: implicit conversion loses integer precision: 'long' to 'int' [-Wshorten-64-to-32]
                      bits_already_occupied = boffset - (field_offset_bytes * 8);
                                            ~ ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:3739:53: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
                          prev_bitfield_size = ftype->ct_size;
                                             ~ ~~~~~~~^~~~~~~
  c/_cffi_backend.c:4023:37: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
          fb->fct->ct_name_position = i;
                                    ~ ^
  c/_cffi_backend.c:4038:40: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
          fb_cat_name(fb, farg->ct_name, strlen(farg->ct_name));
          ~~~~~~~~~~~                    ^~~~~~~~~~~~~~~~~~~~~
  c/_cffi_backend.c:4051:70: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
                  strlen(fresult->ct_name) - fresult->ct_name_position + 1);
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
  c/_cffi_backend.c:4071:30: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      fct = ctypedescr_new(fb->nb_bytes);
            ~~~~~~~~~~~~~~ ~~~~^~~~~~~~
  c/_cffi_backend.c:4121:56: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'unsigned int' [-Wshorten-64-to-32]
      if (ffi_prep_cif(&cif_descr->cif, fabi, funcbuffer.nargs,
          ~~~~~~~~~~~~                        ~~~~~~~~~~~^~~~~
  c/_cffi_backend.c:4517:31: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
      name_size = strlen(ename) + 1;
                ~ ~~~~~~~~~~~~~~^~~
  35 warnings generated.

  Installed /private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/cffi-0.7.2-py2.7-macosx-10.8-intel.egg
  Searching for pycparser
  Reading http://pypi.python.org/simple/pycparser/
  Best match: pycparser 2.10
  Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f
  Processing pycparser-2.10.tar.gz
  Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/easy_install-wzr9LK/pycparser-2.10/egg-dist-tmp-Fph2HE
  zip_safe flag not set; analyzing archive contents...

  Installed /private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/pycparser-2.10-py2.7.egg

  warning: no previously-included files matching '__pycache__/*' found anywhere in distribution
Installing collected packages: PyNaCl
Running setup.py install for PyNaCl
  checking build system type... x86_64-apple-darwin12.5.0
  checking host system type... x86_64-apple-darwin12.5.0
  checking for a BSD-compatible install... /usr/bin/install -c
  checking whether build environment is sane... yes
  checking for a thread-safe mkdir -p... /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d
  checking for gawk... no
  checking for mawk... no
  checking for nawk... no
  checking for awk... awk
  checking whether make sets $(MAKE)... yes
  checking whether make supports nested variables... yes
  checking whether UID '501' is supported by ustar format... yes
  checking whether GID '20' is supported by ustar format... yes
  checking how to create a ustar tar archive... gnutar
  checking whether make supports nested variables... (cached) yes
  checking whether to enable maintainer-specific portions of Makefiles... no
  checking for pkg-config... /usr/local/bin/pkg-config
  checking pkg-config is at least version 0.25... yes
  checking for style of include used by make... GNU
  checking for gcc... gcc
  checking whether the C compiler works... yes
  checking for C compiler default output file name... a.out
  checking for suffix of executables...
  checking whether we are cross compiling... no
  checking for suffix of object files... o
  checking whether we are using the GNU C compiler... yes
  checking whether gcc accepts -g... yes
  checking for gcc option to accept ISO C89... none needed
  checking whether gcc understands -c and -o together... yes
  checking dependency style of gcc... none
  checking for gcc option to accept ISO C99... -std=gnu99
  checking dependency style of gcc -std=gnu99... none
  checking how to run the C preprocessor... gcc -std=gnu99 -E
  checking for grep that handles long lines and -e... /usr/bin/grep
  checking for egrep... /usr/bin/grep -E
  checking for ANSI C header files... yes
  checking for sys/types.h... yes
  checking for sys/stat.h... yes
  checking for stdlib.h... yes
  checking for string.h... yes
  checking for memory.h... yes
  checking for strings.h... yes
  checking for inttypes.h... yes
  checking for stdint.h... yes
  checking for unistd.h... yes
  checking minix/config.h usability... no
  checking minix/config.h presence... no
  checking for minix/config.h... no
  checking whether it is safe to define __EXTENSIONS__... yes
  checking whether C compiler accepts -fvisibility=hidden... yes
  checking whether C compiler accepts -fPIC... yes
  checking whether the linker accepts -fPIC... yes
  checking whether C compiler accepts -fPIE... yes
  checking whether the linker accepts -fPIE... yes
  checking whether the linker accepts -pie... yes
  checking whether C compiler accepts -fwrapv... yes
  checking whether C compiler accepts -fno-strict-aliasing... yes
  checking whether C compiler accepts -fno-strict-overflow... yes
  checking whether C compiler accepts -fstack-protector-all... yes
  checking whether the linker accepts -fstack-protector-all... yes
  checking whether C compiler accepts -Winit-self... yes
  checking whether C compiler accepts -Wwrite-strings... yes
  checking whether C compiler accepts -Wdiv-by-zero... yes
  checking whether C compiler accepts -Wsometimes-uninitialized... no
  checking whether C compiler accepts  -Wall... yes
  checking whether C compiler accepts  -Wall -Wextra... yes
  checking for clang... no
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init -Wparentheses... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init -Wparentheses -Wpointer-arith... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init -Wparentheses -Wpointer-arith -Wredundant-decls... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init -Wparentheses -Wpointer-arith -Wredundant-decls -Wstrict-prototypes... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init -Wparentheses -Wpointer-arith -Wredundant-decls -Wstrict-prototypes -Wswitch-enum... yes
  checking whether C compiler accepts  -Wall -Wextra -Wbad-function-cast -Wcast-align -Wcast-qual -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat=2 -Wimplicit -Wmissing-declarations -Wmissing-prototypes -Wnormalized=id -Woverride-init -Wparentheses -Wpointer-arith -Wredundant-decls -Wstrict-prototypes -Wswitch-enum -Wvariable-decl... no
  checking whether the linker accepts -Wl,-z,relro... no
  checking whether the linker accepts -Wl,-z,now... no
  checking whether the linker accepts -Wl,-z,noexecstack... no
  checking how to print strings... printf
  checking for a sed that does not truncate output... /usr/bin/sed
  checking for fgrep... /usr/bin/grep -F
  checking for ld used by gcc -std=gnu99... /usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld
  checking if the linker (/usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld) is GNU ld... no
  checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm
  checking the name lister (/usr/bin/nm) interface... BSD nm
  checking whether ln -s works... yes
  checking the maximum length of command line arguments... 196608
  checking whether the shell understands some XSI constructs... yes
  checking whether the shell understands "+="... yes
  checking how to convert x86_64-apple-darwin12.5.0 file names to x86_64-apple-darwin12.5.0 format... func_convert_file_noop
  checking how to convert x86_64-apple-darwin12.5.0 file names to toolchain format... func_convert_file_noop
  checking for /usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld option to reload object files... -r
  checking for objdump... no
  checking how to recognize dependent libraries... pass_all
  checking for dlltool... no
  checking how to associate runtime and link libraries... printf %s\n
  checking for ar... ar
  checking for archiver @FILE support... no
  checking for strip... strip
  checking for ranlib... ranlib
  checking command to parse /usr/bin/nm output from gcc -std=gnu99 object... ok
  checking for sysroot... no
  checking for mt... no
  checking if : is a manifest tool... no
  checking for dsymutil... dsymutil
  checking for nmedit... nmedit
  checking for lipo... lipo
  checking for otool... otool
  checking for otool64... no
  checking for -single_module linker flag... yes
  checking for -exported_symbols_list linker flag... yes
  checking for -force_load linker flag... yes
  checking for dlfcn.h... yes
  checking for objdir... .libs
  checking if gcc -std=gnu99 supports -fno-rtti -fno-exceptions... no
  checking for gcc -std=gnu99 option to produce PIC... -fno-common -DPIC
  checking if gcc -std=gnu99 PIC flag -fno-common -DPIC works... yes
  checking if gcc -std=gnu99 static flag -static works... no
  checking if gcc -std=gnu99 supports -c -o file.o... yes
  checking if gcc -std=gnu99 supports -c -o file.o... (cached) yes
  checking whether the gcc -std=gnu99 linker (/usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld) supports shared libraries... yes
  checking dynamic linker characteristics... darwin12.5.0 dyld
  checking how to hardcode library paths into programs... immediate
  checking whether stripping libraries is possible... yes
  checking if libtool supports shared libraries... yes
  checking whether to build shared libraries... no
  checking whether to build static libraries... yes
  checking for ar... (cached) ar
  checking for emmintrin.h... yes
  checking for tmmintrin.h... yes
  checking for smmintrin.h... no
  checking for immintrin.h... no
  checking for avxintrin.h... no
  checking for x86intrin.h... no
  checking for wmmintrin.h... no
  checking for access to floating-point rounding mode... yes
  checking for inline... inline
  checking whether byte ordering is bigendian... no
  checking whether __STDC_LIMIT_MACROS is required... no
  checking whether we can assemble basic amd64 code... yes
  checking for 128-bit arithmetic... yes
  checking for cpuid instruction... yes
  configure: data alignment is not required on this target
  checking for clock_gettime... no
  checking for clock_gettime in -lrt... no
  checking for fegetenv... yes
  checking for SecureZeroMemory... no
  checking for arc4random... yes
  checking for arc4random_buf... yes
  checking if gcc/ld supports -Wl,--output-def... not needed, shared libraries are disabled
  checking that generated files are newer than configure... done
  configure: creating ./config.status
  config.status: creating libsodium.pc
  config.status: creating Makefile
  config.status: creating msvc-scripts/Makefile
  config.status: creating src/Makefile
  config.status: creating src/libsodium/Makefile
  config.status: creating src/libsodium/include/Makefile
  config.status: creating src/libsodium/include/sodium/version.h
  config.status: creating src/libsodium/include/sodium/crypto_scalarmult_curve25519.h
  config.status: creating src/libsodium/include/sodium/crypto_stream_salsa20.h
  config.status: creating test/default/Makefile
  config.status: creating test/Makefile
  config.status: executing depfiles commands
  config.status: executing libtool commands
  Making all in msvc-scripts
  make[1]: Nothing to be done for `all'.
  Making all in src
  Making all in libsodium
  Making all in include
  make[3]: Nothing to be done for `all'.
    CC       crypto_auth/libsodium_la-crypto_auth.lo
    CC       crypto_auth/hmacsha256/libsodium_la-auth_hmacsha256_api.lo
    CC       crypto_auth/hmacsha256/ref/libsodium_la-hmac_hmacsha256.lo
    CC       crypto_auth/hmacsha256/ref/libsodium_la-verify_hmacsha256.lo
    CC       crypto_auth/hmacsha512256/libsodium_la-auth_hmacsha512256_api.lo
    CC       crypto_auth/hmacsha512256/ref/libsodium_la-hmac_hmacsha512256.lo
    CC       crypto_auth/hmacsha512256/ref/libsodium_la-verify_hmacsha512256.lo
    CC       crypto_box/libsodium_la-crypto_box.lo
    CC       crypto_box/curve25519xsalsa20poly1305/libsodium_la-box_curve25519xsalsa20poly1305_api.lo
    CC       crypto_box/curve25519xsalsa20poly1305/ref/libsodium_la-after_curve25519xsalsa20poly1305.lo
    CC       crypto_box/curve25519xsalsa20poly1305/ref/libsodium_la-before_curve25519xsalsa20poly1305.lo
    CC       crypto_box/curve25519xsalsa20poly1305/ref/libsodium_la-box_curve25519xsalsa20poly1305.lo
    CC       crypto_box/curve25519xsalsa20poly1305/ref/libsodium_la-keypair_curve25519xsalsa20poly1305.lo
    CC       crypto_core/hsalsa20/ref2/libsodium_la-core_hsalsa20.lo
    CC       crypto_core/hsalsa20/libsodium_la-core_hsalsa20_api.lo
    CC       crypto_core/salsa20/ref/libsodium_la-core_salsa20.lo
    CC       crypto_core/salsa20/libsodium_la-core_salsa20_api.lo
    CC       crypto_core/salsa2012/ref/libsodium_la-core_salsa2012.lo
    CC       crypto_core/salsa2012/libsodium_la-core_salsa2012_api.lo
    CC       crypto_core/salsa208/ref/libsodium_la-core_salsa208.lo
    CC       crypto_core/salsa208/libsodium_la-core_salsa208_api.lo
    CC       crypto_generichash/libsodium_la-crypto_generichash.lo
    CC       crypto_generichash/blake2/libsodium_la-generichash_blake2_api.lo
    CC       crypto_generichash/blake2/ref/libsodium_la-blake2b-ref.lo
    CC       crypto_generichash/blake2/ref/libsodium_la-generichash_blake2b.lo
    CC       crypto_hash/libsodium_la-crypto_hash.lo
    CC       crypto_hash/sha256/libsodium_la-hash_sha256_api.lo
    CC       crypto_hash/sha256/ref/libsodium_la-hash_sha256.lo
    CC       crypto_hash/sha512/libsodium_la-hash_sha512_api.lo
    CC       crypto_hash/sha512/ref/libsodium_la-hash_sha512.lo
    CC       crypto_hashblocks/sha256/ref/libsodium_la-blocks_sha256.lo
    CC       crypto_hashblocks/sha256/libsodium_la-hashblocks_sha256_api.lo
    CC       crypto_hashblocks/sha512/ref/libsodium_la-blocks_sha512.lo
    CC       crypto_hashblocks/sha512/libsodium_la-hashblocks_sha512_api.lo
    CC       crypto_onetimeauth/libsodium_la-crypto_onetimeauth.lo
    CC       crypto_onetimeauth/poly1305/libsodium_la-onetimeauth_poly1305.lo
    CC       crypto_onetimeauth/poly1305/libsodium_la-onetimeauth_poly1305_api.lo
    CC       crypto_onetimeauth/poly1305/libsodium_la-onetimeauth_poly1305_try.lo
    CC       crypto_onetimeauth/poly1305/53/libsodium_la-auth_poly1305_53.lo
    CC       crypto_onetimeauth/poly1305/53/libsodium_la-verify_poly1305_53.lo
    CC       crypto_onetimeauth/poly1305/donna/libsodium_la-auth_poly1305_donna.lo
    CC       crypto_onetimeauth/poly1305/donna/libsodium_la-verify_poly1305_donna.lo
    CC       crypto_scalarmult/libsodium_la-crypto_scalarmult.lo
    CC       crypto_scalarmult/curve25519/libsodium_la-scalarmult_curve25519_api.lo
    CC       crypto_secretbox/libsodium_la-crypto_secretbox.lo
    CC       crypto_secretbox/xsalsa20poly1305/libsodium_la-secretbox_xsalsa20poly1305_api.lo
    CC       crypto_secretbox/xsalsa20poly1305/ref/libsodium_la-box_xsalsa20poly1305.lo
    CC       crypto_shorthash/libsodium_la-crypto_shorthash.lo
    CC       crypto_shorthash/siphash24/libsodium_la-shorthash_siphash24_api.lo
    CC       crypto_shorthash/siphash24/ref/libsodium_la-shorthash_siphash24.lo
    CC       crypto_sign/libsodium_la-crypto_sign.lo
    CC       crypto_sign/ed25519/libsodium_la-sign_ed25519_api.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_0.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_1.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_add.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_cmov.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_copy.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_frombytes.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_invert.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_isnegative.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_isnonzero.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_mul.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_neg.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_pow22523.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_sq.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_sq2.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_sub.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-fe_tobytes.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_add.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_double_scalarmult.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_frombytes.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_madd.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_msub.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p1p1_to_p2.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p1p1_to_p3.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p2_0.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p2_dbl.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p3_0.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p3_dbl.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p3_to_cached.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p3_to_p2.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_p3_tobytes.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_precomp_0.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_scalarmult_base.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_sub.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-ge_tobytes.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-keypair.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-open.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-sc_muladd.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-sc_reduce.lo
    CC       crypto_sign/ed25519/ref10/libsodium_la-sign.lo
    CC       crypto_sign/edwards25519sha512batch/libsodium_la-sign_edwards25519sha512batch_api.lo
    CC       crypto_sign/edwards25519sha512batch/ref/libsodium_la-fe25519_edwards25519sha512batch.lo
    CC       crypto_sign/edwards25519sha512batch/ref/libsodium_la-ge25519_edwards25519sha512batch.lo
    CC       crypto_sign/edwards25519sha512batch/ref/libsodium_la-sc25519_edwards25519sha512batch.lo
    CC       crypto_sign/edwards25519sha512batch/ref/libsodium_la-sign_edwards25519sha512batch.lo
    CC       crypto_stream/libsodium_la-crypto_stream.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-afternm_aes128ctr.lo
    CC       crypto_stream/aes128ctr/libsodium_la-stream_aes128ctr_api.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-beforenm_aes128ctr.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-common_aes128ctr.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-consts_aes128ctr.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-int128_aes128ctr.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-stream_aes128ctr.lo
    CC       crypto_stream/aes128ctr/portable/libsodium_la-xor_afternm_aes128ctr.lo
    CC       crypto_stream/aes256estream/hongjun/libsodium_la-aes256-ctr.lo
    CC       crypto_stream/aes256estream/libsodium_la-stream_aes256estream_api.lo
    CC       crypto_stream/salsa20/libsodium_la-stream_salsa20_api.lo
    CC       crypto_stream/salsa2012/libsodium_la-stream_salsa2012_api.lo
    CC       crypto_stream/salsa2012/ref/libsodium_la-stream_salsa2012.lo
    CC       crypto_stream/salsa2012/ref/libsodium_la-xor_salsa2012.lo
    CC       crypto_stream/salsa208/libsodium_la-stream_salsa208_api.lo
    CC       crypto_stream/salsa208/ref/libsodium_la-stream_salsa208.lo
    CC       crypto_stream/salsa208/ref/libsodium_la-xor_salsa208.lo
    CC       crypto_stream/xsalsa20/libsodium_la-stream_xsalsa20_api.lo
    CC       crypto_stream/xsalsa20/ref/libsodium_la-stream_xsalsa20.lo
    CC       crypto_stream/xsalsa20/ref/libsodium_la-xor_xsalsa20.lo
    CC       crypto_verify/16/libsodium_la-verify_16_api.lo
    CC       crypto_verify/16/ref/libsodium_la-verify_16.lo
    CC       crypto_verify/32/libsodium_la-verify_32_api.lo
    CC       crypto_verify/32/ref/libsodium_la-verify_32.lo
    CC       randombytes/libsodium_la-randombytes.lo
    CC       randombytes/salsa20/libsodium_la-randombytes_salsa20_random.lo
    CC       randombytes/sysrandom/libsodium_la-randombytes_sysrandom.lo
    CC       sodium/libsodium_la-compat.lo
    CC       sodium/libsodium_la-core.lo
    CC       sodium/libsodium_la-utils.lo
    CC       sodium/libsodium_la-version.lo
    CC       crypto_scalarmult/curve25519/donna_c64/libsodium_la-base_curve25519_donna_c64.lo
    CC       crypto_scalarmult/curve25519/donna_c64/libsodium_la-smult_curve25519_donna_c64.lo
    CPPAS    crypto_stream/salsa20/amd64_xmm6/libsodium_la-stream_salsa20_amd64_xmm6.lo
    CCLD     libsodium.la
  make[2]: Nothing to be done for `all-am'.
  Making all in test
  Making all in default
  make[2]: Nothing to be done for `all'.
  make[2]: Nothing to be done for `all-am'.
  make[1]: Nothing to be done for `all-am'.
  Making check in msvc-scripts
  make[1]: Nothing to be done for `check'.
  Making check in src
  Making check in libsodium
  Making check in include
  make[3]: Nothing to be done for `check'.
  make[3]: Nothing to be done for `check-am'.
  make[2]: Nothing to be done for `check-am'.
  Making check in test
  Making check in default
  make  auth auth2 auth3 auth5 box box2 box7 box8 core1 core2 core3 core4 core5 core6 generichash generichash2 hash hash3 onetimeauth onetimeauth2 onetimeauth7 randombytes scalarmult scalarmult2 scalarmult5 scalarmult6 scalarmult7 scalarmult8 secretbox secretbox2 secretbox7 secretbox8 shorthash sodium_core sodium_utils sodium_version stream stream2 stream3 stream4 stream5 stream6
    CC       auth.o
    CCLD     auth
    CC       auth2.o
    CCLD     auth2
    CC       auth3.o
    CCLD     auth3
    CC       auth5.o
    CCLD     auth5
    CC       box.o
    CCLD     box
    CC       box2.o
    CCLD     box2
    CC       box7.o
    CCLD     box7
    CC       box8.o
    CCLD     box8
    CC       core1.o
    CCLD     core1
    CC       core2.o
    CCLD     core2
    CC       core3.o
    CCLD     core3
    CC       core4.o
    CCLD     core4
    CC       core5.o
    CCLD     core5
    CC       core6.o
    CCLD     core6
    CC       generichash.o
    CCLD     generichash
    CC       generichash2.o
    CCLD     generichash2
    CC       hash.o
    CCLD     hash
    CC       hash3.o
    CCLD     hash3
    CC       onetimeauth.o
    CCLD     onetimeauth
    CC       onetimeauth2.o
    CCLD     onetimeauth2
    CC       onetimeauth7.o
    CCLD     onetimeauth7
    CC       randombytes.o
    CCLD     randombytes
    CC       scalarmult.o
    CCLD     scalarmult
    CC       scalarmult2.o
    CCLD     scalarmult2
    CC       scalarmult5.o
    CCLD     scalarmult5
    CC       scalarmult6.o
    CCLD     scalarmult6
    CC       scalarmult7.o
    CCLD     scalarmult7
    CC       scalarmult8.o
    CCLD     scalarmult8
    CC       secretbox.o
    CCLD     secretbox
    CC       secretbox2.o
    CCLD     secretbox2
    CC       secretbox7.o
    CCLD     secretbox7
    CC       secretbox8.o
    CCLD     secretbox8
    CC       shorthash.o
    CCLD     shorthash
    CC       sodium_core.o
    CCLD     sodium_core
    CC       sodium_utils.o
    CCLD     sodium_utils
    CC       sodium_version.o
    CCLD     sodium_version
    CC       stream.o
    CCLD     stream
    CC       stream2.o
    CCLD     stream2
    CC       stream3.o
    CCLD     stream3
    CC       stream4.o
    CCLD     stream4
    CC       stream5.o
    CCLD     stream5
    CC       stream6.o
    CCLD     stream6
  make  check-TESTS
  PASS: auth
  PASS: auth2
  PASS: auth3
  PASS: auth5
  PASS: box
  PASS: box2
  PASS: box7
  PASS: box8
  PASS: core1
  PASS: core2
  PASS: core3
  PASS: core4
  PASS: core5
  PASS: core6
  PASS: generichash
  PASS: generichash2
  PASS: hash
  PASS: hash3
  PASS: onetimeauth
  PASS: onetimeauth2
  PASS: onetimeauth7
  PASS: randombytes
  PASS: scalarmult
  PASS: scalarmult2
  PASS: scalarmult5
  PASS: scalarmult6
  PASS: scalarmult7
  PASS: scalarmult8
  PASS: secretbox
  PASS: secretbox2
  PASS: secretbox7
  PASS: secretbox8
  PASS: shorthash
  PASS: sodium_core
  PASS: sodium_utils
  PASS: sodium_version
  PASS: stream
  PASS: stream2
  PASS: stream3
  PASS: stream4
  PASS: stream5
  PASS: stream6
  make[5]: Nothing to be done for `all'.
  ============================================================================
  Testsuite summary for libsodium 0.4.5
  ============================================================================
  # TOTAL: 42
  # PASS:  42
  # SKIP:  0
  # XFAIL: 0
  # FAIL:  0
  # XPASS: 0
  # ERROR: 0
  ============================================================================
  make[2]: Nothing to be done for `check-am'.
  make[1]: Nothing to be done for `check-am'.
  Making install in msvc-scripts
  make[2]: Nothing to be done for `install-exec-am'.
  make[2]: Nothing to be done for `install-data-am'.
  Making install in src
  Making install in libsodium
  Making install in include
  make[4]: Nothing to be done for `install-exec-am'.
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include'
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/sodium'
   /usr/bin/install -c -m 644  /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/core.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_auth.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_auth_hmacsha256.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_auth_hmacsha512256.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_box.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_box_curve25519xsalsa20poly1305.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_core_hsalsa20.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_core_salsa20.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_core_salsa2012.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_core_salsa208.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_generichash.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_generichash_blake2b.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_hash.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_hash_sha256.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_hash_sha512.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_hashblocks_sha256.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_hashblocks_sha512.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_onetimeauth.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_onetimeauth_poly1305.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_onetimeauth_poly1305_53.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_onetimeauth_poly1305_donna.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_scalarmult.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_secretbox.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_secretbox_xsalsa20poly1305.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_shorthash.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_shorthash_siphash24.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_sign.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_sign_ed25519.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_sign_edwards25519sha512batch.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_stream.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_stream_aes128ctr.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_stream_aes256estream.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_stream_salsa2012.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_stream_salsa208.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_stream_xsalsa20.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_int32.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_int64.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_uint16.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_uint32.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_uint64.h '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/sodium'
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/sodium'
   /usr/bin/install -c -m 644  /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_uint8.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_verify_16.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/crypto_verify_32.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/export.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/randombytes.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/randombytes_salsa20_random.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/randombytes_sysrandom.h /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium/utils.h '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/sodium'
   /usr/bin/install -c -m 644  /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/src/libsodium/include/sodium.h '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/.'
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include'
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/sodium'
   /usr/bin/install -c -m 644  sodium/crypto_scalarmult_curve25519.h sodium/crypto_stream_salsa20.h sodium/version.h '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/include/sodium'
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib'
   /bin/sh ../../libtool   --mode=install /usr/bin/install -c   libsodium.la '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib'
  libtool: install: /usr/bin/install -c .libs/libsodium.lai /private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib/libsodium.la
  libtool: install: /usr/bin/install -c .libs/libsodium.a /private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib/libsodium.a
  libtool: install: chmod 644 /private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib/libsodium.a
  libtool: install: ranlib /private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib/libsodium.a
  make[4]: Nothing to be done for `install-data-am'.
  make[3]: Nothing to be done for `install-exec-am'.
  make[3]: Nothing to be done for `install-data-am'.
  Making install in test
  Making install in default
  make[3]: Nothing to be done for `install-exec-am'.
  make[3]: Nothing to be done for `install-data-am'.
  make[3]: Nothing to be done for `install-exec-am'.
  make[3]: Nothing to be done for `install-data-am'.
  make[2]: Nothing to be done for `install-exec-am'.
   /var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/src/libsodium/install-sh -c -d '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib/pkgconfig'
   /usr/bin/install -c -m 644 libsodium.pc '/private/var/folders/4v/j9ylswv946z277d1j6h1gy680000gn/T/pip-Kwyu_F-build/build/temp.macosx-10.8-intel-2.7/lib/pkgconfig'
  building '_cffi__xd64d2119xefb54d7c' extension
  clang -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -Ibuild/temp.macosx-10.8-intel-2.7/include -c src/nacl/_lib/__pycache__/_cffi__xd64d2119xefb54d7c.c -o build/temp.macosx-10.8-intel-2.7/src/nacl/_lib/__pycache__/_cffi__xd64d2119xefb54d7c.o
  clang: warning: argument unused during compilation: '-mno-fused-madd'
  clang -bundle -undefined dynamic_lookup -Wl,-F. -arch i386 -arch x86_64 build/temp.macosx-10.8-intel-2.7/src/nacl/_lib/__pycache__/_cffi__xd64d2119xefb54d7c.o -Lbuild/temp.macosx-10.8-intel-2.7/lib -Lbuild/temp.macosx-10.8-intel-2.7 -lsodium -lsodium -o build/lib.macosx-10.8-intel-2.7/nacl/_lib/_cffi__xd64d2119xefb54d7c.so
  ld: warning: ignoring file build/temp.macosx-10.8-intel-2.7/lib/libsodium.a, file was built for archive which is not the architecture being linked (i386): build/temp.macosx-10.8-intel-2.7/lib/libsodium.a

Successfully installed PyNaCl
Cleaning up...

--version should give peep version info

Right now if you do peep --version, it just gives you pip and python version information. I'd prefer for diagnostic/testing purposes that it told me the following:

1. peep version
2. pip version
3. python version

better output when peep throws an exception

When peep throws an exception, it tells you some things, but it'd be better if it told you a lot more diagnostic output including instructions on how to take that output and write up an issue in the peep issue tracker.

Option to automatically update requirements file

peep should offer to rewrite the requirements file for you. Most people are just going to copy its output in anyway so there's limited benefit from the increased risk of human error.

Offer "missing" output that takes into account multiple archives

PyPI sometimes offers multiple archives formats: for example mock 0.8.0 offers both a zip and a tarball. Pip's choice of download is not always stable. Thus, we added support for multiple hashes for each requirement. However, the discovery of such multi-archive distributions is still an unpleasant manual one: wait for peep to freak out about your hash being wrong. It would be better if, when a requirment has no declared hashes, peep would download all available archives for the chosen version and output all their hashes.

Bonus points are to be given for automatically ensuring that all archives have identical contents, but that ties in with things that are out of scope with this bug, such as improving the vetting UX (handing the user a dir of tarballs to vet, or something).

Always downloads Django 1.6.3

Should be skipping it since it's installed, but it doesn't seem to be able to detect this one for some reason. It's getting Django-1.6.3-py2.py3-none-any.whl. I'll see what I can find out when I get time.

Append peep features directly in pip ?

Have you plan to append the peep features to pip upstream project ?

Report only mode

I would like to run peep in a way that acquires all the tarballs from the requirements file, possibly by downloading them or pulling from a cache, hashes them, and reports if everything matches, but doesn't actually install them. It would be nice if this additionally checked that the requirements tree was complete. This would be a great check to put in a CI job, and would give me confidence that I didn't break something when I changed the requirements file.

PS: It would be ever better (especially for the CI job) if running peep check (or whatever it is called) followed by peep install wouldn't download everything twice somehow.

Support arch-specific binary packages somehow

Pulling down binary packages on different archs will yield different hashes, obviously. We should do something about that. Here are some extremely nebulous sketches:

# sha256: qF4YU3XbdcEJ-Z7N49VUFfA15waKgiUs9PFsZnrDj0k
# sha256: some-alternative-hash-that-is-also-allowed
Jinja2==1.2.3

Of course, that loosens things up a bit. If somebody managed to find a hash collision (hugely unlikely) or, much more likely, if the package for the wrong arch somehow gets downloaded by mistake, it would slip past peep's checks.

# sha256 some-arch: qF4YU3XbdcEJ-Z7N49VUFfA15waKgiUs9PFsZnrDj0k
# sha256: some-other-arch: some-other-hash
Jinja2==1.2.3

This is more self-documenting (Why is this hash here?) but harder to write (What's the syntax, and how do I spell my platform?). Actually, the latter is moot because peep writes requirements files for you.

Does pip install make an attempt to find an install binary versions of packages at all?

Support for "standard" hexadecimal hashes

Given that the many tools output base-16 representations of the hashes, what about adding support for this? The length of the hash could be used to detect the format (base64 vs hex):

# sha256: 1YiBrxeoKyxVnuPvaVEF96AfayYZap-zXLHxUPxPjzw
# sha256: 3f7a8ec45765cc33daac2448c609ab08e76ffb5a
peep==1.3

SECURITY: setup.py executed before checking hash

An important security fix to peep will be released in the next week. Please get ready to upgrade.

Shortly after the new version is released, details of the attack vector will be posted here.

does peep do anything for dependencies of dependencies

Check.py uses the latest versions of pyflakes and pep8. If we have check.py managed by peep, would peep complain when check installs a new version of pyflakes and pep8?

Merge peep functionality into pip

Make pip check against some kind of local hashes.

Put the hashes somewhere proper, not in comments. Either mess with the reqs.txt syntax (and then we'd need a fallback parser in pip—eww) or find some little hole in the grammar to abuse. For example Daniel Holth at one point had the idea of abusing the extras syntax, like requests[ed25519=….]==1.2.0. http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optional-features-with-their-own-dependencies

No output while downloading files

When peep is run, it seems to just sit there for a bit, while it churns away downloading files. It would be nice to at least get a message that says "Downloading files", but even better to print out each package as it is downloaded. Even better than that would be a progress bar, but that might be asking for too much.

Describe CD use case in readme

Mention motivation: continuous deployment I can trust. If it would deploy something unvetted, it stops and screams instead. So I can commit with impunity and trust my deployment script. (Worst case, deployment stops for awhile.)

Give an example pseudocode deployment script, perhaps.

Get peep recognized as conforming to Mozilla deployment policy

A bit of a dump from IRC:

peep saves you/us having to maintain a dir of submodules, deal with updating them, etc. It makes us depend on PyPI being up but verifies we're getting the data from it that we got last time.

I think those are the relevant tradeoffs for this situation. There's more detail at https://pypi.python.org/pypi/peep/.

Someday soon I want to make it work fine without PyPI being accessible (as long as the downloads are cached).

I don't want to say "this is the One True Way" yet because we haven't even got it landed in Socorro's deployment process yet.

But I certainly hope to make it the most convenient, trustworthy way in the near future. :-)

Of the 4 goals under "Purpose", peep takes care of 1, 2, and 4. I'm not sure what 3 means; I'd love somebody to explain it to me.

I'm really happy that whoever wrote the policy wrote out the motivating goals.

Reading a bit further, it sounds like peep meets all the guidelines for "non-distribution centric manager[s]" except "mirror the packages to a Mozilla internal mirror", which is what it's designed to avoid having to do, without sacrificing security. Not really sure where that requirement came from. It doesn't seem to be motivated by a goal from the Purpose section.

solarce: the lack of gpg signing on pypi is something i've heard of as a negative, but you'll want to talk to kang, he seems to be the SME on opsec side for this

ErikRose: RIGHT! That's actually coming with the new "wheel" format, but I don't think it's sufficient. Peep checks the packages against a local hash, so even a compromised upstream key won't harm you. (As now, we are tasked with vetting the package. Then we say "Yep, a package of this hash is good.")

Add support for --download

peep adds its own --download argument when calling pip, and gets all confused if the user provides one too. (The motivating use-case is automatically preparing a directory of known-good files ready for e.g. building a native DEB/RPM package).

travis should run tests in all environment combinations

Travis should run the tests in all the environment combinations. Plus it'd be nice to have a way to make sure tox.ini and .travis.yml are in sync.