myint / yolk

Command-line tool for querying PyPI and Python packages installed on your system (with Python 3 support)

yolk

Contents

• Installation
• Summary
• Useful tips

Installation

$ pip install --upgrade yolk3k

Summary

Yolk is a Python tool for obtaining information about installed Python packages and querying packages available on PyPI (Python Package Index). yolk3k is a fork of the original yolk. yolk3k add Python 3 support (while maintaining Python 2 support). It also adds additional features.

You can see which packages are active, non-active or in development mode and show you which have newer versions available by querying PyPI.

Usage Examples:

$ yolk --upgrade
    Upgrade all packages

$ yolk --upgrade --user
    Upgrade user packages

$ yolk -l
    List all installed Python packages

$ yolk -a
    List only the activated packages installed (Activated packages are
    normal packages on sys.path you can import)

$ yolk -n
    List only the non-activated (--multi-version) packages installed

$ yolk -l -f License,Author nose==1.0
    Show the license and author for version 1.0 of the package `nose`

$ yolk --entry-map nose
    Show entry map for the nose package

$ yolk --entry-points nose.plugins
    Show all setuptools entry points for nose.plugins

These options query PyPI:

$ yolk -U pkg_name
    Shows if an update for pkg_name is available by querying PyPI

$ yolk -U
     Checks PyPI to see if any installed Python packages have updates
     available.

$ yolk -F Paste
    Download source tarball for latest version of Paste to your current
    directory

$ yolk -F Paste -T svn
    Do a subversion checkout for Paste to a directory named Paste_svn in
    your current directory.

$ yolk -L 2
    Show list of CheeseShop releases in the last two hours

$ yolk -C 2
    Show detailed list of changes in the CheeseShop in the last two hours

$ yolk -M Paste==1.0
    Show all the metadata for Paste version 1.0

$ yolk -M Paste
    Show all the metadata for the latest version of Paste listed on PyPI

$ yolk -D cheesecake
    Show all (source, egg, svn) URL's for the latest version of cheesecake
    packages

$ yolk -T source -D cheesecake
    Show only source code releases for cheesecake

$ yolk -H twisted
    Launches your web browser at Twisted's home page

Useful tips

Check invalid reStructuredText in descriptions on PyPI (using rstcheck):

$ yolk --query-metadata=pip --fields=description | rstcheck -

yolk's Issues

Pylama has no license row.

$ yolk -l -f license #yolk3k
pycparser (2.18) 
    License: BSD

pycrypto (2.6.1) 
    License: UNKNOWN

pydocstyle (2.0.0) 
    License: MIT

pyflakes (1.5.0) 
    License: MIT

pylama (7.3.3) 

pytest (3.0.7) 
    License: MIT license

python-dateutil (2.6.1) 
    License: Simplified BSD

python-json-logger (0.1.7) 
    License: BSD

"yolk -V <package>" only shows the latest version

The doc string for the -V option says

show available versions for given package listed on PyPI

However the actual behavior I get is that it only shows the latest available version, not all available versions. For example:

$ yolk -V pandas
pandas 1.2.3

I have yolk3k 0.9 installed.

yolk -S foo raises AttributeError: 'NoneType' object has no attribute 'insert'

➤ yolk -S foo
Traceback (most recent call last):
  File "/home/io/.local/bin/yolk", line 10, in <module>
    sys.exit(main())
  File "/home/io/.local/share/pipx/venvs/yolk1977/lib/python3.7/site-packages/yolk/cli.py", line 1173, in main
    my_yolk.run()
  File "/home/io/.local/share/pipx/venvs/yolk1977/lib/python3.7/site-packages/yolk/cli.py", line 224, in run
    return getattr(self, action)()
  File "/home/io/.local/share/pipx/venvs/yolk1977/lib/python3.7/site-packages/yolk/cli.py", line 801, in pypi_search
    spec.insert(0, search_arg.strip())
AttributeError: 'NoneType' object has no attribute 'insert'

yolk -l pkgname doesn't appear to work

When I run yolk -l pkgname with yolk3k 0.7.2 installed in a virtualenv with Python 2.7.5, I observe the following error:

yolk: error: unrecognized arguments: pkgname

This error did not occur with yolk 0.4.3.

Something strange with yolk and import.

In [1]: from yolk import yolklib

In [2]: list(yolklib.get_distributions('all', 'numpy'))
Out[2]: 
[(numpy 1.11.2 (/usr/local/lib/python2.7/dist-packages), False),
 (numpy 1.12.0 (/usr/local/lib/python2.7/dist-packages), True),
 (numpy 1.12.1 (/usr/local/lib/python2.7/dist-packages), False),
 (numpy 1.8.2 (/usr/lib/python2.7/dist-packages), False)]

In [3]: import numpy

In [4]: numpy.__version__
Out[4]: '1.12.1'

As you can see yolk gets 1.12.0 as active version instead of import, which is load 1.12.1

Why this happens ?

UnboundLocalError: local variable 'dist' referenced before assignment

Happy to provide more environment information upon request. Thanks!

$ yolk --upgrade --user                                                                                                                           
Checking alembic                                                                                                                                  Traceback (most recent call last):
  File "/home/jesse/.local/bin/yolk", line 11, in <module>
    load_entry_point('yolk3k==0.9', 'console_scripts', 'yolk')()
  File "/home/jesse/.local/lib64/python2.7/site-packages/yolk/cli.py", line 1171, in main                                                         
    my_yolk.run()
  File "/home/jesse/.local/lib64/python2.7/site-packages/yolk/cli.py", line 224, in run                                                           
    return getattr(self, action)()
  File "/home/jesse/.local/lib64/python2.7/site-packages/yolk/cli.py", line 282, in upgrade                                                       
    user_installs_only=self.options.user)]
  File "/home/jesse/.local/lib64/python2.7/site-packages/yolk/cli.py", line 1117, in _updates                                                     
    results = pool.map(worker_function, names)
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 253, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 572, in get
    raise self._value
UnboundLocalError: local variable 'dist' referenced before assignment
$ 

$ python2.7 --version
Python 2.7.15

Request: Support local PyPi servers

I was hoping to use this to check the latest version of a package behind our corporate firewall.

XML_RPC_SERVER is hardcoded. It would be nice to support pip.conf and other indexes.

Using yolk with python3 clashes with something cached in ~/.yolk

I've been using yolk with python2. Today for the first time i tried to move my project to python3. I've set up virtualenv to use it, installed yolk and got bizarre syntax errors related to python ast module. rm -rf ~/.yolk solved the problem. I guess there is something cached there related to python version, so mixing 2.x with 3 causes problems.

UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 342: ordinal not in range(128)

% yolk --version
yolk 0.9

Steps to reproduce:

% yolk -M idna==2.6 | cat
Traceback (most recent call last):
  File "........../venv/bin/yolk", line 11, in <module>
maintainer: 
docs_url: None
project_url: []
maintainer_email: 
cheesecake_code_kwalitee_id: None
obsoletes_dist: []
keywords: 
package_url: https://pypi.org/project/idna/
requires_external: []
author: Kim Davies
home_page: https://github.com/kjd/idna
download_url: 
platform: 
version: 2.7
obsoletes: []
provides: []
cheesecake_documentation_id: None
_pypi_hidden: False
    sys.exit(main())
  File ".......site-packages/yolk/cli.py", line 1171, in main
    my_yolk.run()
  File ".......site-packages/yolk/cli.py", line 224, in run
    return getattr(self, action)()
  File ".......site-packages/yolk/cli.py", line 692, in query_metadata_pypi
    print(u'{}: {}'.format(key, metadata[key]))
  File ".......site-packages/yolk/cli.py", line 92, in write
    self.stdout.write(inline)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 342: ordinal not in range(128)

Update to new PyPI API

With the new PyPI warehouse, seems like the XML API will be deprecated in 2018. Will this package be updated to use one of the other APIs?

newer Python packages store metadata in METADATA, not PKG-INFO

See pull request cakebread#25; it would be nice to incorporate this fix into yolk3k.

nothing happens for yolk -D wheel but -l works

something is wrong with yolk if behind a firewalll and pypi proxy
i was able to install yolk3k with pip install yolk3k but any yolk command except -l show nothing just blocked

[08:53:25][iwetzel@devPC:~/Testautomation]$ yolk -D wheel
<urlopen error [Errno 110] Connection timed out>

Yolk crashes when PyPi package descriptions contain certain characters

Yolk (0.8.7) reports and encoding error when printing information about packages.

For example, when I

yolk -M crypto-enigma

I get

UnicodeEncodeError: 'ascii' codec can't encode character u'\u03b2' in
position 2322: ordinal not in range(128)

because the PyPi page for the package has a β in it. Or if I try

yolk -M Metafone

I get

UnicodeEncodeError: 'ascii' codec can't encode characters in position 3291-3292: ordinal not in range(128)

Is there a way to avoid this error? It seems like a bug that Yolk would crash just because there's a Greek character or mathematical symbol in the PyPi page for a package.

Fails with 403 error on any command that queries PyPI

Yolk is failing with "HTTP Error 403: Must access using HTTPS instead of HTTP" when it runs with any command which queries PyPI. This started just in the last few days.

Bugfix fork addressing urllib exceptions

I was having trouble running yolk -U, as the entire process would halt whenever the occasion of a transient urllib2.URLError would bubble up to the CLI:

… so I forked yolk3k and vendored in retrying – which I used here and here, rather coarsely, to elide halting the program on these exceptions.

The fork works as it should, but it seemed like a biggish change for an unsolicited PR; if you are interested in integrating this stuff upstream, I’d be happy to clean it up and squash it down. Indeed. Thanks in any case, yes!

yolk doesn't work in python3

 vagrant@vagrant-openSUSE-Leap  ~/cinder   master  pip install yolk                                 
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).
Collecting yolk
  Downloading https://files.pythonhosted.org/packages/2b/c0/73510e50668b70f3fad25328c58d7a29a300a2e452058dfdcb7c7538af7b/yolk-0.4.3.tar.gz (86kB)
     |████████████████████████████████| 92kB 463kB/s 
Requirement already satisfied: setuptools in ./.tox/docs/lib/python3.4/site-packages (from yolk) (41.2.0)
Building wheels for collected packages: yolk
  Building wheel for yolk (setup.py) ... done
  Stored in directory: /home/vagrant/.cache/pip/wheels/70/ea/49/1364272c97de7b2d9068d5759b097a9ded59df1bddb80a5a9c
Successfully built yolk
Installing collected packages: yolk
Successfully installed yolk-0.4.3
 vagrant@vagrant-openSUSE-Leap  ~/cinder   master  yolk -V sphinx                                    Traceback (most recent call last):
  File "/home/vagrant/cinder/.tox/docs/bin/yolk", line 6, in <module>
    from yolk.cli import main
  File "/home/vagrant/cinder/.tox/docs/lib/python3.4/site-packages/yolk/cli.py", line 262
    print " %s %s (%s)" % (project_name, dist.version,
                      ^
SyntaxError: invalid syntax

UserWarning: Distutils was imported before Setuptools

Since setuptools 49.2.0, a warning is printed whenever I run yolk:

/usr/lib/python3.8/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.

See https://setuptools.readthedocs.io/en/latest/history.html#v49-2-0
and pypa/setuptools#2230

Maybe the imports can be reordered to get rid of this visual disturbance?
Thanks for yolk!

Main README is missing the -V option.

Seems like this (yolk -V PKG_SPEC) would be one of the most common use cases.
It's in the -h output but not present in the repo's top page.

can load "cli" module when invoking the via "-m (with Python 3.5)

C:\Users\nad2000>py -3 -m yolk
Traceback (most recent call last):
  File "C:\Python35-32\lib\runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python35-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python35-32\lib\site-packages\yolk\__main__.py", line 22, in <module>
    from cli import main
ImportError: No module named 'cli'

However it work OK with 2.7 and 3.4

feature_request(path): Custom packages path support

1. Summary

It would be nice, if yolk3k will support check licenses from custom path.

Similar issue for pip-licenses package.

2. Argumentation

Example: I use Pelican — static site generator and plugins for it, written by Python. Pelican plugins must installed to custom folder — usually pelican-plugins, not to default site-packages. I don't know, how possible check licenses from pelican-plugins folder.

3. Example of expected behavior

If:

yolk -l -f license --path output/pelican-plugins

yolk3k will check licenses from output/pelican-plugins folder recursively.

4. Not helped

I don't find already existing packages, that solve this problem. For example, I try:

• license_lister,
• license_info,
• plicense,
• pip-licenses,
• pylicense.

Thanks.

yolk -V reports versions even if they have no download available

Is this intended/documented behavior?

% yolk -V theano # Reports unavailable version
Theano 0.7.0rc2
% #####################################################################
% # Demonstrations that it's unavailable at the moment
% #####################################################################
% pip install theano==0.7.0rc2
Collecting theano==0.7.0rc2
  Could not find a version that satisfies the requirement theano==0.7.0rc2 (from versions: 0.3.0rc4, 0.3.0, 0.3.1rc1, 0.3.1rc2, 0.3.1, 0.4.0rc1, 0.4.0rc2, 0.4.0rc3, 0.4.0rc4, 0.4.0, 0.4.1rc1, 0.4.1rc2, 0.4.1, 0.5.0rc1, 0.5.0rc2, 0.5.0, 0.6.0rc1, 0.6.0rc2, 0.6.0rc3, 0.6.0rc5, 0.6.0)
  No distributions matching the version for theano==0.7.0rc2
% wget -q -O - https://pypi.python.org/simple/theano/ | grep 0.7.0rc2
% 

HTTPS failures on mac

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>

other libraries deal with this by taking a dependency on certifi

https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error

yolk --upgrade fails with HTTPTooManyRequests

Appears to lack appropriate API pacing/throttling.
Or PyPi has recently added API restrictions?

[I] ➜ yolk --upgrade
Checking geoip2                                                                                                                                                                  Traceback (most recent call last):
  File "/Users/edgeb1/.pyenv/versions/3.9.0/bin/yolk", line 33, in <module>
    sys.exit(load_entry_point('yolk3k==0.9', 'console_scripts', 'yolk')())
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/cli.py", line 1171, in main
    my_yolk.run()
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/cli.py", line 224, in run
    return getattr(self, action)()
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/cli.py", line 279, in upgrade
    names = [values[0]
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/cli.py", line 279, in <listcomp>
    names = [values[0]
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/cli.py", line 1117, in _updates
    results = pool.map(worker_function, names)
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/cli.py", line 1109, in worker_function
    (project_name, versions) = pypi.query_versions_pypi(
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/pypi.py", line 147, in query_versions_pypi
    versions = self.package_releases(pypi_pkg)
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/pypi.py", line 197, in package_releases
    return self.xmlrpc.package_releases(package_name)
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/xmlrpc/client.py", line 1116, in __call__
    return self.__send(self.__name, args)
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/xmlrpc/client.py", line 1456, in __request
    response = self.__transport.request(
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/site-packages/yolk/pypi.py", line 67, in request
    return self.parse_response(fhandle)
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/xmlrpc/client.py", line 1348, in parse_response
    return u.close()
  File "/Users/edgeb1/.pyenv/versions/3.9.0/lib/python3.9/xmlrpc/client.py", line 662, in close
    raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: 'HTTPTooManyRequests: The action could not be performed because there were too many requests by the client. Limit may reset in 1 seconds.'>

From https://warehouse.pypa.io/api-reference/#:~:text=Due%20to%20the%20heavy%20caching,consumer%20based%20on%20irresponsible%20activity.

"Due to the heavy caching and CDN use, there is currently no rate limiting of PyPI APIs at the edge. The XML-RPC API may be rate limited if usage is causing degradation of service."