pypa / packaging.python.org

Python Packaging User Guide

Python Packaging User Guide

http://packaging.python.org

The "Python Packaging User Guide" (PyPUG) aims to be the authoritative resource on how to package and install distributions in Python using current tools.

To follow the development of Python packaging, see the Python Packaging Authority.

Code of Conduct

Everyone interacting in the Python Packaging User Guide project's codebases, issue trackers, chat rooms, and mailing lists are expected to follow the PSF Code of Conduct.

Contributing

This guide is community-maintained and contributions are welcome! Please see the contributing guide for details on our writing style guide and how to build the guide locally to test your changes.

License

The Python Packaging User Guide is licensed under a Creative Commons Attribution-ShareAlike license: http://creativecommons.org/licenses/by-sa/3.0 .

History

This Guide was forked from the “Hitchhiker's Guide to Packaging” in March 2013, which was maintained by Tarek Ziadé. Thank you Tarek for all your efforts in Python packaging.

packaging.python.org's Issues

add a FAQ

in addition to the tutorial, I think a FAQ would be another way to help users find the information they need.

Catalog of bdist_{rpm, deb, nsi, wininst, msi, esky, wheel} utilities?

There are myriad ways to generate OS packages and installers from setuptools. The question has been presented a number of times. Would a separate page for cataloging bdist_* methods be in scope for this project?

• http://www.reddit.com/r/Python/comments/1qx2nb/distributing_python_3_applications_very_tempted/cdhnwku
• http://www.reddit.com/r/Python/comments/1ew4l5/im_giving_a_demo_of_python_to_a_bunch_of_java/ca4gf55

docs.python.org updates

issue to track docs.python.org changes related to packaging/installation

so far, we just have some disclaimers added with links back to the PUG
Python 2.7 docs: http://hg.python.org/cpython/rev/a9f91a38a265
Python 3.3 docs: http://hg.python.org/cpython/rev/16b7536e418b

Use HTMLDir

The default on RTD is to use Sphin'x HTML output, which does something lame like https://packaging.python.org/en/latest/current.html which is not a pretty URL. It would be great if instead this used HTMLDir which would give an URL like https://packaging.python.org/en/latest/current/ which is prettier!

Ensure the user guide provides a clear answer to this SO question

This is a good example of the kind of confusion that we want the Installation & Packaging Guide to address:
http://stackoverflow.com/questions/14665216/how-to-easily-distribute-python-software-that-has-python-module-dependencies-fr

(No, I'm not copying the whole thing in, it's huge)

---

migrated from https://bitbucket.org/pypa/python-packaging-user-guide/issue/9

refactor virtualenv docs

use the Standard Pypa Docs Template

"Project" glossary term

migrated from https://bitbucket.org/pypa/python-packaging-user-guide/issue/33/glossary-item-project-and-subsidiary-terms

---

Graham Wideman created an issue 2013-11-04
The word "project" is pretty important in the packaging and distribution landscape. For example in the single page Building and Distributing Packages with Setuptools it appears almost 300 times, with many qualities and characteristics of a "project" referred to or implied.

A reader might well not know whether "project" has some formal status, as it does, for example, in IDEs for Java or other languages. In the Python realm, I'm pretty sure it does not have a formal status. The Setuptools page referenced above seems to use "project" to refer, in general terms, to "a coherent collection of files being worked on by a developer; often organized in a single directory tree", and often seems to be used as a rough synonym for "package" or "distribution".

There is a separate more specialized meaning that includes the idea of a bunch of developers working in concert on an open-source, er, project, that's offered on, for example, PyPI.

I think the Glossary entry for "project" could do a better job in making these things clear. Some issues:

• The first line currently punts to "library, framework, script....", where the meanings of these (in the Python packaging context) are not in the glossary. Entries for "library" and "script" would be helpful, especially for "script" -- it took a long time for me to stumble on the fact that "script" implies "a module intended to be launched directly from the command line (or OS GUI)" and not just any module written in Python. And that fact is pertinent in packaging and distribution.
• "Python projects must have unique names." That suggests that there is something on the developer's machine that (a) defines a project and (b) has a slot for a name. This seems not to be true, at least in the sense that "project" is used in the setuptools doc. But what about in the PyPI sense? I took a quick look at PyPI, and although I've not completed an upload process, I don't see a slot on the "Submitting package information" form where it asks for a project name. (Package name, yes.).
  Anyhow, because of the frequency with which "project" appears, some increased clarity on what range of things people intend when they use it would be a big help for the newcomers who most need the docs :-).

convert FIXMEs to TODOs

convert the FIXMEs to TODOs and use the sphinx todo plugin to gather them into a list that's displayed.

---

migrated from https://bitbucket.org/pypa/python-packaging-user-guide/issue/43

Quick Recommendations page -- obstacles to understanding or adoption

migrated from https://bitbucket.org/pypa/python-packaging-user-guide/issue/27/quick-recommendations-page-obstacles-to

---

Graham Wideman created an issue 2013-10-23
On another recent issue I've commented that I'm trying to assemble coherent recommendations for packaging and deployment doc improvements, and that is still in process.
In the meantime, a couple of most-pressing issues with the Quick Recommendations page, which, if clarified either on that page or here, would help at least me considerably.
A) Quick Recommendations appears to recommend a non-functional process:
setuptools for packaging
setuptools docs says setuptools produces eggs
pip for installing such a package
Quick Recommendations page and also the pip docs says that pip cannot install eggs.
How should these conflicting assertions be reconciled?
B) Quick Recommendations recommends a process that seems a big leap from plain manual copying files around.
An appreciable portion of the audience for this page will be moving up from simply copying their script and module files from one machine to another manually, and then manually fixing up sys.path, PYTHONPATH, .pth files or whatever.
They are interested in doing this process in a less manual, more repeatable and reliable fashion, and don't necessarily have any use for (and might want to deliberately avoid) sending their distributables through some server (PyPI).
Is it safe to assume that the deployment tools can produce a distributable locally, and can consume a distributable locally? I think this scenario should be prominently mentioned, with discussion of whether or not the setuptools + pip satisfies this scenario.
Comments (11)

Graham Wideman
I've finally got to somewhat of a grip on a set of conceptual problems that are distributed across PPUG, and the PyPA Setuptools and pip docs, and the python.org docs on Distributing and Installing (aka Distutils) docs.
A) The advice on the Quick Recommendation page to "Use Setuptools to build and package" takes the reader off to investigate Setuptools. PyPA Setuptools doc advertises that that it creates eggs. But then pip docs says that pip cannot install eggs.
What's not evident from this sequence of reading is that:
Counter to naive expectations, the fact setup.py calls setuptools.setup() does not imply that setuptools is providing the API that the user will encounter, and does not imply that the Setuptools docs are sufficient to understand how to "Use Setuptools".
I ended up resorting to reading the setuptools source to find out that setup.py's call to setuptools.setup() is actually a call to distutils setup(), hence the command vocabulary is that of distutils, except for the commands added by setuptools.
So "Use Setuptools" really means "use Distutils maybe with a few features that are added in by Setuptools"... and therefore read the Distutils docs to figure out what is going on.
In particular, the advice to "use Setuptools" does not imply "invoke Setuptools to produce the one archive format that Setuptools docs tell how to produce."
So rather crucial on the current page is that "Use Setuptools" should be replaced by "Use Setuptools, and at the 'create archive' step use the sdist command supplied by underlying distutils so that the resulting archive is in "source distribution" form.' or whatever the correct course of action is.
(FWIW, I have noted that sdist appears in the example on the Packaging Guide page. Confusingly, although the example setup.py shows import from setuptools, the discussion under "Create your first release" proceeds as though setup is calling distutils directly, and setuptools is discussed later as an alternative thing to do to get more functionality.)
B) The comment "Use easy_install or buildout if you need to install from the binary egg format, which pip can’t currently handle" is ambiguous. It may well suggest (as I originally read it days ago) that what pip can't handle is the binary variant of egg format, but otherwise pip can handle "non binary" egg format. Of course, that leads to speculation of what the "binary version of egg format" might be -- perhaps an egg in which the files are compiled rather than pure python source code.
Based on my current understanding (which still could be wrong) that pip can't install any eggs, this statement should be revised to remove the word 'binary' since it adds nothing and just makes the sentence ambiguous.
C) Although not solely the fault of this page per se, a great deal of ambiguity results from virtually every page related to this topic failing to be clear about specifically what files and formats a particular tool produces or consumes. As a minimal remediation, this Quick Recommendations page should recommend what archive format(s) to use (and why).
Examples of problem PyPA docs in this regard:
[http://docs.python.org/3.3/distutils/commandref.html#creating-a-source-distribution-the-sdist-command] vaguely says it produces a "source distribution", whatever that means. It probably has a specific meaning in the mind of the writer, but not for the reader.
http://pythonhosted.org/setuptools/setuptools.html, with the heading "Building and Distributing Packages with Setuptools", says that Setuptools produces eggs. No mention that when seemingly invoking Setuptools you are free to use the commands of Distutils and might well produce other formats of distributable.
http://www.pip-installer.org/en/latest/index.html the pip doc says vaguely ""A tool for installing and managing Python packages", and I found no other page documenting what format packages it digest. The only docs are conflicting statements that pip does NOT install eggs (http://www.pip-installer.org/en/latest/other-tools.html), and examples and flag docs showing that pip DOES install eggs. (http://www.pip-installer.org/en/latest/usage.html, http://www.pip-installer.org/en/latest/logic.html). That inconsistency of doc seems unusable for what's intended to be the recommended way of doing things.
So, beyond the scope of the current page, what the reader needs somewhere is a reliable discussion of what python distribution formats are in common use, and which tools work with them. And each tool's doc needs to provide a clear specification of what formats it produces or consumes.
Mark as spam Delete 2013-10-24

Marcus Smith
thanks for writing all this out. this is very helpful to see how people interpret things. I'll respond to all your points. may be this weekend though.
Edit Mark as spam Delete 2013-10-24

Marcus Smith
I can already say that we need to be keeping a list of todo's for the pip and setuptools docs (with setuptools having the biggest list). It's clearly not just about writing this guide.
Edit Mark as spam Delete 2013-10-24

Marcus Smith
First off, note that I just updated the Quick Rec page, but here's specific responses.
setuptools docs says setuptools produces eggs pip for installing such a package Quick Recommendations page and also the pip docs says that pip cannot install eggs. How should these conflicting assertions be reconciled?
yes, the setuptools docs needs to emphasize how it's really being used by devs, as an sdist generator (not an egg generator). I've created a dedicated issue to track changes we need to make to setuptools docs. https://bitbucket.org/pypa/python-packaging-user-guide/issue/30/modernize-and-improve-setuptools-docs
Is it safe to assume that the deployment tools can produce a distributable locally, and can consume a distributable locally? I think this scenario should be prominently mentioned, with discussion of whether or not the setuptools + pip satisfies this scenario.
yes, certainly. not sure right off where this should be mentioned, but I'll cut a ticket for it.
So "Use Setuptools" really means "use Distutils maybe with a few features that are added in by Setuptools"... and therefore read the Distutils docs to figure out what is going on.
I'm thinking setuptools should go ahead and fill out sections for all the commands for completeness, even the ones it doesn't alter, and just link, when it's not altering anything. That will make it easier on users to understand the layering.
suggest (as I originally read it days ago) that what pip can't handle is the binary variant of egg format, but otherwise pip can handle "non binary" egg format.
Egg is a binary distribution format. Not sure where you're getting the "non binary" egg format from?
Maybe you're talking about "egg-info" directories? Internally, pip actually uses setuptools to install from sdist. By default, setuptools would convert sdists to eggs and install them as eggs, but pip passes certain options that makes setuptools do the installation with the packages exposed and using "egg-info" dirs, not fully encapsulated eggs. That could be explained better here http://www.pip-installer.org/en/latest/logic.html#setuptools-pkg-resources I'll add an issue to pip for that.
examples and flag docs showing that pip DOES install eggs. (http://www.pip-installer.org/en/latest/usage.html,
you mean the VCS examples with "egg="? that's not installing from eggs, but point taken, we need to explain the origin of that syntax. I'll add a pip issue for that too.
Edit Mark as spam Delete 2013-10-27

Nick Coghlan
pull request #10 is somewhat related to this - it splits the quick recommendations into "these aren't going away" stable recommendations and "we aren't happy with the state of these, but they're the best current option" interim recommendations.
Mark as spam Delete 2013-11-02

Nick Coghlan
Reading Graham's comments more closely, I realised the tool split is actually less closely related than I thought ( commit 2b1ad9ff758d568087752853f5bea0447eb58ce2 splits the installation and packaging tool recommendations).
Agreed there's still a lot of work to be done on the build tooling side.
Mark as spam Delete 2013-11-02

Graham Wideman
I see that the new wording (as of 2013-11-03) is clearer. (And FWIW bypasses the ambiguity I'd stumbled over about "binary Egg format.)
Still mysterious I think to people just trying to get started on reliably distributing their Python code to colleagues:
Do distutils, setuptools, pip etc do anything for you if all you want to do is get your code from dev machine A to work on user machine B, and you don't want to futz with PyPI as intermediary?
Does "build" mean anything for distributing python projects as source code? (And if so, what?)
And a comment on footnote [3] regarding pyvenv instead of virtualenv: What's behind the advice to wait until 3.4 to use pyvenv, when pyvenv is in 3.3? I've used pyvenv in 3.3 and it sets up a virtual environment... but maybe I'm missing something?
Mark as spam Delete 2013-11-04

Nick Coghlan
"pip wheel" is useful for internal redistribution (since other people won't need build dependencies). For pure Python apps, it's probably easier to just chuck everything needed into a zip archive with a main.py file (that's why that feature was added back in Python 2.6 - with any luck we will have better support for creating such archives in Python 3.4)
The build step can be useful even for pure Python projects if they distribute command line scripts. It also deals with various metadata related things
The main reason to avoid pyvenv in 3.3 is the fact it doesn't bootstrap pip automatically, whereas virtualenv does.
Mark as spam Delete 2013-11-04

Graham Wideman
Hi Nick -- followup:
Again regarding build: I am very familiar with what "build" normally means for other environments, like Java. But in Python packaging usage, it's not clear (at least to me) what the word "build" is intended to encompass. Does it refer to steps taken before deployment (as in preparing a binary distribution), or after distribution (like compiling, on the user's machine, from a source distribution), or both? And is it generally reckoned that there's a build step on both developer's and user's machines? For both source and binary distributions?
Hmmm "Build" would be a great term to have in the Glossary!
Regarding chucking everything into a zip: Yes, but colleagues may still need 3rd party dependencies to be fetched. Also: "that's why that feature was added back in Python 2.6". What feature are you referring to? Some setuptools or distutils command? Or something else?
Thanks.
Mark as spam Delete 2013-11-04

Nick Coghlan
Agreed, build would be a good glossary term (it refers to doing anything starting from source, whether creating a wheel file or installing directly on the target system). When wheels are used, there's no build step on the deployment target, so pip can also skip installing the build dependencies.
The feature I'm referring to is CPython's ability to execute a zip archive with a top level main.py file as a script, implicitly adding the zip file as the first entry on sys.path so you can import from it. For scripts with only pure Python dependencies, that's a quick and easy way to make them redistributable without needing an install step.
Mark as spam Delete 2013-11-05

Graham Wideman
Hi Nick: Thanks a lot for your comments.
So "build" specifically doesn't refer to steps that are just copying files to the right locations, or adding/changing .pth files, launch scripts/exes and so on?
On the python-executable zip (using main.py): OK, I hadn't previously noted that scheme. I guess that main.py could be used directly as the entry point for an application (the other modules of which are in the zip), OR main.py could be the entry point for the installer of whatever else is in the zip?
Is there a tool that creates either of these kinds of distributable? Or is the idea to create these manually (or with DIY scripts)?

refactor pip docs

use the Standard Pypa Docs Template

Glossary Project entry

" It’s possible to install a distribution from the project ‘spam’ and have it provide a package importable only as ‘eggs’."

Very unfortunate example, as "egg" is one of the distribution formats, giving this sentence multiple confusing meanings.

setup.py/install_requires vs requirements.txt

3 distinctions to cover I think:

• abstract vs concrete requirements (https://caremad.io/blog/setup-vs-requirement/)
• minimally defined vs exhaustive or frozen (http://stackoverflow.com/a/7085000)
• project requirements vs environment requirements.

API/ABI difference needs to be explained

I wanted to add there links to extensions.rst page:

There were links here, but @theacodes removed them because they linked to NSFW spam

But could not find a good place to inject this. I am not sure that I completely get the difference between Public API and Stable API from a perspective of extension developer, and also there is no explanation what ABI is about and what is its practical purpose.

Packaging guide links to (contradictory) setuptools documentation

The packaging guide currently links to the setuptools documentation (http://pythonhosted.org/setuptools/). Given that the latter contradicts the packaging guide in multiple instances, I think that having the link is confusing to newcomers to Python and its packaging ecosystem. I think the link to http://pythonhosted.org/setuptools/ should be removed, or at least clearly marked with something saying that it is obsolete and that its advice should not be followed.

The link is found (at least) in the glossary for "setuptools". Given that pretty much every instance of the word "setuptools" in the packaging guide links to the glossary, it's easy for a newbie to get from the (correct) packaging.python.org documentation to the (obsolete) setuptools ones, and continue from there.

Consider switching to "python -m pip" in examples

The bare "pip" invocation can fail in various circumstances, and it isn't clear how to invoke it such that it applies to the expected version of Python when multiple versions are installed in parallel. "python -m pip" works in more cases, and works cleanly with version switching invocations, like the Python Launcher for Windows, and python vs python3 at the system level on *nix systems.

http://bugs.python.org/issue21699 covers a new bug discovered with running pip from Windows directories containing spaces in Python 3.4

"sphinx" should be found in the docs

Since documentation of python packages is done with sphinx it should be mentioned somewhere.

But up to now it is not. See:
https://python-packaging-user-guide.readthedocs.org/en/latest/search.html?q=sphinx

Please add some information about sphinx.

A link to the sphinx docs is enough. Nobody needs a redundant copy of the sphinx docs.

Clarify why we recommend entry points

Add a bit more details on why entry points are recommended rather than .py scripts or batch files. Include a mention of the limitations of PATHEXT on Windows and the difficulty for single-platform developers of maintaining robust cross-platform solutions.

refactor setuptools docs

Opening this issue to track the effort to fix/improve the setuptools docs.

This relates to this guide, because it makes it easier to link/reuse it's content, vs just trying to paste a tutorial on top of it.

Specific things to address:

• use the Standard Pypa Docs Template
• de-emphasize eggs and easy_install (currently listed as top 2 features)
• emphasize that setuptools is used (ignoring pip's use of it) primarily as an sdist generator.
• Take the best of the 2 "Egg Specs" we have and combine them. Although Egg is in demise, we shouldn't lose how it works (http://pythonhosted.org/setuptools/formats.html and http://peak.telecommunity.com/DevCenter/PythonEggs)
• setuptools should mention "bdist_wheel" being available from the wheel project as a setuptools extension (probably anywhere "bdist_egg" is mentioned)
• the layering of setuptools on top of distutils is hard to follow. the setuptools docs should cover all the "setup.py" sub-commands and keywords, modified or not, even if in certain ones, it just links to distuitls, and be clear on what's different, if they are modified.
• be clear that "easy_install" is not the pypa-recommended installer, but be open about the pros/cons, and mention this chart https://python-packaging-user-guide.readthedocs.org/en/latest/technical.html#pip-vs-easy-install

document installation of optional dependencies in the tutorial

The tutorial should show the syntax required to install an optional dependency for a project.

Add topic for "Supporting Multiple Python Versions"

Similar to the "Binary Extensions" topic, it would be good to have a topic covering how to support multiple Python versions effectively.

Useful projects/resources to reference:

• six
• tox
• Travis and Shining Panda CI
• Ned Batchelder's What's in Which version pages
  • http://nedbatchelder.com/blog/201310/whats_in_which_python_3.html
  • http://nedbatchelder.com/blog/201109/whats_in_which_python.html
• Lennart Regebro's "Porting to Python 3"
• the Python 3 porting how to in the main docs
• cross reference to the stable ABI discussion in the binary extensions topic (once that exists)
• mention version classifiers for distribution metadata

---

migrated from https://bitbucket.org/pypa/python-packaging-user-guide/issue/41

Please include sphinx

Please include some basic instructions where to put sphinx docs.

I don't want a copy of sphinx introduction, but a basic help how to layout
the directory of the docs.

I know that anybody can use the structure he wants, but a basic best practice guideline would help people which are new to python.

Alternative TOC

thoughts on a different TOC (motivated from comments in #16)

• Introduction
• Installation & Packaging Quickstart (like now but with more links and notes)
  • Install the Tools
  • Create an Environment
  • Install Packages
  • Cache Wheels
  • Create your own Project
  • Build & Upload your project to PyPI
• Tool Recommendations (like now, but also add justifications where needed)
• The Future of Python Packaging
• Advanced Topics
  • Technical Details
  • Installation Schemes (the "What is installation" section from the old TOC)
  • Packaging Formats (the "What is packaging" section from the old TOC)
  • <all the existing topics>
  • Platform Integration
  • <all the existing topics>
  • Binary Extensions
  • <all the existing topics>
• Project Summaries
• PEP Summaries
• A Packaging Timeline
• Glossary

cc @pfmoore @ncoghlan

typo in history.rst

Being able to able to bootstrap pip into Python 3.4.

should be

Being able to bootstrap pip into Python 3.4.

Mention wheel's `universal = 1` setting

Maybe in http://python-packaging-user-guide.readthedocs.org/en/latest/quickstart.html#build-upload-your-project-to-pypi ?

No search hit for "requirements.txt"

There is no search hit for "requirements.txt". The word is in the docs, but some how I does not appear if you search for it.

See: http://python-packaging-user-guide.readthedocs.org/en/latest/search.html?q=requirements.txt&check_keywords=yes&area=default#

Reconcile RTD URL and canonical packaging.python.org URL

So right now we're using the excellent RTD platform for hosting this guide. However the old URL is still floating around and a lot of people don't know about the https://packaging.python.org/ URL.

Maybe it would be a good idea to move to hosting this ourselves (or use a different hidden name) and have the old long name use a HTML redirect or even just say that the guide has moved. Maybe we can even get RTD to do a proper redirect?

Provide a clearer visual brand identity for the PyPA documentation

Currently, there's no obvious indication when viewing the Python Packaging User Guide itself that it's an officially blessed project of not only the pip/virtualenv/pypi/twine/setuptools development teams, but also of the CPython core developers. The "packaging.python.org" URL isn't enough, especially when the ReadTheDocs link is still better known to search engines.

It would be desirable to align the theming of the Packaging User Guide with that of the main documentation to help make that connection clearer to readers.
The current docs theme components: http://hg.python.org/cpython/file/default/Doc/tools/sphinxext/

(Also see #304 for discussion of the considerations of theme switching in general)

Have separate tutorials for project maintainers and installing packages

I offered high-level feedback on the Packaging User Guide in its early stages -- back when the source files were hosted on Bitbucket. I'm very happy with how the documentation has come along since then. Thank you! I especially like how the documentation aims for simplicity.

With that in mind, I would like to suggest that the tutorial for installing packages be separated from the tutorial for project maintainers. This will make the User Guide experience simpler for the "90% case" of package installers (because, for example, they will be scrolling and reading a smaller section). It will also be simpler for project maintainers, because the project-maintenance specific information will be in its own, smaller section.

I believe that other gains in simplicity and organization will become possible once this change is made.

If you do this change, I think it would help if the title of the second tutorial used the term "Project" (like the corresponding subsections do), instead of "Package" or "Distribution." For example, it could be called something like, "Tutorial for Project Maintainers." The other tutorial can be called something like, "Tutorial for Installing Packages."

Lastly, I seem to recall that earlier versions of the Guide had this organization (and possibly went back and forth more than once). If there were certain reasons you last switched from having two tutorials, I would like to know those reasons to see if they can be addressed. I do think it helps to move in the direction of having information specific to certain audiences be in their own sections. Thanks a lot.

LCA2014 presentation link

linux.conf.au 2014: Python Packaging 2.0: Playing Well With Others (http://lwn.net/Articles/580399/ and https://www.youtube.com/watch?v=7An2GobbSWU)

Had another go at finding a video for PyTexas, no joy.

Do we want to point people at Noah's upcoming PyCon US 2014 talk? (https://us.pycon.org/2014/schedule/presentation/204/)

This intro talk from Asheesh Laroia may also be worth highlighting: https://us.pycon.org/2014/schedule/presentation/219/

Provide packagers with clarification on pythonhosted.org

One question that I'm sure newcomers to packaging have is, "where should I host my documentation?" or "is there an official or preferred place to host my documentation?" (I sure had this question.)

Of course, PyPI displays the package's "long_description" (which is also worth mentioning in the guide), but PyPI also says this on the edit page for a package--

You can now host documentation at http://pythonhosted.org/project_name. To upload documentation, prepare a .zip file that is unpacked into this URL. Only static pages are supported. The zip file must have a top-level "index.html".

I know the Guide tries to avoid taking sides on third-party tools (e.g. test frameworks, etc). But given that pythonhosted.org is owned by PSF and recommended by PyPI, is this a candidate for a recommendation (e.g. on the "Tool Recommendations" page)?

Since pythonhosted.org is more a service rather than a tool, perhaps the page title could be changed to "Tool & Service Recommendations." (PyPI itself is another service that is implicitly recommended on the page; it doesn't have its own bullet point but could under a "Services" section for completeness and clarity.)

Revisit "Package" vs "Distribution" vs "Distribution Packages"

To align with my changes to the stdlib docs, I tweaked a couple of section titles to use the term "Distribution Packages" rather than "Distributions": 4c2e875

However, that introduced something of an internal inconsistency within PyPUG, so I created this issue to specifically consider how to handle this long term (the bare "Distributions" really didn't work in the context of the standard library docs, and I'd like to keep the two as consistent as we feasibly can).

Articulate distinction between "library" and "application" projects

There is a distinction between "library" projects and "application" projects that I only recently became more clearly and consciously aware of. For example, see: http://www.fullstackpython.com/application-dependencies.html

The installation and distribution procedures in these two cases are somewhat different. For example, on the packaging and distribution side, it seems libraries should normally use setup()'s install_requires argument, while applications can use requirements.txt, for example.

I think it would be good if the User Guide started articulating this distinction (e.g. in the glossary), so that it can eventually be made more clear which use cases the different advice in the Guide applies to. (For example, I think most of the advice in the tutorials is for installing and packaging libraries rather than applications, even though the language uses the more general term "project.")

This is a bit related to the following (but more general): #10

Mention Twine's support for GPG signatures

Either twine -a or by just including the *.asc files in its argument. Probably belongs in http://python-packaging-user-guide.readthedocs.org/en/latest/quickstart.html#build-upload-your-project-to-pypi

Please explain entry point console_scripts

Please explain the entry point 'console_scripts'.

The sampleproject contains an example, but docs would be nice.

# setup.py of sampleproject

    # To provide executable scripts, use entry points in preference to the
    # "scripts" keyword. Entry points provide cross-platform support and allow
    # pip to create the appropriate form of executable for the target platform.
    entry_points={
        'console_scripts': [
            'sample=sample:main',
        ],
    },

The "Scripts" section is still empty: https://github.com/pypa/python-packaging-user-guide/blob/master/source/tutorial.rst#scripts

document `--user` option for core tool installs

consider documenting (assuming we test it more) that you can get-pip.py --user, and mention --user as a possibility for the install of virtualenv (and wheel, and twine too, if not in a virtualenv)

see pypa/pip#1668

Add a section on building Windows binary wheels using a CI services (appveyor)

I want to include a section in the official user guide on how to set up automatic wheel builds for Windows. The idea is to give projects a simple "cookbook" approach to getting wheels, without needing access to Windows environments, etc.

Some possible issues:

1. We don't cover testing in the PUG, so is it OK to describe setting up to CI services?
2. Is it OK to promote appveyor here? To my knowledge, there is no available alternative at the moment, but obviously that may change.

Also, a technical issue - I need to provide a couple of files to be copied into the user's project (appveror.yml and a couple of scripts to install pip and wheel and set up the compiler environment). How should I include these? I'd like to have them inline as text, and also linked so that people can download them. Where to host them?

Alternatively, if this is not suitable for the PUG, where would be a good place to publish a document like this? Maybe we should start a series of "Packaging HOWTO" guides? Or a PyPA blog?

mention the Pypa sampleproject

mention https://github.com/pypa/sampleproject here https://python-packaging-user-guide.readthedocs.org/en/latest/packaging.html#getting-started-with-setuptools, but also in a new "Quickstart" section that @pfmoore is working on

"Terminology Usage and Pronunciation Guide"

pasted from #87 (comment)
(from @ncoghlan )

So my proposal would be that we add a separate "Terminology Usage and Pronunciation Guide", distinct from the existing glossary, as it's primarily useful to folks writing documentation and creating presentations, rather than being necessary for folks reading or watching them.

Writing that would be where I agree the more explicit "distribution package" and "import package" terms can help:

• "distribution package" always refers to "thing you can download and install"
• "distribution" usually refers to "distribution package" but, depending on context, may refer to the overall field or to Linux distributions. In PyPA documentation, it should only be used in the "distribution package" sense.
• "software distribution and packaging" always refers to the overall field, as do "software distribution" and "packaging" on their own. Hence, these are the preferred terms for this concept in PyPA documentation (as "distribution" is reserved to mean "distribution package"). The PyPA, PyPI and PyPUG names use the term "Packaging" in this sense.
• "distro" always refers to a Linux distribution, and hence is the preferred for this concept in PyPA documentation (as "distribution" is reserved to mean "distribution package").
  • "import package" always refers to "thing you can import that may have submodules"
  • "package" usually refers to "import package" but, depending on context, may refer to "distribution package". In PyPA documentation, it should only be used in the "import package" sense.

As far as disambiguation when speaking goes, perhaps we should include a pronunciation guide in the tutorial or glossary, and explicitly give the reason why the pronunciation matters? I always say pie-pee-eye myself, specifically due to pie-pie being a homonym with PyPy. (I'm also a fan of any extra subtle advertising we can give PyPy, since that's a highly valuable project that's genuinely advancing the state of the art in dynamic language interpreters)

So that would be something like:

• PyPI (Python Packaging Index): said aloud by using its full name, or by spelling out the last two letters, as in "Py-P-I". Alternatives are to abbreviate as the "Packaging Index", or to use the colloquial nickname "the Cheese Shop" (which refers to the name of a Monty Python's Flying Circus sketch). Referring to the packaging index aloud as "Py-Py" is strongly discouraged, as "PyPy" is the name of a popular alternative Python interpreter.
• PyPA (Python Packaging Authority): said aloud by using its full name, or by spelling out the last two letters, as in "Py-P-A". Alternatives are to abbreviate as the "Packaging Authority" or to use the mailing list name where the Python packaging related interoperability standards are discussed, distutils-sig (which refers to the distutils standard library module, created in 1998 as the initial iteration of a standardised source based packaging and distribution system for Python).
• PyPUG (Python Packaging User Guide): said aloud by using its full name, or by pronouncing the second half like the dog breed, as in "Py-pug". Alternatives are to abbreviate as the "Packaging User Guide", or to use the domain name where the guide is hosted, packaging.python.org.

If this sounds acceptable to folks, @cjrh suggested I run a "how you can help improve Python packaging" session at the September Brisbane Python User Group session, and (assuming that happens) this could be a good thing for us to work on contributing.

refactor wheel docs

use the Standard Pypa Docs Template

Mention `python setup.py register`

Probably in http://python-packaging-user-guide.readthedocs.org/en/latest/quickstart.html#build-upload-your-project-to-pypi

mention pip's new automatic download cache

mention pip's new automatic download cache in the "mirrors and caches" section when v1.6 is released.

Recommend a default cross platform virtual environment management tool

(Note: the issue scope broadened considerably over the course of the subsequent discussion. When reading the comments below, note that the early ones were responding to the original issue title and description, which are preserved below)

As you accumulate more virtual environments, it's useful to have a tool to help manage them (creating new ones, listing the ones that already exist, activating them, deactivating them, etc). virtualenvwrapper, pyenv, vex, pew - these kinds of things.

However, as with packaging tools themselves, new users are unlikely to have the knowledge they need to evaluate the trade-offs between the different tools that are available. In particular, the differences between creating a subshell (vex, pew), modifying the current shell session (virtualenvwrapper), and permanently modifying user configuration settings (pyenv) have different trade-offs in terms of the amount of assumed knowledge regarding the operation of the system shell, as well as the chances of inadvertently impacting the operation of other Python applications on the system.

Considering just those four tools, and their complexity and potential for side effects, I'd assess them as follows:

• vex: aims to be shell independent, deliberately simple execution model & UI. Subshell model minimises chance of side-effects.
• pew: aims to be shell independent, aims to provide a full featured execution model & UI (comparable functionality to virtualenvwrapper). Subshell model minimises chance of side-effects.
• virtualenvwrapper, pyenv: shell specific (written in shell code, not Python), which allows tighter integration with the shell, but also requires that the user have a greater understanding of shell programming (i.e. more than the bare essentials of "enter commands to run programs"). Altering the current shell session and/or user configuration files creates a relatively high chance of side effects on unrelated tools (potentially including system utilities and other Python applications that aren't configured to run in isolated mode).

In that context, I'd suggest that vex would be an appropriate recommendation as a minimal "getting started with environment management" solution. If a given user eventually decides they want something more configurable, or more closely integrated with their preferred shell environment, then they may want to explore pew, virtualenvwrapper or pyenv, and the packaging guide could point them in those directions (with some discussion of the associated trade-offs).

Outside the Python package management space, conda then takes the step up to being a full cross-platform platform, managing the Python runtime and arbitrary external dependencies, rather than just Python virtual environments. However, as an all-in-one solution that bundles environment management, dependency management and binary distribution, conda's operating in a different category from the more single-purpose tools being discussed here.

---

Original issue title: Mention "vex" somewhere (at least in the "Other projects" section)

Original issue description:

https://pypi.python.org/pypi/vex is a nice alternative to things like virtualenvwrapper that I discovered via the PyCon New Zealand lightning talks.

It works by creating new subshells, rather than trying to alter the existing one. So deactivation is just a matter of exiting from the subshell.

This is really nice, because it's shell independent, and provides all the niceties of an integrated command line application, rather than a collection of shell functions.

document `--universal` flag

upon the new wheel release, document the --universal flag, and that "bdist_wheel" is the canonical setup.cfg section to use, not "wheel"

see https://bitbucket.org/pypa/wheel/pull-request/32

also, log an issue to pythonwheels.com to change it's docs.

Please explain a good way to handle version numbers

Most python project handle their own version number in a different way.

Goal: A common way to define it once, use it several times.

There is a solution in the example setup.py (find_version()), but please tell us something in the docs about it in this paragraph:

https://python-packaging-user-guide.readthedocs.org/en/latest/tutorial.html#version

Related:

• http://stackoverflow.com/questions/458550/standard-way-to-embed-version-into-python-package
• http://stackoverflow.com/questions/16465490/python-package-how-to-avoid-redefining-author-version-etc

how to update and upgrade a project on pypi

It wasn't clear to me from this tutorial how to upgrade a project on PyPI to a new version:
https://packaging.python.org/en/latest/distributing.html#uploading-your-project-to-pypi

How to build packages with c / c++ code

I would like to write something about it like I already did here.

Where should I put it in the hierarchy?

Get a python.org subdomain

packaging.python.org?

---

migrated from https://bitbucket.org/pypa/python-packaging-user-guide/issue/42

Document the license argument to setup.py

It would be good to document permissible values for the "license" argument to setup.py and how it is used: https://python-packaging-user-guide.readthedocs.org/en/latest/tutorial.html#metadata

Tutorial page: "package" needs clarification

Regarding page: https://packaging.python.org/en/latest/tutorial.html

[Edit: Preview isn't working, and I can't figure out how to either use or suppress github's auto-numbering... grrr]

ONE. the word "package" appears in the narrative over 30 times (and many more times in the source code). In a significant proportion of cases, it's difficult to tell (particularly given that this is a tutorial) which meaning of package is intended, and indeed the article intermingles mentions of both. Similar fuzziness with the relationship between "package" and "project".

I suggest that every instance of "package" be replaced with a term which explicitly disambiguates between the two meanings. Possibly "distribution package" and "module package". (I see that the glossary has two clearly distinct entries, but doesn't recommend distinct terms by which clear discussion might be enabled.)

TWO. In section "Creating your own Project", there's a subhead "Layout":

TWO-A. It's not clear what, in the subsequent narrative, is talking about "layout". "Layout" could mean directory organization, but directories are not mentioned.

TWO-B. However "a single top-level package" is mentioned: package meaning#1 or package meaning#2? Top-level of what? Can there be multiple or nested packages? Etc.

TWO-C. The 'single top-level package' link points to a sample on github in which the __init__.py file which contains the program main() function. Will a tutorial reader really understand what this is about, coming from virtually any other book or tutorial in which their entry function is in myprogram.py? This may be a way to go about things, or possibly a really good way, but it needs significantly more intro to explain why this obtuseness is a good idea, what's the downstream impact and so on, I think.

THREE. Glossary entry for Package Meaning #1 needs revision since PEP 420 does away with the requirement to mark module packages with __init__.py.

Add an Example Deployment Strategies section

The section would give people a launching-off point for solving their own deployment needs. I envision covering 3 types of deployments, but always using a venv and unpacking everything:

1. Always pull from PyPI (e.g. push a requirements file and then running pip on it)
2. Pulling from a private wheel repo (same as above but not hitting PyPI to save on speed/bandwidth)
3. cp a single wheel/zipfile that is either unzipped in-place or pip installs (e.g. what Twitter does to deploy code to not even have to reach across the network to a single repo server)

That covers the gamut of needs from a small deployment to internet-scale. All solutions should be cross-platform (e.g. avoid symlinks) so as to keep them simple and avoid the caveats.