tox-dev / tox Goto Github PK
View Code? Open in Web Editor NEWCommand line driven CI frontend and development task automation tool.
Home Page: https://tox.wiki
License: MIT License
Command line driven CI frontend and development task automation tool.
Home Page: https://tox.wiki
License: MIT License
I accidentally tried to run tox on a project who's setup.py was an empty file, the error message I got was obscure: http://paste.pocoo.org/show/399000/
To reproduce:
virtualenv /tmp/tox-1.3
cd /tmp/tox-1.3
source bin/activate
wget http://pypi.python.org/packages/source/t/tox/tox-1.3.zip
unzip tox-1.3.zip
cd tox-1.3
python setup.py develop
py.test -x
Output:
...
% py.test -x
============================================= test session starts ==============================================
platform linux2 -- Python 2.6.6 -- pytest-2.0.3
tox comes from: '/home/npinto/venv/tox-1.3/tox-1.3/tox/__init__.pyc'
collected 108 items
tests/test_config.py ............................................................
tests/test_venv.py ......................E
==================================================== ERRORS ====================================================
_______________________________ ERROR at teardown of TestVenvTest.test_patchPATH _______________________________
self = {'_': '/usr/bin/py.test', 'MKL_NUM_THREADS': '8', 'WINDOWID': '16777219', 'HISTFILE': '/home/npinto/.history', 'LESS':...ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:', 'PAGER': 'less', 'USER': 'npinto'}
key = 'PIP_REQUIRE_VIRTUALENV'
def __delitem__(self, key):
unsetenv(key)
> del self.data[key]
E KeyError: 'PIP_REQUIRE_VIRTUALENV'
/usr/lib64/python2.6/os.py:496: KeyError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================== 82 passed, 1 error in 1.57 seconds ======================================
[1] 31350 exit 2 py.test -x
VERSIONS: tox 1.0, tox 1.2, virtualenv 1.6.4
OS: MACOSX 10.6, 10.7
PRECONDITION: py2x python interpreter is not the default python interpreter.
PRECONDITION: virtualenv for py2X is not created yet
The creation of the virtualenv environment fails sometimes. Currently, tox uses the following syntax (which uses virtualenv.py from the default-python site-packages directory):
/absolute-path/to/python2.X /absolute-path/to/python/lib/site-packages/virtualenv.py ... py2X ...
SHOULD-BE:
/absolute-path/to/python2.X /absolute-path/to/python2.X/lib/site-packages/virtualenv.py ... py2X ...
FAILING EXAMPLE:
Python 2.6 is the default python interpreter, py27 should be created on MACOSX. A readline package was installed where virtualenv.py was located. This lead (for me) to an assertion error in the readline package, which caused the virtualenv creation to fail.
NOTE:
After I removed readline.py/.pyc, everything worked again. But I assume there exist other cases. In addition, I think the SHOULD-BE case is slightly more correct. But it assumes that virtualenv is installed for all Python versions under test.
It is a sad fact that many packages have a setup.py which imports other packages. This is sub-optimal, but there is unfortunately no way to avoid it given the current state of python packaging tools. For example, any package which uses the Numpy C API has to import numpy in its setup.py in order to ask it for build information. So, for example, the packages 'scipy' and 'pandas' both have a setup-time requirement that numpy is already installed.
The official word from the 'pip' designers is that the only way to install such packages is by running multiple pip invocations: pypa/pip#25
That is, this does not (and will never) work: pip install numpy scipy pandas
This does work, and is the only supported method: pip install numpy; pip install scipy; pip install pandas
My package that I want to test with tox depends on these other packages. But when I try to run tox, it always generates the first form (which doesn't work), and there's no way to tell it to generate the second form (which does). The result is that tox cannot produce working virtualenvs for testing.
AFAICT the only advantage of the first form is that if there is some problem with the packages, pip can notice earlier and avoid trashing your python install. However, tox only ever installs into throwaway virtualenvs, so this is not really an advantage. Therefore I'd suggest that tox always and unconditionally process the deps= option by calling 'pip install' separately for each entry, and in the order they are listed in the config file.
I have a bash script inside my project which I would like to run before tests… However, the obvious way of running it:
commands=
./reset
nosetests
Causes tox to complain:
[TOX] WARNING:test command found but not installed in testenv
cmd: /usr/bin/reset
env: …/.tox/py27
Maybe forgot to specify a dependency?
I'm working around this for now using:
commands=
bash ./reset
nosetests
But it would be nice if that wasn't necessary.
Hello,
Tox 1.1 fails with UnicodeDecodeError, when GCC throws a warning message. For example:
"src/tpm/delta_UT.c:15:14: warning: ‘rcsid’ defined but not used".
I can run pip-3.2 (versions 1.0.1 and 1.0.2) outside Tox and the same warning message doesn't cause any trouble.
The same error occurs when using Tox installed on Python 2.6 and on Python 3.2.
I am attaching the last few lines from the Tox log.
Thanks,
Prasanth
Last several lines from tox/log/1.log:
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/pip-sienco-build/src/tpm -I/home/phn/bin/python32/include/python3.2m -c src/tpm/delta_UT.c -o build/temp.linux-i686-3.2/src/tpm/delta_UT.o
Exception:
Traceback (most recent call last):
File "/home/phn/tmp/tox/py32/lib/python3.2/site-packages/pip-1.0.2-py3.2.egg/pip/basecommand.py", line 126, in main
self.run(options, args)
File "/home/phn/tmp/tox/py32/lib/python3.2/site-packages/pip-1.0.2-py3.2.egg/pip/commands/install.py", line 228, in run
requirement_set.install(install_options, global_options)
File "/home/phn/tmp/tox/py32/lib/python3.2/site-packages/pip-1.0.2-py3.2.egg/pip/req.py", line 1100, in install
requirement.install(install_options, global_options)
File "/home/phn/tmp/tox/py32/lib/python3.2/site-packages/pip-1.0.2-py3.2.egg/pip/req.py", line 572, in install
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
File "/home/phn/tmp/tox/py32/lib/python3.2/site-packages/pip-1.0.2-py3.2.egg/pip/init.py", line 230, in call_subprocess
line = console_to_str(stdout.readline())
File "/home/phn/tmp/tox/py32/lib/python3.2/site-packages/pip-1.0.2-py3.2.egg/pip/backwardcompat.py", line 60, in console_to_str
return s.decode(console_encoding)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 35: ordinal not in range(128)
This may seem like a corner case, but it's a bit frustrating and seems like it might not be too hard to fix.
I work in a mercurial repo in my Dropbox so I can go from home (on a Mac) to work (on Windows 7) and have my source files automatically sync.
Unfortunately, every time I run tox on one platform, it completely blows away all the files from the other platform and rebuilds everything. This is slow and makes it impossible to do certain convenient things, like set up tools to use the tox virtualenvs (since they keep being deleted!).
It would be nice if tox set up its files/directories so virtualenvs from different platforms could co-exist in the same .tox directory.
Bitbucket: https://bitbucket.org/hpk42/tox/issue/5
Originally reported by: Anonymous
Originally created at: 2011-06-19T12:14:16.894
setup.py}}} unconditionally depends on {{{argparse}}}, which is included as of Python 2.7 and 3.2. In order to package tox cleanly, this dependency has to be patched out if the system python is 2.7 or 3.2.
{{{setup.py
The attached changeset includes the argparse
dependency only when required (on Python < 2.6 or on Python 3.0 and 3.1).
tox 1.1 doesn't require virtualenv anymore. Instead, it has inlined the virtualenv.py (1.6.1) script. Unfortunately, the directory virtualenv_support (with distribute and pip distributions) is not (yet) inlined. Therefore, working offline is not possible.
Running tox on Windows 7 with Python 3.3b1 installed at c:\Python3.3, tox simply reports:
ERROR: py33: InterpreterNotFound: python3.3
I notice in _venv.py is win32map, which is an explicit mapping from versions to installations, but doesn't include 3.3. Perhaps this mapping could be generalized?
At the bottom of http://tox.testrun.org/latest/example/pytest.html, there is a link pointing to
http://codespeak.net/py/dist/test/customize.html#constructing-the-package-name-for-test-modules
which does not exist. I looked around but couldn't figure out where it is supposed to be pointing at.
BR
Florian
it would be nice if one could select environments based on glob-style-patterns, like in e.g.
tox -e py26-*
which would select all environments matching that pattern...
Bitbucket: https://bitbucket.org/hpk42/tox/issue/14
Originally reported by: @ssbarnea
Originally created at: 2011-10-07T13:34:20.310
[tox testenv:py25] ___________________________________________________
[TOX] ***creating virtualenv py25
[TOX] C:\Automation Scripts\Scripts\Sample\Sorin\tendo.tox$ c:\python25\python.exe ............\Python25\lib\site-packages\tox-1.1-py2.5.egg\tox\virtualenv.py --distribute --no-site-packages py25 >py25\log\0.log
[TOX] ERROR: invocation failed, logfile: C:\Automation Scripts\Scripts\Sample\Sorin\tendo.tox\py25\log\0.log
[TOX] ERROR: C:\Automation Scripts\Scripts\Sample\Sorin\tendo.tox$ c:\python25\python.exe ............\Python25\lib\site-packages\tox-1.1-py2.5.egg\tox\virtualenv.py --distribute --no-site-packages py25 >py25\log\0.log
c:\python25\python.exe: can't open file '............\Python25\lib\site-packages\tox-1.1-py2.5.egg\tox\virtualenv.py': [Errno 2] No such file or directory
[TOX] ERROR: InvocationError: c:\python25\python.exe ............\Python25\lib\site-packages\tox-1.1-py2.5.egg\tox\virtualenv.py --distribute --no-site-packages py25 (see C:\Automation Scripts\Scripts\Sample\Sorin\tendo.tox\py25\log\0.log)
0.log content:
C:\Automation Scripts\Scripts\Sample\Sorin\tendo\.tox$ c:\python25\python.exe ..\..\..\..\..\..\Python25\lib\site-packages\tox-1.1-py2.5.egg\tox\virtualenv.py --distribute --no-site-packages py25 >py25\log\0.log
c:\python25\python.exe: can't open file '..\..\..\..\..\..\Python25\lib\site-packages\tox-1.1-py2.5.egg\tox\virtualenv.py': [Errno 2] No such file or directory
I have no clue how this could happen:
#!python
Traceback (most recent call last):
File "/usr/bin/py.test-2.7", line 9, in <module>
load_entry_point('pytest==2.2.3', 'console_scripts', 'py.test')()
File "/usr/lib64/python2.7/site-packages/_pytest/core.py", line 467, in main
config = _prepareconfig(args, plugins)
File "/usr/lib64/python2.7/site-packages/_pytest/core.py", line 460, in _prepareconfig
pluginmanager=_pluginmanager, args=args)
File "/usr/lib64/python2.7/site-packages/_pytest/core.py", line 419, in __call__
return self._docall(methods, kwargs)
File "/usr/lib64/python2.7/site-packages/_pytest/core.py", line 430, in _docall
res = mc.execute()
File "/usr/lib64/python2.7/site-packages/_pytest/core.py", line 348, in execute
res = method(**kwargs)
File "/usr/lib64/python2.7/site-packages/_pytest/helpconfig.py", line 25, in pytest_cmdline_parse
config = __multicall__.execute()
File "/usr/lib64/python2.7/site-packages/_pytest/core.py", line 348, in execute
res = method(**kwargs)
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 10, in pytest_cmdline_parse
config.parse(args)
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 343, in parse
self._preparse(args)
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 321, in _preparse
self._setinitialconftest(args)
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 300, in _setinitialconftest
self._conftest.setinitial(args)
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 167, in setinitial
self.getconftestmodules(x)
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 188, in getconftestmodules
clist.append(self.importconftest(conftestpath))
File "/usr/lib64/python2.7/site-packages/_pytest/config.py", line 217, in importconftest
self._conftestpath2mod[conftestpath] = mod = conftestpath.pyimport()
File "/usr/lib64/python2.7/site-packages/py/_path/local.py", line 532, in pyimport
__import__(modname)
File "/var/tmp/portage/dev-python/tox-1.4.1/work/tox-1.4.1/tests/conftest.py", line 10, in <module>
from tox._cmdline import Action
ImportError: cannot import name Action
This page:
http://tox.readthedocs.org/en/latest/example/jenkins.html
misses one important step. The Jenkins Python plugin must be installed first. At least, this plugin is not installed by default in the Ubuntu package of Jenkins.
Users should be able to call tox from any sub-directory, and tox should detect the root location based on tox.ini file.
This would make tox accesible just like scm tools: svn hg git
tox 1.4 assumes that bold white text is visible on all terminals. With a terminal configured for dark text on a light background (black on white), this assumption means portions of tox's output is entirely invisible.
For example, messages like "py27 sdist-reinst: …" as tox does its thing to set up my environment before actually running the tests appear as blank lines.
I have worked around this by piping the output of tox through tee so the library that changes the colors assumes it doesn't have a configurable terminal on stdout, but that's a pain. I would like some way to either disable the color output entirely or control the colors so I can see the output of tox.
In this section (http://tox.testrun.org/latest/config.html#substition-for-values-from-other-sections), shouldn't the substitution examples refer to [base] instead of [tools]?
First, thanks for tox. It is very useful.
I have a setuptools-based setup.py with install_requires, setup_requires and tests_require. I was surprised to find that I needed to duplicate the test dependencies in tox.ini; I checked the doc and many examples from projects I follow and finally asked for confirmation on IRC. Ronny suggested that I report this.
IMO, tox should fetch setup_requires and tests_require dependencies too. This would certainly shorten a lot of people’s tox.ini :)
introduce a config option to support pep381 mirroring
pip and easy_install both support --find-links as an alternative to using an index server.
we use --find-link stores at my company to store our local distributions.
as it is, to use tox, we have to add the findlink directives to the pip config for the user account running tox.
instead of just adding options to pip/easy_install one by one you could consider just offering an "advanced" option to allow users to construct their own pip/easy_install commands in the config.
If one forgets to specify a dep like "pytest" then the invoking shell's py.test is used and thus the wrong version/wrong interpreter is used.
Hi all,
as described in this e-mail thread http://lists.idyll.org/pipermail/testing-in-python/2012-April/004898.html , tox is currently lacking an easy option to run 2to3 over a test suite that is not part of the package.
Holger suggested adding specific options to tox.ini, but maybe a documentation fix is sufficient. IOW, is there a good (cross-platform) way, to specify these steps in tox.ini:
cp test /test # note cp is spelled copy on windows!
2to3 /test
py.test /test
Cheers,
Florian
This was mentioned on the mailing list (see http://lists.idyll.org/pipermail/testing-in-python/2011-June/004173.html).
Here is an example tox session
#!bash
$tox -v -c tox.ini.test
[TOX] emptying /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/log
[TOX] using tox.ini: /Users/aaronmeurer/Documents/python/sympy/sympy/tox.ini.test
[TOX] using tox-0.9 from /Library/Python/2.6/site-packages/tox-0.9-py2.6.egg/tox/__init__.pyc
______________________________________________________________________________________________ [tox sdist] _______________________________________________________________________________________________
[TOX] ***creating sdist package
[TOX] emptying /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/dist
[TOX] /Users/aaronmeurer/Documents/python/sympy/sympy$ /usr/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
[TOX] ***copying new sdistfile to '/Users/aaronmeurer/.tox/distshare/sympy-0.6.7-git.zip'
___________________________________________________________________________________________ [tox testenv:py27] ___________________________________________________________________________________________
[TOX] ***recreating virtualenv py27 (configchange/incomplete install detected)
[TOX] emptying /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/py27
[TOX] /Users/aaronmeurer/Documents/python/sympy/sympy/.tox$ virtualenv --distribute --no-site-packages -p /sw/bin/python2.7 py27 >py27/log/0.log
[TOX] ***installing dependencies: notarealdep
[TOX] /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/py27/log$ ../bin/pip install --download-cache=/Users/aaronmeurer/Documents/python/sympy/sympy/.tox/_download notarealdep >1.log
[TOX] ERROR: invocation failed, logfile: /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/py27/log/1.log
[TOX] ERROR: /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/py27/log$ ../bin/pip install --download-cache=/Users/aaronmeurer/Documents/python/sympy/sympy/.tox/_download notarealdep >1.log
Downloading/unpacking notarealdep
Could not find any downloads that satisfy the requirement notarealdep
No distributions at all found for notarealdep
Storing complete log in /Users/aaronmeurer/.pip/pip.log
[TOX] ERROR: could not install deps [notarealdep]
_____________________________________________________________________________________________ [tox summary] ______________________________________________________________________________________________
[TOX] ERROR: py27: could not install deps [notarealdep]
Aaron-Meurer:sympy aaronmeurer(master$%=)$tox -v -c tox.ini.test
[TOX] emptying /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/log
[TOX] using tox.ini: /Users/aaronmeurer/Documents/python/sympy/sympy/tox.ini.test
[TOX] using tox-0.9 from /Library/Python/2.6/site-packages/tox-0.9-py2.6.egg/tox/__init__.pyc
______________________________________________________________________________________________ [tox sdist] _______________________________________________________________________________________________
[TOX] ***creating sdist package
[TOX] emptying /Users/aaronmeurer/Documents/python/sympy/sympy/.tox/dist
[TOX] /Users/aaronmeurer/Documents/python/sympy/sympy$ /usr/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
[TOX] ***copying new sdistfile to '/Users/aaronmeurer/.tox/distshare/sympy-0.7.0.rc2.zip'
___________________________________________________________________________________________ [tox testenv:py27] ___________________________________________________________________________________________
[TOX] ***reusing existing matching virtualenv py27
[TOX] $ ../bin/pip install --download-cache=/Users/aaronmeurer/Documents/python/sympy/sympy/.tox/_download ../../dist/sympy-0.7.0.rc2.zip >2.log
_____________________________________________________________________________________________ [tox summary] ______________________________________________________________________________________________
[TOX] py27: commands succeeded
[TOX] congratulations :)
$cat tox.ini.test
[tox]
envlist = py27
[testenv:py27]
deps = notarealdep
Notice how even though installing the dependency fails, it runs without any problems the second time around. This happens when there is any kind of error installing a dependency, not just downloading.
Running tox on Mac OS X Lion causes this error.
#!shell
$ cat .tox/py27/log/0.log
/Volumes/data/Projects/mom/.tox$ /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 ../../../../../Library/Python/2.7/site-packages/tox/virtualenv.py --distribute --no-site-packages py27 >py27/log/0.log
install_name_tool: for architecture cputype (16777223) cpusubtype (-2147483645) object: py27/bin/python malformed object (unknown load command 4)
New python executable in py27/bin/python
Could not call install_name_tool -- you must have Apple's development tools installed
Traceback (most recent call last):
File "../../../../../Library/Python/2.7/site-packages/tox/virtualenv.py", line 1922, in <module>
main()
File "../../../../../Library/Python/2.7/site-packages/tox/virtualenv.py", line 796, in main
never_download=options.never_download)
File "../../../../../Library/Python/2.7/site-packages/tox/virtualenv.py", line 887, in create_environment
site_packages=site_packages, clear=clear))
File "../../../../../Library/Python/2.7/site-packages/tox/virtualenv.py", line 1151, in install_python
py_executable])
File "../../../../../Library/Python/2.7/site-packages/tox/virtualenv.py", line 864, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command install_name_tool -change /System/Library/Fram.../Versions/2.7/Python @executable_path/../.Python py27/bin/python failed with error code 1
Please take a look at https://423859.bugs.gentoo.org/attachment.cgi?id=316467 and see if the related tests can be fixed.
it's reasonable to want to maintain both install_requires and a requirements.txt. http://stackoverflow.com/a/7085000
if you're doing that, there seems to be no way to lay down the requirements first before your distribution gets installed, which is what you want.
I see a number of people adding "pip install -r requirements.txt" lines into the the test command option, but that's happening after the distribution is installed.
It appears that tox (or at least the test phase) also depends on pytest and pip. Please make this explicit in the setup.py (and preferably also in the installation docs) to make life easier for packagers.
The internal config.getargvlist does not implement cross-section substitution. Example of failure:
[testenv]
commands = py.test
[testenv:2]
commands = {[testenv]commands} -n3
This will bail out on configuration parsing already when you call "tox".
The ‘Basic Example’ on the home-page is misleading because it does not specify py.test as a dependency.
$ export PYTHONHOME=/opt/local/Library/Frameworks/Python.framework/Versions/2.7
$ export PATH=$PYTHONHOME/bin:$PATH
$ export DYLD_LIBRARY_PATH=$PYTHONHOME/lib:$DYLD_LIBRARY_PATH
$ tox -e py32
[...]
[TOX] ERROR: invocation failed, logfile: /Users/olivier/Workspace/src/python-nose/.tox/py32/log/0.log
[TOX] ERROR: /Users/olivier/Workspace/src/python-nose/.tox$ /opt/local/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2 ../../../../../../opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tox-1.1-py2.7.egg/tox/virtualenv.py --no-site-packages py32 >py32/log/0.log
Fatal Python error: Py_Initialize: unable to load the file system codec
LookupError: no codec search functions registered: can't find encoding
[TOX] ERROR: InvocationError: /opt/local/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2 ../../../../../../opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tox-1.1-py2.7.egg/tox/virtualenv.py --no-site-packages py32 (see /Users/olivier/Workspace/src/python-nose/.tox/py32/log/0.log)
$ unset PYTHONHOME
$ tox -e py32
[...]
[TOX] py32: commands succeeded
[TOX] congratulations :)
there are various setups, where install of a sdist for testing may be unwanted, or something like a setup.py might not even exist
yet one would want to run something like py.test in there in different envs
In virtualenv 1.7.1.2 the option --no-site-packages is the default and deprecated. The option --system-site-packages has to be given explicitly. Currently tox only adds or omits the deprecated --no-site-packages option. If sitepackages=True this seems to be insufficient.
In some cases (I think I know the case), tox will install the pip requirements into whatever virtual environment was active at the time of the tox command. Here's an example of what that looks like:
#!bash
~/dev/project/www$ mkvirtualenv test
~/dev/project/www$ workon test
(test)~/dev/project/www$ pip freeze
tox==1.2
(test)~/dev/project/www$ tox
...
(test)~/dev/project/www$ pip freeze
eautifulSoup==3.2.0
Django==1.3.1
Fabric==1.3.2
...
tox==1.2
unittest2==0.5.1
virtualenv==1.7
wsgiref==0.1.2
(test)~/dev/project/www$
It seems that I had something weird setup with my PIP environment variables.
#!bash
(test)~/dev/project/www$ env | grep VIR
PIP_REQUIRE_VIRTUALENV=true
VIRTUAL_ENV=/Users/streeter/.virtualenvs/test
VIRTUALENVWRAPPER_HOOK_DIR=/Users/streeter/.virtualenvs
PIP_VIRTUALENV_BASE=/Users/streeter/.virtualenvs
PIP_RESPECT_VIRTUALENV=true
VIRTUALENVWRAPPER_LOG_DIR=/Users/streeter/.virtualenvs
It seems that the PIP_REQUIRE_VIRTUALENV was throwing off the pip install, forcing the pip used by the tox installed venv to use the activated venv. Removing that environment variable caused it to work. Here's the output of several commands: http://pastie.org/2992756
I tried installing PyPy for the first time to add a pypy test environment. I used MacPorts to install PyPy, but tox wasn't able to find it. It looks like MacPorts expects you to invoke PyPy using "pypy-c".
I'd like my tox configuration to be usable by everyone since it's part of a public repo, so I don't want to include something non-standard in the config file. What is the correct way to handle this? Perhaps tox can look for pypy in the pypy-c location on its own? Thanks.
Running the test suite fails on systems that do not have older versions of Python installed (2.5, 2.6, etc.). It would be nice if the tox tests took this into account so that developers adding new features can run the test suite without having the older versions.
In certain circumstances it would be very useful to override the setting of toxworkdir. For instance, in a CI environment such as Bamboo where each job of a build plan is run in it's own directory, it would be very useful to have toxworkdir point somewhere outside of the individual job directories in order to avoid repeated rebuilds of the virtual environments when the package list changes. Having toxworkdir be overridable means that the setting in tox.ini could be used as a default which is then overridden in the CI environment.
I had two dependencies, flexmock and argparse, and putting them as
deps = flexmock, argparse
failed. (Note that I am testing with py26 and py27, so the recent fix against argparse might be the culprit.)
If dependencies should be on multiple lines, it would be better to explicitly specify in the docs.
Thanks,
Ustun
The official way to run the numpy testsuite is:
python -c 'import numpy; numpy.test()'
However, if I put that in a commands= entry, then the ; is treated as starting a comment, and there is no possible way to escape it. I guess technically this is a py.iniparser bug, but there it is.
tox should have support for multiple PyPy versions, similar to multiple CPython versions, using environment names like pypy16
, pypy17
.
Sometimes, when running tests in virtualenvs in Jenkins, you can get screwed by the fact that PyPI is about as reliable as a drunk cat. So, if your Jenkins virtualenv based system needs to be reliable, having a job that pre-creates the virtualenvs and then archives them so that the real jobs can use the "last good venv" is important. Problem is, to do that right, you have to wind up moving the virtualenvs- which means after creating them, you have to make them relocatable:
virtualenv --relocatable .venv
Then when you unpack your tarball, you have to fix the things that virtualenv --relocatable didn't do (shown here as command line perl - sorry, it's easier to do this way in our script)
perl -MCwd -ple '$CWD=Cwd::abs_path();s,^VIRTUAL_ENV=._,VIRTUAL_ENV="$CWD/.venv",' -i .venv/bin/activate perl -MCwd -ple '$CWD=Cwd::abs_path();s,^/._/.venv/src/(._),$CWD/.venv/src/$1,' -i .venv/lib/python2.7/site-packages/easy-install.pth perl -MCwd -ple '$CWD=Cwd::abs_path();s,^/._/workspace/._,$CWD,' -i .venv/lib/python2.7/site-packages/_.pth
It would be STELLAR if tox just knew how to be resilient in the case of me wanting to tar up an entire .tox dir and drop it elsewhere.
I erroneously filed this against pytest earlier: pytest-dev/pytest#107 for the discussion there.
ci servers that run on win32 don't have a assigned homedir in the env
it makes sense to avoid using it there or generally making the substitution lazily calculated
Currently 0.6.19 is required, but 0.6.21 has been released upstream. If tox can't be made to use whatever the latest is, it should at least be updated to 0.6.21.
One of us will add an issue I guess, but here's the general gist of what we're trying to solve.
First of all, here's an example tox.ini file:
https://github.com/openstack/python-glanceclient/blob/master/tox.ini
The OpenStack CI infrastructure is an automated gated trunk. That means that we push changes up to a gerrit instance code review, and then if they are approved, Jenkins grabs the changes, tests them, and if they pass tests it submits a message back to Gerrit which causes the change to be merged. We run all of these tests in clean environments, and for unittests and pep8 checks, we use tox to manage the creation of virtualenvs and the running of tests in those environments.
https://jenkins.openstack.org/view/Gate/job/gate-python-glanceclient-python26/configure
https://jenkins.openstack.org/view/Gate/job/gate-python-glanceclient-python27/configure
https://jenkins.openstack.org/view/Gate/job/gate-python-glanceclient-pep8/configure
Perfect, right?
Problem is, we test somewhere around 200 changes a day all hands off, so things like transient network problems between our build hosts and PyPI (which happen often) become a major impediment - even if all we're talking about is the index look up for the dep name.
So as an optimization, we have an out-of-band job which runs pip bundle on our deps lists:
https://jenkins.openstack.org/view/Gate/job/python-glanceclient-venv/configure
https://github.com/openstack/openstack-ci-puppet/blob/master/modules/jenkins_slave/files/slave_scripts/build-bundle.sh
Installing deps into a virtualenv from a bundle file avoids all network accesses, as all of the index lookups are pre-done. Problem is, there isn't a great way to do conditional deps lists - I can't override deps in tox:jenkins. So we make a bunch of 'fake' environments in our tox.ini file that jenkins can use jenkins26, jenkins27, etc.
This is a bit ugly though, as it encodes specifics of our CI implementation and specifically the caching concerns we have on a system that runs gobs of tests per day that don't apply to any of the individual developers.
You may remember I took a stab at making tox virtualenvs relocatable a while back as an attempt to solve this systemically - turns out that virtualenvs just are not possible to relocate in a sensible manner.
What I REALLY want is a way for tox to understand bundle-based caching, so that I could run a tox command in build-bundle that would produce something I could later drop into a location on a different machine, so that when tox decides it needs to create a virtualenv it could use the discovered bundle file instead of the directly listed deps, at which point I could get rid of the jenkins* envs inside of tox.ini and treat the problem of caching as a step that the CI system needs to do inside of a pipeline of activity.
As a half-way step to that (which I figured would be a much more invasive patch) we wrote the patch to do dep override on the command line, because at least that way we can have jenkins job do things that are specific to jenkins and have nothing to do with a normal use of the repo, like::
tox -epy27 --deps=.cache.bundle
py27 can then simply describe how to run the job, and just because we've already pre-created the dep bundle, we can tell it to use that. That gets us 95% of the way there- we can clean up the tox.ini files in the close to 30 branches we're managing this way and then can treat the further solution to the problem on the CI side.
Does that make sense? Sorry for the novel here.
this is a extension of the current context detection,
which just detects hudon by the hudson url
For the following tox.ini configuration:
#!text
[tox]
envlist =
py25,py26,py27,jython,pypy
[testenv]
commands =
python setup.py test -q
deps =
Sphinx
WebTest
repoze.sphinx.autointerface
virtualenv
[testenv:jython]
commands =
jython setup.py test -q
Running "tox -e pypy" (which finds pypy 1.5a0 on my PATH), produces this:
#!text
[chrism@thinko pyramid]$ tox -e pypy
_________________________________ [tox sdist] __________________________________
[TOX] ***creating sdist package
[TOX] /home/chrism/projects/pyramid$ /home/chrism/opt/Python-2.6.6/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
[TOX] ***copying new sdistfile to '/home/chrism/.tox/distshare/pyramid-1.1b2.zip'
______________________________ [tox testenv:pypy] ______________________________
[TOX] ***creating virtualenv pypy
[TOX] /home/chrism/projects/pyramid/.tox$ /home/chrism/projects/pypy/pypy/translator/goal/pypy-c ../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py --distribute --no-site-packages pypy >pypy/log/0.log
[TOX] ERROR: invocation failed, logfile: /home/chrism/projects/pyramid/.tox/pypy/log/0.log
[TOX] ERROR: /home/chrism/projects/pyramid/.tox$ /home/chrism/projects/pypy/pypy/translator/goal/pypy-c ../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py --distribute --no-site-packages pypy >pypy/log/0.log
New pypy executable in pypy/bin/pypy-c
Also creating executable in pypy/bin/pypy
Installing distribute........
Complete output from command /home/chrism/project....tox/pypy/bin/pypy-c -c "#!python
\"\"\"Bootstra... main(sys.argv[1:])
" --always-copy -U distribute:
Traceback (most recent call last):
File "app_main.py", line 53, in run_toplevel
File "app_main.py", line 506, in run_it
File "<string>", line 20, in <module>
ImportError: No module named tempfile
----------------------------------------
...Installing distribute...done.
Traceback (most recent call last):
File "app_main.py", line 53, in run_toplevel
File "../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py", line 1922, in <module>
main()
File "../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py", line 796, in main
never_download=options.never_download)
File "../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py", line 893, in create_environment
search_dirs=search_dirs, never_download=never_download)
File "../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py", line 594, in install_distribute
search_dirs=search_dirs, never_download=never_download)
File "../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py", line 563, in _install_req
cwd=cwd)
File "../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py", line 864, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /home/chrism/project....tox/pypy/bin/pypy-c -c "#!python
\"\"\"Bootstra... main(sys.argv[1:])
" --always-copy -U distribute failed with error code 1
[TOX] ERROR: InvocationError: /home/chrism/projects/pypy/pypy/translator/goal/pypy-c ../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py --distribute --no-site-packages pypy (see /home/chrism/projects/pyramid/.tox/pypy/log/0.log)
________________________________ [tox summary] _________________________________
[TOX] ERROR: pypy: InvocationError: /home/chrism/projects/pypy/pypy/translator/goal/pypy-c ../../../opt/Python-2.6.6/lib/python2.6/site-packages/tox-1.1-py2.6.egg/tox/virtualenv.py --distribute --no-site-packages pypy (see /home/chrism/projects/pyramid/.tox/pypy/log/0.log)
On my Windows box I have pypy.exe in the path (i.e. typing pypy
at the command line works), but if I add pypy
to the tox envlist, I get
ERROR: InterpreterNotFound: pypy
Specifying "<" or ">" deps on windows leads to invocation errors. See also the attached example project.
Is it possible to pass arguments to setup.py? I couldn't see how. Additionally, I might like to do this in a platform-specific way.
Specifically, my use case is this (from a mailing list post):
I'd like to build and test NumPy with Tox. On linux, this works fine (just "tox" with a standard tox.ini). However, on my Windows machine I need to pass "--compiler=mingw32" to NumPy's setup.py.
Thanks!
Using tox 1.4.2 and python 2.7 on OSX I get
ERROR: unknown environment 'py32 # Python 3.1 and 3.0 might work as well.'
when running tox with the toxfile here: https://github.com/erikrose/more-itertools/blob/c0339068f094933504d095bb0d2c40577acf7c56/tox.ini
Moving the comment to a different line fixes the issue.
[update by @obestwalter to clarify the problem]
Example, comments like this will make tox fail, because it does not interpret:
[tox]
skipsdist = True # THIS IS NO COMMENT FOR TOX!
envlist = standard-greeting
[testenv:standard-greeting]
deps =
plumbum # THIS NEITHER!
flask
commands =
python -c 'print("Hello, world!")' # AS YOU SEE IT'S UNIVERSAL!
output:
Traceback (most recent call last):
File "/home/oliver/.virtualenvs/tox/bin/tox", line 11, in <module>
load_entry_point('tox', 'console_scripts', 'tox')()
File "/home/oliver/work/tox/tox/tox/session.py", line 38, in main
config = prepare(args)
File "/home/oliver/work/tox/tox/tox/session.py", line 26, in prepare
config = parseconfig(args)
File "/home/oliver/work/tox/tox/tox/config.py", line 242, in parseconfig
parseini(config, inipath)
File "/home/oliver/work/tox/tox/tox/config.py", line 807, in __init__
config.skipsdist = reader.getbool("skipsdist", all_develop)
File "/home/oliver/work/tox/tox/tox/config.py", line 998, in getbool
"boolean value %r needs to be 'True' or 'False'")
tox.ConfigError: ConfigError: boolean value %r needs to be 'True' or 'False'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.