kevinconway / venvctrl

Python API for interacting with virtual environments.

venvctrl

Python API for interacting with virtual environments.

Why?

The only real interface provided by a virtual environment is the CLI. This package wraps the command line tools to provide a more friendly programmatic interface.

Creating A Venv

from venvctrl import api

api.VirtualEnvironment('/some/path/here').create()

Installing Packages

from venvctrl import api

venv = api.VirtualEnvironment('/some/existing/venv')
if not venv.has_package('venvctrl'):

    venv.install_package('venvctrl')

else:

    venv.uninstall_package('venvctrl')

Moving A Venv

from venvctrl import api

venv = api.VirtualEnvironment('/some/existing/venv')
venv.move('/some/new/path')

Alternatively, use the CLI.

venvctrl-relocate \
    --source=/some/existing/venv \
    --destination=/some/new/path \
    --move

License

(MIT License)

Copyright (C) 2015 Kevin Conway

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.

Contributing

All contributions to this project are protected under the agreement found in the CONTRIBUTING file. All contributors should read the agreement but, as a summary:

You give us the rights to maintain and distribute your code and we promise
to maintain an open source distribution of anything you contribute.

venvctrl's Issues

Dependency on virtualenv not declared

setup.* should declare a dependency on virtualenv

venvctrl 0.4.0 sometimes fails in venvctrl-relocate

Hi Kevin, thanks very much for making rpmvenv available, it's a really good tool.

We had our builds break recently after you released venvctrl 0.4.0 (error messages below). This failed on our Jenkins server but work on another server I build on, so I can't explain how to reproduce it. Forcing install of venvctrl==0.3.0 appears to correct the issue.

The failure is in venvctrl-relocate. Error looks like this:

• venvctrl-relocate --source=/tmp/rpmvenvCUXWSz/BUILDROOT/spoken-sbc-api-mpact-3.0.0-rc.41472.x86_64//usr/local/spoken/sbc-api-mpact/env --destination=//usr/local/spoken/sbc-api-mpact/env
  Traceback (most recent call last):
  File "/usr/bin/venvctrl-relocate", line 11, in
  sys.exit(main())
  File "/usr/lib/python2.7/site-packages/venvctrl/cli/relocate.py", line 52, in main
  relocate(args.source, args.destination, args.move)
  File "/usr/lib/python2.7/site-packages/venvctrl/cli/relocate.py", line 24, in relocate
  venv.relocate(destination)
  File "/usr/lib/python2.7/site-packages/venvctrl/venv/relocate.py", line 75, in relocate
  next_dir = dirs.pop()
  IndexError: pop from empty list
  error: Bad exit status from /var/tmp/rpm-tmp.P0sH7s (%install)

Thanks,
Kevin

Python 3.12 CLI relocate : 'NoneType' object has no attribute 'replace'

Hi there, when I try to run this command on Python 3.12 : python -m venvctrl.cli.relocate --source=source_dir --destination=dest_dir, I get this traceback :

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "python3.12/site-packages/venvctrl/cli/relocate.py", line 57, in <module>
    main()
  File "python3.12/site-packages/venvctrl/cli/relocate.py", line 52, in main
    relocate(args.source, args.destination, args.move)
  File "python3.12/site-packages/venvctrl/cli/relocate.py", line 24, in relocate
    venv.relocate(destination)
  File "python3.12/site-packages/venvctrl/venv/relocate.py", line 28, in relocate
     activate.vpath = destination
     ^^^^^^^^^
  File "python3.12/site-packages/venvctrl/venv/base.py", line 240, in vpath
    new_line = old_line.replace(old_vpath, new_vpath)
                      ^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'replace'

It seems to appear only in Python3.12, I try the same build in Python3.9 and Python3.11 and it works without any problems. Any ideas ?

Nushell activation scripts are not rewritten when relocated

virtualenv recently added support for nushell (https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst#v2080-2021-09-16). The new activation scripts are not modified when the virtualenv is relocated.

Adding new activation scripts is usually a straightforward, three step process:

• Create a new sublcass of ActivateFile in https://github.com/kevinconway/venvctrl/blob/master/venvctrl/venv/base.py#L239-L260 that provides the patterns needed to find and replace the file path of the virtualenv.
• Create a new property on BinDir in https://github.com/kevinconway/venvctrl/blob/master/venvctrl/venv/base.py#L281-L299 that returns the new ActivateFile subclass.
• Add the new property to the set in https://github.com/kevinconway/venvctrl/blob/master/venvctrl/venv/base.py#L267-L279.

The bulk of the work is figuring out the regex required to match the path value in the new script. The rest of the work is minimal and the existing tests will validate the changes.

Failing to rewrite nushell activation script

A recent update to virtualenv changed the nushell activation script in a way that breaks the current regex matching for paths. This is causing build errors for rpmvenv. The new format looks like:

    let virtual_env = '/tmp/rpmvenvz_kldppd/BUILDROOT/test-pkg-1.2.3.4-1.x86_64/usr/share/python/test-pkg-venv'

The biggest differences seem to be 1) virtual-env is renamed to virtual_env and 2) there are now spaces in front of the line.

and the deactivate line is also changed to:

export alias deactivate = overlay hide activate

The following need to be adjusted to work with the latest nushell script changes: https://github.com/kevinconway/venvctrl/blob/master/venvctrl/venv/base.py#L263-L293. However, they also need to handle the old style of scripts so that existing usage of venvctrl doesn't break.

test_pip fails in an offline VM

When building venvctrl in a rpmbuild offline VM, test_pip fails because it requires ability to install confpy via pip, which by default requires PyPI access.

This could be solved by installing a small dummy wheel which is kept in the source repo, or create-able from the source repo.

The second half of test_pip installs the packages found in https://github.com/kevinconway/venvctrl/blob/master/requirements.txt , which is currently empty. This seems like a no-op atm.

Another approach is to enable system-packages in the virtualenv , so that pip can find these packages are already installed.

venvctrl relocate doesn't handle exec

When building a rpm with rpmvenv that has a very long long path name, files in /usr/bin/ have an exec added to the top of the python files rather then a python shebang, see example of the top /usr/bin/circusd:

#!/bin/sh
'''exec' /tmp/rpmvenvN0vF_Y/BUILDROOT/python-circus-0.14.0-10.el6.company_name.x86_64/usr/share/python-circus/python_circus_venv/bin/python "$0" "$@"
' '''

vencctrl doesn't relocate the files when they have an exec and the rpmvenv build will complete 'successfully'. Preferred behaviour would be to handle exec or raise an exception with a detailed error message when exec is found.

Missing tags and tests

https://github.com/kevinconway/venvctrl/tags doesnt have 0.4.2

PyPI sdist doesnt include tests.

Combined that means there is no (reasonable) way for me to download a 0.4.2 with tests.

And so I cant add tests to https://build.opensuse.org/package/show/home:jayvdb:py-new/python-venvctrl , and so cant submit it to openSUSE.

I would prefer sdists contains tests, but tags are the simpler solution.

Relocate broken

Venv commit: python/cpython@ebc8103

base.py has

class ActivateFile(BinFile):
    read_pattern = re.compile(r"""^VIRTUAL_ENV=["'](.*)["']$""")

As new activation script is:

# on Windows, a path can contain colons and backslashes and has to be converted:
if [ "$OSTYPE" = "cygwin" ] || [ "$OSTYPE" = "msys" ] ; then
    # transform D:\path\to\venv to /d/path/to/venv on MSYS
    # and to /cygdrive/d/path/to/venv on Cygwin
    export VIRTUAL_ENV=$(cygpath "__VENV_DIR__")
else
    # use the path as-is
    export VIRTUAL_ENV="__VENV_DIR__"
fi

Above regexp won't match anything making the entire thing collapse. Also, there are number of new file types, like .fish.

Suggested patch:

--- venv/base.py-orig   2023-12-28 14:54:09.350289838 +0200
+++ venv/base.py        2023-12-28 16:46:10.395259729 +0200
@@ -207,7 +207,10 @@

     """The virtual environment /bin/activate script."""

-    read_pattern = re.compile(r"""^VIRTUAL_ENV=["'](.*)["']$""")
+    read_pattern_pre_312 = re.compile(r"""^VIRTUAL_ENV=["'](.*)["']$""")
+    read_pattern_312 = re.compile(r"""^    export VIRTUAL_ENV=["'](.*)["']$""")
+    read_pattern_312_csh = re.compile(r"""^setenv VIRTUAL_ENV ["'](.*)["']$""")
+    read_pattern_312_fish = re.compile(r"""^set -gx VIRTUAL_ENV ["'](.*)["']$""")

     def _find_vpath(self):
         """
@@ -221,7 +224,16 @@

             for count, line in enumerate(file_handle):

-                match = self.read_pattern.match(line)
+                match = self.read_pattern_pre_312.match(line)
+                if not match:
+                    match = self.read_pattern_312.match(line)
+
+                if not match:
+                    match = self.read_pattern_312_csh.match(line)
+
+                if not match:
+                    match = self.read_pattern_312_fish.match(line)
+
                 if match:

                     return match.group(0), match.group(1), count

Path (.pth) Files Are Not Rewritten

ref: kevinconway/rpmvenv#24

The project does not correctly rewrite .pth files which can contain references to the original build location.