daylinmorgan / viv

viv isn't venv

viv isn't venv

Documentation

Try before you buy!

python3 <(curl -fsSL viv.dayl.in/viv.py) run frogmouth -- gh daylinmorgan/viv

---

Viv is a standalone dependency-free venv creator (just needs python + pip). Viv helps you ignore silly things like managing temporary or rarely used virtual environments, while still unleashing the full power of python scripting with it's entire ecosystem at your disposal.

Viv's uncompromising insistence on portability means that it will always, only use the standard library and never exceed a single script.

See the documentation or the examples to get started.

Setup

Run the below command to install viv.

python3 <(curl -fsSL viv.dayl.in/viv.py) manage install

To access viv from within scripts you should add its location to your PYTHONPATH. By default viv will be installed to $XDG_DATA_HOME/viv or ~/.local/share/viv; you can customize this with --src.

export PYTHONPATH="$PYTHONPATH:$HOME/.local/share/viv"

Pypi (Not Recommended)

pip install viv

Why is this not recommended? Mainly because viv is all about hacking your sys.path. Placing it in its own virtual environment or installing in a user site directory may complicate this endeavor.

Usage

In any Python script with external dependencies you can add this line to automate vivenv creation and installation of dependencies.

As a cli:

viv run frogmouth -- gh daylinmorgan/viv

As a python module:

__import__("viv").use("click")

As an app installer:

viv shim ruff

To remove all vivenvs you can use the below command:

viv env remove $(viv list -q)

To remove viv altogether you can use the included purge command:

python3 <(curl -fsSL viv.dayl.in/viv.py) manage purge

Equivalent commands from alternatives

pip-run

pip-run cowsay -- -m cowsay "moove over, pip-run"
python3 <(curl -fsSL viv.dayl.in/viv.py) run cowsay -- "moove over, pip-run"

python -m pip-run requests -- -c "import requests; print(requests.get('https://pypi.org/project/pip-run').status_code)"
python -m viv run requests -b python -- -c "import requests; print(requests.get('https://pypi.org/project/viv').status_code)"

pipx

pipx install pycowsay
viv shim pycowsay

pipx run https://gist.githubusercontent.com/cs01/fa721a17a326e551ede048c5088f9e0f/raw/6bdfbb6e9c1132b1c38fdd2f195d4a24c540c324/pipx-demo.py
python3 <(curl -fsSL viv.dayl.in/viv.py) run \
  -s https://gist.githubusercontent.com/cs01/fa721a17a326e551ede048c5088f9e0f/raw/6bdfbb6e9c1132b1c38fdd2f195d4a24c540c324/pipx-demo.py

Bonus: use viv with just standalone snippet (37LOC)

--standalone will auto-generate a mini-function version of viv to accomplish the same basic task as using a local copy of viv. After generating this standalone shim you can freely use this script across Unix machines which have Python > 3.8. See examples/black for output of the below command.

viv freeze also supports --standalone.

python3 <(curl -fsSL viv.dayl.in/viv.py) shim black -o ./black --standalone --freeze

viv's Issues

improve configuration

Currently all configuration is handled by env variables.

This is practical and quite usable when viv is run as an imported module.
But some folks might prefer a standard configuration file.

Could support a viv.toml or viv.ini in a standard location like ~/.config/viv/.

Some of config should likely exist as command-line flags as well.
With a precedence of CLI > ENVVAR > TOML/INI file.

Could also possibly accept a local config file in the Path.cwd() at either viv.toml/.viv.toml/viv.ini/.viv.ini

id/name collision with the same `spec`

Shouldn't fall back on hash with named env's since there can be collisions with the same spec. A quick fix would be to add the name to the hash.

 def _match_vivenv(self, name_id: str) -> ViVenv:  # type: ignore[return]
        # TODO: improve matching algorithm to favor names over id's
        matches: List[ViVenv] = []
        for k, v in self.vivenvs.items():
            if name_id == k or v.name == name_id:
                matches.append(v)
            elif k.startswith(name_id) or v.id.startswith(name_id):
                matches.append(v)
            elif v.name.startswith(name_id):
                matches.append(v)

        if len(matches) == 1:
            return matches[0]
        elif len(matches) > 1:
            echo(f"matches {','.join((match.name for match in matches))}", style="red")
            error("too many matches maybe try a longer name?", code=1)
        else:
            error(f"no matches found for {name_id}", code=1)

an empty spec isn't always an error i.e. with `viv run -s ./script.py`

If a user tries to use viv run on a script that does not invoke viv nor do they pass in any deps on the cli it may be worth raising a warning rather than an error. Since it's valid to use viv run with only the stdlib but it might be unexpected and worth informing the user that the spec was empty an no environment will be generated.

running the following script:

#!/usr/bin/env viv run -s
print("Hello World!")

raises:

viv |ERROR| unexepected input in package spec
viv |ERROR| check your packages definitions: []

`viv freeze -k` breaks hardcoded paths

As a tool for debugging viv exe allows users to execute python/pip. However, with viv freeze the spec is recalculated after the environment is generated. Meaning the hardcoded paths to the original spec are invalid... As a solution we can just regenerate the venv a new once we have the resolved spec.

support additional pip arguments

pip has alot of arguments and it would be nice to to specify them, proxies, requires-python and so forth.

possible ergonmics:

viv run simple \
  --pip-flags '--extra-index-url http://my.package.repo/simple'
# or 
VIV_PIP_FLAGS='--extra-index-url http://my.package.repo/simple' \
   viv run simple

__import__("viv").use(
    "simple",
    pip_flags = "--extra-index-url http://my.package.repo/simple"
)

# /// script
# dependencies = ["simple"]
#
# [tool.viv]
# pip_flags = "--extra-index-url http://my.package.repo/simple"
# ///

setuptools in `vivenv` should be opt-out

In order to keep virtual environments lean I don't install pip or setuptools.
But in testing viv it seems a frustrating number of tools still rely on pkg_resources and thus expect a local copy of setuptools. Would probably make more sense to make the non-setuptools an opt-in feature (i.e. VIV_NO_SETUPTOOLS environment variable) since this might be a non-obvious caveat to many users and not worth the IO/space savings.

call to venv

standalone output could be made more readable and require less but bytes with some more lines

could also just not use autospacing for # noqa as in v1.1
v1

def·_viv_use(*pkgs,·track_exe=False,·name=""):··························································#·noqa␊
····T,F,N=True,False,None;i,s,m,spec=__import__,str,map,[*pkgs]·········································#·noqa␊
····e,w=lambda·x:·T·if·x·else·F,lambda·p,t:·p.write_text(t)·············································#·noqa␊
····if·not·{*m(type,pkgs)}=={s}:·raise·ValueError(f"spec:·{pkgs}·is·invalid")···························#·noqa␊
····ge,sys,P,ew=i("os").getenv,i("sys"),i("pathlib").Path,i("sys").stderr.write·························#·noqa␊
····(cache:=(P(ge("XDG_CACHE_HOME",P.home()/".cache"))/"viv"/"venvs")).mkdir(parents=T,exist_ok=T)······#·noqa␊
····((sha256:=i("hashlib").sha256()).update((s(spec)+···················································#·noqa␊
····(((exe:=("N/A",s(P(i("sys").executable).resolve()))[e(track_exe)])))).encode()))····················#·noqa␊
····if·{env:=cache/(((_id:=sha256.hexdigest()),name)[e(name)])}-{*cache.glob("*/")}·or·ge("VIV_FORCE"):·#·noqa␊
········v=e(ge("VIV_VERBOSE"));ew(f"generating·new·vivenv·->·{env.name}\n")·····························#·noqa␊
········i("venv").EnvBuilder(with_pip=T,clear=T).create(env)············································#·noqa␊
········w(env/"pip.conf","[global]\ndisable-pip-version-check=true")····································#·noqa␊
········if·(rc:=(p:=i("subprocess").run([env/"bin"/"pip","install","--force-reinstall",*spec],text=T,···#·noqa␊
············stdout=(-1,N)[v],stderr=(-2,N)[v])).returncode)!=0:·········································#·noqa␊
············if·env.is_dir():i("shutil").rmtree(env)·····················································#·noqa␊
············ew(f"pip·had·non·zero·exit·({rc})\n{p.stdout}\n");sys.exit(rc)······························#·noqa␊
········w(env/"viv-info.json",i("json").dumps(··························································#·noqa␊
············{"created":s(i("datetime").datetime.today()),"id":_id,"spec":spec,"exe":exe}))··············#·noqa␊
····sys.path=[p·for·p·in·(*sys.path,s(*(env/"lib").glob("py*/si*")))if·p!=i("site").USER_SITE]··········#·noqa␊
····return·env··························································································#·noqa␊

v1.1

def·_viv_use(*pkgs,·track_exe=False,·name=""):·#·noqa␊
····T,F,N=True,False,None;i,s,m,spec=__import__,str,map,[*pkgs]·#·noqa␊
····e,w=lambda·x:·T·if·x·else·F,lambda·p,t:·p.write_text(t)·#·noqa␊
····if·not·{*m(type,pkgs)}=={s}:·raise·ValueError(f"spec:·{pkgs}·is·invalid")·#·noqa␊
····ge,sys,P,ew=i("os").getenv,i("sys"),i("pathlib").Path,i("sys").stderr.write·#·noqa␊
····(cache:=(P(ge("XDG_CACHE_HOME",P.home()/".cache"))/"viv"/"venvs")).mkdir(parents=T,exist_ok=T)·#·noqa␊
····((sha256:=i("hashlib").sha256()).update((s(spec)+·#·noqa␊
····(((exe:=("N/A",s(P(i("sys").executable).resolve()))[e(track_exe)])))).encode()))·#·noqa␊
····if·{env:=cache/(((_id:=sha256.hexdigest()),name)[e(name)])}-{*cache.glob("*/")}·or·ge("VIV_FORCE"):·#·noqa␊
········v=e(ge("VIV_VERBOSE"));ew(f"generating·new·vivenv·->·{env.name}\n")·#·noqa␊
········i("venv").EnvBuilder(with_pip=T,clear=T).create(env)·#·noqa␊
········w(env/"pip.conf","[global]\ndisable-pip-version-check=true")·#·noqa␊
········if·(rc:=(p:=i("subprocess").run([env/"bin"/"pip","install","--force-reinstall",*spec],text=T,·#·noqa␊
············stdout=(-1,N)[v],stderr=(-2,N)[v])).returncode)!=0:·#·noqa␊
············if·env.is_dir():i("shutil").rmtree(env)·#·noqa␊
············ew(f"pip·had·non·zero·exit·({rc})\n{p.stdout}\n");sys.exit(rc)·#·noqa␊
········w(env/"viv-info.json",i("json").dumps(·#·noqa␊
············{"created":s(i("datetime").datetime.today()),"id":_id,"spec":spec,"exe":exe}))·#·noqa␊
····sys.path=[p·for·p·in·(*sys.path,s(*(env/"lib").glob("py*/si*")))if·p!=i("site").USER_SITE]·#·noqa␊
····return·env·#·noqa␊

v2

def·_viv_use(*pkgs,·track_exe=False,·name=""):␊
····import·hashlib,·json,·os,·site,·shutil,·sys,·venv··#·noqa␊
····from·pathlib·import·Path··#·noqa␊
····from·datetime·import·datetime··#·noqa␊
····from·subprocess·import·run··#·noqa␊
␊
····if·not·{*map(type,·pkgs)}·==·{str}:␊
········raise·ValueError(f"spec:·{pkgs}·is·invalid")␊
····force,·verbose,·xdg·=·map(os.getenv,·("VIV_FORCE",·"VIV_VERBOSE",·"XDG_CACHE_HOME"))␊
····cache·=·(Path(xdg)·if·xdg·else·Path.home()·/·".cache")·/·"viv"·/·"venvs"␊
····cache.mkdir(parents=True,·exist_ok=True)␊
····exe·=·str(Path(sys.executable).resolve())·if·track_exe·else·"N/A"␊
····(sha256·:=·hashlib.sha256()).update((str(spec·:=·[*pkgs])·+·exe).encode())␊
····_id·=·sha256.hexdigest()␊
····if·(env·:=·cache·/·(name·if·name·else·_id))·not·in·cache.glob("*/")·or·force:␊
········sys.stderr.write(f"generating·new·vivenv·->·{env.name}\n")␊
········venv.EnvBuilder(with_pip=True,·clear=True).create(env)␊
········(env·/·"pip.conf").write_text("[global]\ndisable-pip-version-check=true")␊
········run_kw·=·{}·if·verbose·else·{"stdout":·None,·"stderr":·None}␊
········p·=·run([env·/·"bin"·/·"pip",·"install",·"--force-reinstall",·*spec],·**run_kw)␊
········if·(rc·:=·p.returncode)·!=·0:␊
············if·env.is_dir():␊
················shutil.rmtree(env)␊
············sys.stderr.write(f"pip·had·non·zero·exit·({rc})\n{p.stdout.decode()}\n")␊
············sys.exit(rc)␊
········meta·=·{"created":·str(datetime.today()),·"id":·_id,·"spec":·spec,·"exe":·exe}␊
········(env·/·"viv-info.json").write_text(json.dumps(meta))␊
····sys.path.remove(site.USER_SITE)␊
····sys.path.append(str(*(env·/·"lib").glob("py*/si*")))␊
····return·env␊

   20    98  1558 standalone-v1.1.py
   21    98  2221 standalone-v1.py
   30   164  1629 standalone-v2.py

`viv shim` should use the option passed to `bin` as the default file name.

PEP723 Support

PEP723 is a direct competitor to PEP722.
Based on online discussions it seems like there might be more preference for it over PEP722.

Implementing this for python <3.11 is tricky since I don't want to expand viv to more than one file or add a third-party dependency.
I see two options:

• use a zipapp with tomli
• embed a fully vendored albeit slightly modified tomli within viv.py

I've implemented the second option in the pep723 branch.
Examples:

python3 <(curl -fsSL https://raw.githubusercontent.com/daylinmorgan/viv/pep723/src/viv/viv.py) run -s https://raw.githubusercontent.com/daylinmorgan/viv/pep723/examples/pep723.py

Installation options for now:

python3 <(curl -fsSL viv.dayl.in/viv.py) manage install --ref pep723
viv manage update --ref pep723

Consider Supporting PEP722

PEP722 proposes a standardized dependency specification. Wouldn't be much effort to include parsing this in viv run.

PEP722 Discourse

`viv manage purge` can''t delete itself

Viv & python/pip version info

version: 2024-1004

Steps to Reproduce and Observed Behavior

running viv manage purge with a local copy will not be able to remove the directory containing the source in ~/.local/share/viv.

Expected Behavior

detect if running locally and tell users to use the remote version to safely purge all files and directories.

Viv shim should provide a pre-generate flag

add usage documentation

A number of important configuration switches and use cases are not appropriately documented.

Some of the configuration switches as environment variables

VIV_NO_SETUPTOOLS
VIV_VERBOSE
VIV_CACHE
VIV_DEBUG

need manual version checks for python and pip

shim files should perform a standard import under a try except and inform users of curl/iwr based install.

replace:

vivenv = __import__("viv").use("typer")

with:

try:
    import viv
except ImportError:
    print("viv not installed. install with .....")

if __name__ == "__main__":
    vivenv = viv.use("typer")

Possible considerations:

System or platform specific install instructions (curl vs iwr vs pip/pipx)

ephemeral cache should contain username for use on multi-user systems

Add Windows support

From a quick check, the main problems with Windows support seem to be assumptions about how a virtual environment is laid out (hard coded bin directory, the Python executable is python, not python.exe). To make the code cross-platform, it should be possible to use the context object that's used by the venv creation functions. It is documented here for Python 3.12, but with the exception of the lib_path attribute, the rest of the object is available in older versions of Python.

I'd be happy to help with testing on Windows, if you don't have access to the OS.

pre-commit fails on main

I noticed that pre-commit fails on a missing script when run across the whole project.

Steps to reproduce:

% git clone [email protected]:daylinmorgan/viv.git && cd viv
✂
% git rev-parse HEAD
c8be7c10914a3bf94a519c9253afa036d2c22910
% pre-commit run --all-files
black....................................................................Passed
ruff.....................................................................Passed
sets __version__ in viv.py...............................................Failed
- hook id: set-version
- exit code: 1

Executable `/home/maxwell-k/github.com/daylinmorgan/viv/scripts/bump-dev.sh` not found

The script was removed in 08cf8ee. The fix is likely to update .pre-commit-config.yaml to remove the reference to the script. I'll submit a PR shortly.