zanieb / poetry-relax

Poetry plugin to relax version pins

poetry-relax

A Poetry plugin to relax dependency versions when publishing libraries. Relax your project's dependencies from foobar^2.0.0 to foobar>=2.0.0.

By default, Poetry uses caret constraints which would limit foobar versions to <3.0. poetry-relax removes these upper version bounds, allowing dependencies to be upgraded.

Removing upper version bounds is important when publishing libraries. When searching for versions of dependencies to install, the resolver (e.g. pip) must respect the bounds your library specifies. When a new version of the dependency is released, consumers of your library cannot install it unless a new version of your library is also released.

It is not feasible to release patches for every previous version of most libraries, which forces users to use the most recent version of the library or be stuck without the new version of the dependency. When many libraries contain upper version bounds, the dependencies can easily become unsolvable — where libraries have incompatible dependency version requirements. By removing upper version bounds from your library, control is returned to the user.

Poetry's default behavior is to include upper version bounds. Many people have spoken up against this style of dependency management in the Python ecosystem, including members of the Python core development team. See the bottom of the readme for links and additional context.

Since the Poetry project will not allow this behavior to be configured, maintainers have resorted to manual editing of dependency constraints after adding. poetry-relax aims to automate and simplify this process.

poetry-relax provides:

• Automated removal of upper bound constraints specified with ^
• Safety check if package requirements are still solvable after relaxing constraints
• Upgrade of dependencies after relaxing constraints
• Update of the lock file without upgrading dependencies
• Limit dependency relaxation to specific dependency groups
• Retention of intentional upper bounds indicating true incompatibilities
• CLI messages designed to match Poetry's output

Installation

The plugin must be installed in Poetry's environment. This requires use of the self subcommand.

$ poetry self add poetry-relax

Usage

Relax constraints for which Poetry set an upper version:

$ poetry relax

Relax constraints and check that they are resolvable without performing upgrades:

$ poetry relax --check

Relax constraints and upgrade packages:

$ poetry relax --update

Relax constraints and update the lock file without upgrading packages:

$ poetry relax --lock

Preview the changes poetry relax would make without modifying the project:

$ poetry relax --dry-run

Relax constraints for specific dependency groups:

$ poetry relax --only foo --only bar

Relax constraints excluding specific dependency groups:

$ poetry relax --without foo --without bar

Examples

The behavior of Poetry is quite reasonable for local development! poetry relax is most useful when used in CI/CD pipelines.

Relaxing requirements before publishing

Run poetry relax before building and publishing a package.

See it at work in the release workflow for this project.

Relaxing requirements for testing

Run poetry relax --update before tests to test against the newest possible versions of packages.

See it at work in the test workflow for this project.

Frequently asked questions

Can this plugin change the behavior of poetry add to relax constraints?

Not at this time. The Poetry project states that plugins must not alter the behavior of core Poetry commands. If this behavior would be useful for you, please chime in on the tracking issue.

Does this plugin remove upper constraints I've added?

This plugin will only relax constraints specified with a caret (^). Upper constraints added with < and <= will not be changed.

Is this plugin stable?

This plugin is tested against multiple versions of Poetry and has an integration focused test suite. It is safe to use this in production, though it is recommend to pin versions. Breaking changes will be avoided unless infeasible due to upstream changes in Poetry. This project follows the semantic versioning scheme and breaking changes will be denoted by a change to the major version number.

Will this plugin drop the upper bound on Python itself?

Believe it or not, this is an even more contentious subset of this issue as Poetry will not allow packages with no upper bound on Python to exist alongside those that include one. This basically means that we cannot relax this requirement without breaking the vast majority of use-cases. For this reason, we cannot relax python^3 at this time. See the post on the Poetry discussion board for more details.

Contributing

This project is managed with Poetry. Here are the basics for getting started.

Clone the repository:

$ git clone https://github.com/zanieb/poetry-relax.git
$ cd poetry-relax

Install packages:

$ poetry install

Run the test suite:

$ pytest tests

Run linters before opening pull requests:

$ ./scripts/lint check .
$ ./scripts/lint fix .

References

There's a lot to read on this topic! It's contentious and causing a lot of problems for maintainers and users.

The following blog posts by Henry Schreiner are quite comprehensive:

• Should You Use Upper Bound Version Constraints?
• Poetry Versions

Content from some members of the Python core developer team:

• Semantic Versioning Will Not Save You
• Why I don't like SemVer anymore
• Version numbers: how to use them?
• Versioning Software

Discussion and issues in the Poetry project:

• Please stop pinning to major versions by default, especially for Python itself
• Change default dependency constraint from ^ to >=
• Developers should be able to turn off dependency upper-bound calculations

poetry-relax's Issues

BUG: `run_installer_update` Uses Consumed List `dependencies`

In core_.run_installer_update, line 118 reuses the temporary variable dependencies:

for group_name, dependencies in dependencies_by_group.items():
        ...
        # Ensure if we are given a generator that we can consume it more than once
        dependencies = list(dependencies)
...
installer.whitelist([d.name for d in dependencies])

Should this be moved up into the for loop? This also fails if the group is empty (dependencies would be simply an empty list).

NOTE: Since dependencies is a list, this may actually be the original culprit of the error:

`list` object has no attribute `name`

Relax Group Dependencies

In local testing, it looks like poetry relax isn't relaxing group dependencies. For example:

> poetry add antigravity --group=xkcd
Using version ^0.1 for antigravity
...
> poetry relax
Checking dependencies for relaxable constraints...
No dependency constraints to relax.
> tail -n 2 pyproject.toml
[tool.poetry.group.xkcd.dependencies]
antigravity = "^0.1"

Thanks for building this package!

'PosixPath' object has no attribute 'path'

when testing with a section in pyproject.toml

[tool.poetry.group.foo.dependencies]
antigravity = "^0.1"

and running command:

> pt relax --group=foo  --check -vvv --dry-run
  Loading configuration file /home/wouter/.config/pypoetry/config.toml
  Loading configuration file /home/wouter/.config/pypoetry/auth.toml
  Using virtualenv: /home/wouter/.local/share/pypoetry/var/src/poetry-relax/.venv
  Using poetry file at /home/wouter/.local/share/pypoetry/var/src/poetry-relax/pyproject.toml
  Checking dependencies in group 'foo' for relaxable constraints...
  Found 1 dependencies in group 'foo'.
  Proposing updates to 1 dependencies.
  Proposing update for antigravity constraint from ^0.1 to >=0.1
  Checking new dependencies can be solved...
  
  'PosixPath' object has no attribute 'path'   <------
  
  Updated antigravity constraint from ^0.1 to >=0.1
  Aborted relax due to failure during dependency update.

Use `importlib` instead of `pkg_resources` in CI

ref https://setuptools.pypa.io/en/latest/pkg_resources.html

BUG: `list` Multiple Constraint Dependencies Not Supported

poetry supports multiple constraint dependencies provided as arrays of inline tables in TOML (i.e. lists of dictionaries in Python):

[tool.poetry.dependencies]
foo = [
    {version = "<=1.9", python = ">=3.6,<3.8"},
    {version = "^2.0", python = ">=3.8"}
]

The current source errors in parsing these, leading to the following error when attempting to use the plugin with such a dependency:

'list' object has no attribute 'name'

To fix this, please use the method in the current poetry source:

_constraints = constraints if isinstance(constraints, list) else [constraints]  # Convert to list

for _constraint in _constraints:  # Iterate through each item in constraint list
    # Now parse dictionaries in list

[FEAT]: Add `pre-commit` hook

Could we please add a pre-commit hook for this project?

`.pre-commit-hooks.yaml

- id: poetry-relax
  name: poetry-relax
  description: A poetry plugin to relax dependency versions when publishing libraries.
  entry: poetry relax
  language: python
  language_version: python3
  pass_filenames: false
  files: ^pyproject.toml$

Fails if any dependencies are VCS dependencies

Example pyproject:

$ cat pyproject.toml
[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = [""]
readme = "README.rst"
packages = [{include = "test", from = "src"}]

[tool.poetry.dependencies]
python = "^3.11"
httpx = {git = "https://github.com/encode/httpx.git"}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Tool output:

$ poetry relax
Checking dependencies in group 'main' for relaxable constraints...

Could not parse version constraint:

Drop support for Python 3.7

[BUG]: Fixture `seeded_project_venv` Fails on Windows

As the title states, this fixture fails on Windows since the path to the Python executable on Windows machines is Scripts\python.exe.

Can we please change the code to:

import sys
...
def seeded_project_venv(
    seeded_poetry_project_path: Path,
    poetry_application_factory: PoetryApplication,
) -> VirtualEnv:
    if sys.platform == 'win32':
        executable = seeded_poetry_project_path / ".venv" / "Scripts" / "python.exe"
    else:
        executable = seeded_poetry_project_path / ".venv" / "bin" / "python"

`poetry relax` relationship to `poetry up`

Discussion about this feature at python-poetry/poetry#461

There's a poetry-plugin-up project that allows dependency pins to be easily upgraded. I want add documentation referencing this plugin and differentiating it from the behavior here.

Support for auto-relax after `poetry add`

It's unclear if Poetry would disapprove of this as their documentation states that core command behavior should not be changed but a maintainer notes that a plugin may be a good way to change the default behavior in this comment.

It seems as though it would be straight-forward to add a hook after the add command runs that modifies the pyproject.toml using the mechanisms established for the poetry relax command. It's a little redundant to parse the file again, but I doubt the impact on speed will be noticeable.

If added, this would include a configurable toggle and would likely be off my default.

This issue will track requests for this feature. Feel free to add a 👍 or 👎 reaction here. Additional thoughts are welcome.

Is ``RelaxCommand`` a Mixin or Dependency Injection?

I understand why RelaxCommand inherits from InstallerCommand (to use poetry Installer when running check, update, or lock), but why does it also inherit from InitCommand? Is this class a dependency injection of sorts? I don't see any internal methods from InitCommand used in this class.

Support all groups at once

I recently started to refactor my project.toml-files into multiple groups like this

[tool.poetry.group.dev.dependencies]
bandit = { extras = ["toml"], version = ">=1.7.4" }
black = ">=22.1.0"
flake8 = ">=4.0.1"
flake8-pyproject = ">=1.2.2"
ipdb = ">=0.13.9"
isort = ">=5.10.1"
pre-commit = ">=2.14.1"
pre-commit-hooks = ">=4.1.0"
pylint = ">=2.12.2"

[tool.poetry.group.deploy.dependencies]
supervisor = ">=4.2.4"
gunicorn = ">=20.1.0"
uvicorn = ">=0.21.1"

[tool.poetry.group.docs.dependencies]
autoapi = ">=2.0.1"
recommonmark = ">=0.7.1"
Sphinx = ">=4.4.0"
sphinx-autoapi = ">=1.8.4"
sphinx-markdown-tables = ">=0.0.17"
sphinx-rtd-theme = { version = ">=1.2.0rc3", allow-prereleases = true }

[tool.poetry.group.test.dependencies]
pytest = ">=7.0.1"
pytest-cov = ">=3.0.0"
pytest-freezegun = ">=0.4.2"
pytest-mock = ">=3.10.0"

[tool.poetry.group.typing.dependencies]
mypy = ">=0.931"
types-cachetools = ">=5.3.0.4"
types-python-dateutil = ">=2.8.19"
types-requests = ">=2.28.9"

I basically don't care about what group needs relaxing, because they ALL need relaxing.

For the moment I need to execute:

poetry relax --group=main
poetry relax --group=deploy
poetry relax --group=docs
poetry relax --group=test
poetry relax --group=typing

Remark: Hide Installer Output (Commit `bce8b`)

What are the duplicates you're talking about? Was it anything like what was happening here when --ansi color output was being used?

Use new GitHub Actions syntax for setting step output