astral-sh / packse Goto Github PK

View Code? Open in Web Editor NEW

91.0 91.0 7.0 8.7 MB

Python packaging scenarios

License: Apache License 2.0

Python 99.44% HTML 0.56%

packse's People

Contributors

Stargazers

Watchers

Forkers

bswck konstin jankatins burntsushi 0v00 notatallshaw ibraheemdev

packse's Issues

Help Request: Convert `pip-resolver-brenchmark` format to `packse` format

I'm working on trying to get a process where you can put in real world requirements into pip-resolver-benchmark and get a packse scenario.

I am motivated in particular by the requirements "pandas<=1.4.0,>=1.3.5" "pystac<=1.8.3,>=1.8.2" "pystac-client<=0.3.3,>=0.3.2" "sat-stac<=0.1.1", which should install and does with uv but doesn't with pip.

This is the the pip-resolver-benchmark output: https://gist.github.com/notatallshaw/de066471b0a5458b58e0204842f2e110

And running tests it reproduces the bug in pip:

ERROR: Cannot install pystac-client==0.3.2 and pystac<=1.8.3 and >=1.8.2 because these package versions have conflicting dependencies.

I have wrote a quick hacky script to try and convert the output to packse format:

hacky-prb-packse-converter.py

import json

def convert_to_packse(pip_resolver_benchmark_json):
    packse_json = {
        "name": "pip-resolvelib-bug",
        "description": "A pip/resolvelib bug (https://github.com/sarugaku/resolvelib/issues/134 / https://github.com/pypa/pip/issues/12317)",
        "root": {"requires": []},
        "packages": {},
        "expected": {
            "satisfiable": True,
        },
    }

    # Load the input JSON
    with open(pip_resolver_benchmark_json, 'r') as f:
        pip_resolver_data = json.load(f)

    # Extract the root package information
    packse_json["root"]["requires"] = pip_resolver_data["input"]["requirements"]

    packse_packages = packse_json["packages"]
    for package_name, package_versions in pip_resolver_data["packages"].items():
        packse_packages[package_name] = {"versions": {}}
        packse_versions = packse_packages[package_name]["versions"]
        for package_version, package_details in package_versions.items():
            packse_versions[package_version] = {}
            packse_version = packse_versions[package_version]
            package_dependencies = package_details["depends_by_extra"]
            if package_dependencies:
                for extra_name, extra_dependencies in package_dependencies.items():
                    if extra_name == "":
                        packse_version["requires"] = extra_dependencies
                        continue
                    if "extras" not in packse_version:
                        packse_version["extras"] = {}
                    packse_version["extras"][extra_name] = extra_dependencies

            
            if package_details["requires_python"] != "None":
                packse_version["requires_python"] = package_details["requires_python"]

    return packse_json

# Example usage
pip_resolver_benchmark_json = 'pip-resolver-benchmark-scenario.json'
packse_json = convert_to_packse(pip_resolver_benchmark_json)

# Write the output to a file
with open('packse-format.json', 'w') as f:
    json.dump(packse_json, f, indent=4)

And this is the output: https://gist.github.com/notatallshaw/723f19a55102413cf1fef66631498e08

Which packse successfully builds and serves, but when I run the requirements against the index server the installation works:

$ pip install -v --dry-run --index-url "http://localhost:3141" "pip-resolvelib-bug-pandas-83f1b751<=1.4.0,>=1.3.5" "pip-resolvelib-bug-pystac-83f1b751<=1.8.3,>=1.8.2" "pip-resolvelib-bug-pystac-client-83f1b751<=0.3.3,>=0.3.2" "pip-resolvelib-bug-sat-stac-83f1b751<=0.1.1"

...

Would install pip-resolvelib-bug-certifi-83f1b751-2024.2.2 pip-resolvelib-bug-charset-normalizer-83f1b751-3.3.2 pip-resolvelib-bug-idna-83f1b751-3.7 pip-resolvelib-bug-importlib-resources-83f1b751-6.4.0 pip-resolvelib-bug-numpy-83f1b751-1.26.4 pip-resolvelib-bug-pandas-83f1b751-1.3.5 pip-resolvelib-bug-pystac-83f1b751-1.8.3 pip-resolvelib-bug-pystac-client-83f1b751-0.3.3 pip-resolvelib-bug-python-dateutil-83f1b751-2.7.5 pip-resolvelib-bug-pytz-83f1b751-2024.1 pip-resolvelib-bug-requests-83f1b751-2.31.0 pip-resolvelib-bug-sat-stac-83f1b751-0.1.1 pip-resolvelib-bug-six-83f1b751-1.16.0 pip-resolvelib-bug-urllib3-83f1b751-2.2.1 pip-resolvelib-bug-zipp-83f1b751-3.18.1

Please don't spend much time on this, but there's something obvious I'm missing please let me know.

P.S I have a fork of I have a fork of pip-resolver-benchmark to get this working, happy to publish the whole workflow if I can proove it actually works.

Scenario: Versions are not available that satisfy the Python requirement

We may need to add a way to represent the current Python version in the scenario.

Group by package in `view`

For example, in

      requires-transitive-incompatible-with-transitive-a8c0caa6
      └── a-1.0.0
          ├── requires b
          │   └── satisfied by b-1.0.0
          │       └── requires d==1.0.0
          │           ├── satisfied by d-1.0.0
          └── requires c
              └── satisfied by c-1.0.0
                  └── requires d==2.0.0
                      └── satisfied by d-2.0.0
      └── b-1.0.0
          └── requires d==1.0.0
              ├── satisfied by d-1.0.0
      └── c-1.0.0
          └── requires d==2.0.0
              └── satisfied by d-2.0.0
      ├── d-1.0.0
      └── d-2.0.0

We should create sections for a, b, c, and d then nest versions inside them e.g. d-1.0.0 and d-2.0.0 should be grouped.

How to validate if the correct packages have been installed?

I have a basic test framework working for pip, and I am now debugging the results.

One thing I am immediately confused on, is I expected that "expected" would give the success conditions for each scenario. But looking at example.json: https://github.com/astral-sh/packse/blob/main/scenarios/example.json this is not the case. Expected includes "b": "3.0.0", but the success criteria would be "b" equal to "2.0.0".

Is this just a bug in this example or am I misreading the scenarios?

Scenarios with ambigious solutions with respect to the spec?

This is the scenario: https://github.com/astral-sh/packse/blob/0.3.12/scenarios/prereleases.json#L320

It's listed as:

"expected": {
    "satisfiable": false,
    "explanation": "Since the user did not explicitly opt-in to a prerelease, it cannot be selected."
}

But there are many ways to interpret the spec, especially given the spec does not explicitly consider resolution. Currently pip resolves this scenario with: a==1.0.0 b==1.0.0 c==2.0.0b1.

What is packse's intent here? To set tests against the design principles of uv or against the spec?

Might it worth "expected" being a list of possible options when facing ambigious scenarios like this?

Break up and/or build JSON scenarios from code

I'm finding reading through the JSONs a little challenging, I have two suggestions (and would be happy to work on them at some point):

Break them down into more granular scenarios, e.g. I would split prerelease into "prerelease-direct", and "prerelease-transitive"
Build JSONs from a code model, allowing to easily encode a number of very similiar tests that only vary by small variables, e.g. ">" and ">=" should pass, or a sequence of tuples of succcess/failures [(">", true), (">=", true), ("==", true), ("!=", false), ("<=", false), ("<=", false)] rathar than writing out 6 different scenarios by hand

Short names are not valid as modules when starting with numbers

#74 breaks imports of modules since they sometimes start with numbers

Add support for other package index urls

The package index should be user configurable.

Speed things up

Tests are already taking >60s and there's no reason for that to be the case.

I imagine hatch is relatively slow, I'm not sure what else is slow yet.

Reference the source reasoning for each scenario

At the moment it's difficult to validate whether a specific scenario is following the spec or not because it's not referencing it. My idea would be that either in the existing field "explanation", or a new list field that would look something like "sources": [{"url": "...", "direct": true, "notes": "..."}, ...] the source(s) of the scenario can be referenced.

My idea is that:

Sources should primarily come from the spec, but in some cases the reasoning may be different, e.g. a comment clarifying the spec on a discussion thread, or copying pip's behavior, etc.
"direct": true, would mean the source(s) directly informs the scenario, false would mean this scenario is implied by the source(s) but isn't directly stated, this would apply to almost all cases which involve conflict resolution, as the spec does not cover this directly

This would probably be a lot of work to go back and update every scenario, but if it could be required for new scenarios that it could be slowly fixed over time.

Just a thought anyway, happy to slowly work on it if accepted.

Include scenarios in published package and have a way to point to them?

I'm thinking about how best to set up tests for the scenarios in this repo in other test suites (namely pip).

At the moment it seems like I would either need to vendor the JSONs or git clone the repo?

packse not servering yanked packages as yanked in simple api

For example scenario "requires-package-yanked-and-unyanked-any": https://github.com/astral-sh/packse/blob/main/scenarios/yanked.json#L48

Package a 1,0,0 should be yanked in the simple api, which would present itself as having the link attribute data-yanked, but when I look at the HTML that packse serves for this package I don't see that:

<!DOCTYPE html>
    <html>
        <head>
            <title>Links for requires-package-yanked-and-unyanked-any-a-c1d7f24e</title>
        </head>
        <body>
            <h1>Links for requires-package-yanked-and-unyanked-any-a-c1d7f24e</h1>
                 <a href="[/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-0.1.0-py3-none-any.whl#sha256=2b30c598aab25b18b92d60885285df7b66f80a3390658b1a6d96c8bc36f005e8](view-source:http://127.0.0.1:3141/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-0.1.0-py3-none-any.whl#sha256=2b30c598aab25b18b92d60885285df7b66f80a3390658b1a6d96c8bc36f005e8)">requires_package_yanked_and_unyanked_any_a_c1d7f24e-0.1.0-py3-none-any.whl</a><br>
                 <a href="[/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-0.1.0.tar.gz#sha256=274de419ae486e4b19cbbdd39e97ba272943d165a028effeefcae6c9bb999dea](view-source:http://127.0.0.1:3141/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-0.1.0.tar.gz#sha256=274de419ae486e4b19cbbdd39e97ba272943d165a028effeefcae6c9bb999dea)">requires_package_yanked_and_unyanked_any_a_c1d7f24e-0.1.0.tar.gz</a><br>
                 <a href="[/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-1.0.0-py3-none-any.whl#sha256=b0a7775307307ef338c7815424f883ac29e8363ad7f0baa9404f4e5b0bb963dc](view-source:http://127.0.0.1:3141/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-1.0.0-py3-none-any.whl#sha256=b0a7775307307ef338c7815424f883ac29e8363ad7f0baa9404f4e5b0bb963dc)">requires_package_yanked_and_unyanked_any_a_c1d7f24e-1.0.0-py3-none-any.whl</a><br>
                 <a href="[/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-1.0.0.tar.gz#sha256=ffa236db1626822a9ae069cb7388afb415cb3d9a900ee06f02065aa86ce96853](view-source:http://127.0.0.1:3141/packages/requires-package-yanked-and-unyanked-any-c1d7f24e/requires_package_yanked_and_unyanked_any_a_c1d7f24e-1.0.0.tar.gz#sha256=ffa236db1626822a9ae069cb7388afb415cb3d9a900ee06f02065aa86ce96853)">requires_package_yanked_and_unyanked_any_a_c1d7f24e-1.0.0.tar.gz</a><br>
        </body>
    </html>

If we compare that to say https://pypi.org/simple/apache-airflow/, I can see the data-yanked attribute for the relevant yanked files:

<a href="[https://files.pythonhosted.org/packages/bd/99/4761af2cea96cc818e05e9cd42d794b2ba79ceeccba96ea073398a9b49bd/apache-airflow-1.10.13.tar.gz#sha256=bb637f95a2aef7b2f7d622ecda150d6a5794011bd9e8d610ab0e9a1f518325e9](view-source:https://files.pythonhosted.org/packages/bd/99/4761af2cea96cc818e05e9cd42d794b2ba79ceeccba96ea073398a9b49bd/apache-airflow-1.10.13.tar.gz#sha256=bb637f95a2aef7b2f7d622ecda150d6a5794011bd9e8d610ab0e9a1f518325e9)" data-requires-python="&gt;=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*" data-yanked="Bug: https://github.com/apache/airflow/issues/12659 " >apache-airflow-1.10.13.tar.gz</a><br />

Show `devpi` logs when command fails

Lots of devpi commands are using check_output but not providing stdout/err if the command fails which is necessary for debugging.

Scenario: Pre-release versions

Only pre-release versions available for package
Pre-release causes backtrack
Only pre-release versions available in range
Only pre-release versions available in transitive dep's range

Running packse fetch and serve, get `UnicodeDecodeError`

Steps to Reproduce:

Linux Python 3.12 environment, Pip 24.0
pip install packse[serve]
packse fetch

Output:

Cloning repository https://github.com/astral-sh/packse
Checking out directory 'scenarios' at ref 0.3.7
Found does-not-exist.json
Found example.json
Found excluded.json
Found extras.json
Found incompatible-versions.json
Found local.json
Found prereleases.json
Found requires-python.json
Found wheels.json
Found yanked.json

packse serve scenarios/

Output:

$ packse serve scenarios/
Performing initial build...
Building 'local-simple-5b70bb3c' in directory 'build/local-simple-5b70bb3c'
Building 'package-only-prereleases-85d9c93d' in directory 'build/package-only-prereleases-85d9c93d'
Building 'example-961b4c22' in directory 'build/example-961b4c22'
Building 'local-not-used-with-sdist-207c9df5' in directory 'build/local-not-used-with-sdist-207c9df5'
Building 'local-not-latest-b8eed201' in directory 'build/local-not-latest-b8eed201'
Building 'local-used-without-sdist-ea57116a' in directory 'build/local-used-without-sdist-ea57116a'
Building 'package-only-prereleases-in-range-4c7ed550' in directory 'build/package-only-prereleases-in-range-4c7ed550'
Building 'requires-greater-version-does-not-exist-b33b2dca' in directory 'build/requires-greater-version-does-not-exist-b33b2dca'
Building 'local-transitive-confounding-082fdb86' in directory 'build/local-transitive-confounding-082fdb86'
Building 'requires-less-version-does-not-exist-a71baf60' in directory 'build/requires-less-version-does-not-exist-a71baf60'
Building 'local-transitive-a79dc870' in directory 'build/local-transitive-a79dc870'
Building 'requires-package-does-not-exist-388e8135' in directory 'build/requires-package-does-not-exist-388e8135'
Building 'transitive-requires-package-does-not-exist-ed146f67' in directory 'build/transitive-requires-package-does-not-exist-ed146f67'
Building 'requires-package-only-prereleases-in-range-global-opt-in-8229237e' in directory 'build/requires-package-only-prereleases-in-range-global-opt-in-8229237e'
Building 'requires-package-prerelease-and-final-any-7c7cec06' in directory 'build/requires-package-prerelease-and-final-any-7c7cec06'
Building 'package-prerelease-specified-only-prerelease-available-2b59d4b1' in directory 'build/package-prerelease-specified-only-prerelease-available-2b59d4b1'
Building 'requires-exact-version-does-not-exist-ef648a78' in directory 'build/requires-exact-version-does-not-exist-ef648a78'
Building 'package-prerelease-specified-only-final-available-1284cb1e' in directory 'build/package-prerelease-specified-only-final-available-1284cb1e'
Building 'package-multiple-prereleases-kinds-ca392ea8' in directory 'build/package-multiple-prereleases-kinds-ca392ea8'
Building 'package-prerelease-specified-mixed-available-4204a13b' in directory 'build/package-prerelease-specified-mixed-available-4204a13b'
Building 'transitive-package-only-prereleases-589ddff5' in directory 'build/transitive-package-only-prereleases-589ddff5'
Building 'transitive-prerelease-and-stable-dependency-73a8bb29' in directory 'build/transitive-prerelease-and-stable-dependency-73a8bb29'
Building 'transitive-package-only-prereleases-in-range-opt-in-dc3b4feb' in directory 'build/transitive-package-only-prereleases-in-range-opt-in-dc3b4feb'
Building 'package-multiple-prereleases-numbers-b08385b3' in directory 'build/package-multiple-prereleases-numbers-b08385b3'
Building 'transitive-package-only-prereleases-in-range-91d42144' in directory 'build/transitive-package-only-prereleases-in-range-91d42144'
Building 'transitive-prerelease-and-stable-dependency-many-versions-b550f888' in directory 'build/transitive-prerelease-and-stable-dependency-many-versions-b550f888'
Building 'transitive-prerelease-and-stable-dependency-opt-in-87b86d9c' in directory 'build/transitive-prerelease-and-stable-dependency-opt-in-87b86d9c'
Building 'transitive-prerelease-and-stable-dependency-many-versions-holes-34e5a2d3' in directory 'build/transitive-prerelease-and-stable-dependency-many-versions-holes-34e5a2d3'
Building 'package-only-prereleases-boundary-dd941311' in directory 'build/package-only-prereleases-boundary-dd941311'
Building 'package-prereleases-boundary-16ba0350' in directory 'build/package-prereleases-boundary-16ba0350'
Building 'package-prereleases-global-boundary-ca458d54' in directory 'build/package-prereleases-global-boundary-ca458d54'
Building 'package-prereleases-specifier-boundary-ed960178' in directory 'build/package-prereleases-specifier-boundary-ed960178'
Building 'specific-tag-and-default-0336e09c' in directory 'build/specific-tag-and-default-0336e09c'
Building 'only-wheels-f756804e' in directory 'build/only-wheels-f756804e'
Building 'no-wheels-0bb7827a' in directory 'build/no-wheels-0bb7827a'
Building 'no-wheels-with-matching-platform-c1494f5f' in directory 'build/no-wheels-with-matching-platform-c1494f5f'
Building 'no-sdist-no-wheels-with-matching-platform-46f0c229' in directory 'build/no-sdist-no-wheels-with-matching-platform-46f0c229'
Building 'no-sdist-no-wheels-with-matching-python-7b1e0ba3' in directory 'build/no-sdist-no-wheels-with-matching-python-7b1e0ba3'
Building 'no-sdist-no-wheels-with-matching-abi-2f8e7202' in directory 'build/no-sdist-no-wheels-with-matching-abi-2f8e7202'
Building 'no-wheels-no-build-1db1b462' in directory 'build/no-wheels-no-build-1db1b462'
Building 'only-wheels-no-binary-859a4cea' in directory 'build/only-wheels-no-binary-859a4cea'
Building 'no-build-bb7d81b8' in directory 'build/no-build-bb7d81b8'
Building 'no-binary-b1d20084' in directory 'build/no-binary-b1d20084'
Building 'package-only-yanked-2919761d' in directory 'build/package-only-yanked-2919761d'
Building 'requires-package-yanked-and-unyanked-any-c1d7f24e' in directory 'build/requires-package-yanked-and-unyanked-any-c1d7f24e'
Building 'package-only-yanked-in-range-f1ab2a3f' in directory 'build/package-only-yanked-in-range-f1ab2a3f'
Building 'package-yanked-specified-mixed-available-e9d957b6' in directory 'build/package-yanked-specified-mixed-available-e9d957b6'
Building 'transitive-package-only-yanked-in-range-e2eb8cbc' in directory 'build/transitive-package-only-yanked-in-range-e2eb8cbc'
Building 'transitive-package-only-yanked-fbebea19' in directory 'build/transitive-package-only-yanked-fbebea19'
Building 'transitive-package-only-yanked-in-range-opt-in-637e27eb' in directory 'build/transitive-package-only-yanked-in-range-opt-in-637e27eb'
Building 'transitive-yanked-and-unyanked-dependency-opt-in-b2a53fbd' in directory 'build/transitive-yanked-and-unyanked-dependency-opt-in-b2a53fbd'
Building 'transitive-yanked-and-unyanked-dependency-0abad3b6' in directory 'build/transitive-yanked-and-unyanked-dependency-0abad3b6'
Building 'direct-incompatible-versions-516a39b2' in directory 'build/direct-incompatible-versions-516a39b2'
Building 'transitive-incompatible-with-root-version-c4bc5b1f' in directory 'build/transitive-incompatible-with-root-version-c4bc5b1f'
Building 'transitive-incompatible-with-transitive-4132b57a' in directory 'build/transitive-incompatible-with-transitive-4132b57a'
Building 'python-version-does-not-exist-db416e00' in directory 'build/python-version-does-not-exist-db416e00'
Building 'python-less-than-current-c050f6d5' in directory 'build/python-less-than-current-c050f6d5'
Building 'python-greater-than-current-f9f64a8d' in directory 'build/python-greater-than-current-f9f64a8d'
Building 'python-greater-than-current-many-63ea5ba6' in directory 'build/python-greater-than-current-many-63ea5ba6'
Building 'python-greater-than-current-patch-23c0760c' in directory 'build/python-greater-than-current-patch-23c0760c'
Building 'python-greater-than-current-excluded-2255f47c' in directory 'build/python-greater-than-current-excluded-2255f47c'
Building 'python-greater-than-current-backtrack-f53b1cd6' in directory 'build/python-greater-than-current-backtrack-f53b1cd6'
Building 'incompatible-python-compatible-override-cc53ff2b' in directory 'build/incompatible-python-compatible-override-cc53ff2b'
Building 'compatible-python-incompatible-override-df025f97' in directory 'build/compatible-python-incompatible-override-df025f97'
Building 'incompatible-python-compatible-override-unavailable-no-wheels-161a27c2' in directory 'build/incompatible-python-compatible-override-unavailable-no-wheels-161a27c2'
Building 'incompatible-python-compatible-override-available-no-wheels-6e4d1670' in directory 'build/incompatible-python-compatible-override-available-no-wheels-6e4d1670'
Building 'incompatible-python-compatible-override-no-compatible-wheels-1b13caa9' in directory 'build/incompatible-python-compatible-override-no-compatible-wheels-1b13caa9'
Building 'incompatible-python-compatible-override-other-wheel-497d1feb' in directory 'build/incompatible-python-compatible-override-other-wheel-497d1feb'
Building 'python-patch-override-no-patch-8429e03a' in directory 'build/python-patch-override-no-patch-8429e03a'
Building 'python-patch-override-patch-compatible-b6cdac48' in directory 'build/python-patch-override-patch-compatible-b6cdac48'
Building 'extra-required-bbd42201' in directory 'build/extra-required-bbd42201'
Building 'missing-extra-d7b6e0b1' in directory 'build/missing-extra-d7b6e0b1'
Building 'multiple-extras-required-73ee83b9' in directory 'build/multiple-extras-required-73ee83b9'
Building 'all-extras-required-61571b3c' in directory 'build/all-extras-required-61571b3c'
Building 'extra-incompatible-with-extra-eeb916b5' in directory 'build/extra-incompatible-with-extra-eeb916b5'
Building 'extra-incompatible-with-extra-not-requested-97cf0638' in directory 'build/extra-incompatible-with-extra-not-requested-97cf0638'
Building 'extra-incompatible-with-root-6faf09f6' in directory 'build/extra-incompatible-with-root-6faf09f6'
Building 'extra-does-not-exist-backtrack-05f843a2' in directory 'build/extra-does-not-exist-backtrack-05f843a2'
Building 'excluded-only-version-d5e62e6c' in directory 'build/excluded-only-version-d5e62e6c'
Building 'excluded-only-compatible-version-26dd59e3' in directory 'build/excluded-only-compatible-version-26dd59e3'
Building 'dependency-excludes-range-of-compatible-versions-cb8d51de' in directory 'build/dependency-excludes-range-of-compatible-versions-cb8d51de'
Building 'dependency-excludes-non-contiguous-range-of-compatible-versions-5d038b4f' in directory 'build/dependency-excludes-non-contiguous-range-of-compatible-versions-5d038b4f'
Exception in thread serve-build-scenarios:
Traceback (most recent call last):
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/damian/.pyenv/versions/packse_testing_312/lib/python3.12/site-packages/packse/serve.py", line 79, in build_scenarios
    build(
  File "/home/damian/.pyenv/versions/packse_testing_312/lib/python3.12/site-packages/packse/build.py", line 88, in build
    results = [future.result() for future in futures]
               ^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/packse_testing_312/lib/python3.12/site-packages/packse/build.py", line 179, in build_scenario
    future.result()
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/packse_testing_312/lib/python3.12/site-packages/packse/build.py", line 232, in build_scenario_package
    package_destination = create_from_template(
                          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/packse_testing_312/lib/python3.12/site-packages/packse/template.py", line 63, in create_from_template
    chevron_blue.render(file_path.read_text(), scope, no_escape=True)
                        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/damian/.pyenv/versions/3.12.1/lib/python3.12/pathlib.py", line 1028, in read_text
    return f.read()
           ^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcb in position 0: invalid continuation byte
Build thread failed..

Publish does not show skipped distributions when file is identical to previous upload

When file hashes match, the distributions is shown as published instead of skipped.

"transitive-package-only-prereleases-in-range" doesn't follow the spec

The scanrio is the following currently:

{
    "name": "transitive-package-only-prereleases-in-range",
    "description": "The user requires package `a` which has a dependency on a package which only matches prerelease versions but they did not include a prerelease specifier.",
    "root": {
        "requires": [
            "a"
        ]
    },
    "packages": {
        "a": {
            "versions": {
                "0.1.0": {
                    "requires": [
                        "b>0.1"
                    ]
                }
            }
        },
        "b": {
            "versions": {
                "0.1.0": {},
                "1.0.0a1": {}
            }
        }
    },
    "expected": {
        "satisfiable": false,
        "explanation": "Since there are stable versions of `b` available, the prerelease version should not be selected without explicit opt-in. The available version is excluded by the range requested by the user."
    }
},

In short, the idea is because a prerelease requirement is not opted into then even though a prerelease version is available it is not selected.

However, this is explictly again the following line in the spec (https://packaging.python.org/en/latest/specifications/version-specifiers/#handling-of-pre-releases):

By default, dependency resolution tools SHOULD:

...

accept remotely available pre-releases for version specifiers where there is no final or post release that satisfies the version specifier

...

In this case there is no final or post release that satisfies the version specifier, and therefore the dependency resolution SHOULD accept the removetely available pre-release.

Currently neither uv nor pip follow the spec here, pip needs an overhaul on prereleases and uv has deicded not to follow the spec due to design choices, but poetry does follow the spec and does find this satisfiable.

If packse is attempting to follow the spec, and not be a catalogue of uv's design choices, this should be set to "satisfiable": true and uv should mark this as XFAIL. I'll be happy to make a PR.

Trying to understand "local-transitive-confounding" scenario

In the scenario "local-transitive-confounding": https://github.com/astral-sh/packse/blob/0.3.12/scenarios/local.json#L280

Has expected:

"expected": {
    "satisfiable": true,
    "packages": {
        "a": "2.0.0+foo"
    },

But there is no package a==2.0.0+foo:

"packages": {
    "a": {
        "versions": {
            "1.0.0": {
                "requires": [
                    "b==2.0.0"
                ]
            }
        }
    },

Vendored build backend

Right now, we cannot use Test PyPI as the index URL, just the extra index URL because we require build backend packages which are not available on Test PyPI. It'd be great to isolate if feasible

Scenario: Versions are not available with wheels for the current platform

We may need to add support for platforms and building wheels for multiple versions 🤯

Feature Request: Have `packse` support every version of Python that it's scenarios run against

Same motivation as #163 but with a different solution.

For a simpler test environment for pip I am trying to avoid setting up multiple Python environment versions, packse requires Python 3.12+ so I am only running tests with a Python 3.12 environment.

Most scenarios in packse are set to Python 3.8, for most scenarios this is not relevant to the results of the test. However for a few scenarios it is actually important to resolve the scenario against the Python version specified.

One solution is I could set the pip tests to only run on the Python version that the scenario is set against. It would make testing locally awkward but it would simplify the testing solution itself.

Trying to understand "package-only-prereleases-boundary" scenario

This is the scenario: https://github.com/astral-sh/packse/blob/0.3.12/scenarios/prereleases.json#L520

In this scenario package a only has prereleases, but the specifier used is an exclusive specificier a<0.2.0, the spec says:

The exclusive ordered comparison <V MUST NOT allow a pre-release of the specified version unless the specified version is itself a pre-release. Allowing pre-releases that are earlier than, but not equal to a specific pre-release may be accomplished by using <V.rc1 or similar.

Now, I appreciate there are other parts of the spec that may be contradictory. So I'm not sure if this is an intentional design choice or not? But my understanding is the intent of the spec is it should not be allowed: pypa/packaging#776 (comment)

Scenario: Incompatible URL-dependencies

When dependencies are specified by URL, we need to resolve them and determine that they are incompatible versions.

Multiple URLs cannot be provided for a dependency.

Add scenarios based on legacy versions and legacy specifiers

I'm willing to write these scenarios myself, but I'm making an issue first in case of any objection, in particular because this is about following the spec, and I beleive uv deviates from the spec for this behavior to be more compatible with the ecosystem:

With requirement A and A has a new version with a non-legacy version and an old version with a legacy version then A should successfully resolve to the new non-legacy version
With requirement A and A has a new version with a legacy version and an old version with a non-legacy version then A should successfully resolve to old non-legacy version
With requirement A and A only has legacy versions then A should not resolve
With a requirement on A that involves a legacy version in the specifier then that requirement should not resolve
Same as all of above but the where A depends on B and B suffers from the legacy versions/specifiers

Consider excluding scenarios from tests

We already test all of the scenarios in CI, is it really worth including them all in the test suite itself? It means every scenario change causes a snapshot change.

Feature Request: Default `sdist: false` where not specified in scenario

For a simpler test environment for pip I am trying to avoid setting up multiple Python environment versions, packse requires Python 3.12+ so I am only running tests with a Python 3.12 environment.

To get pip to run as though it were Python 3.8 when it's Python environment is Python 3.12 I must use the --python-version flag, however this flag is incompatible with sdists.

I beleive, but correct me if I'm wrong, that packse will default to sdist: true when it's not explictly specified by the scenario. This is problematic, as it means I can not distinguish when it's required to process sdists or not, from the output of packse inspect.

Fix dependency tree in `view`

For example, in

      requires-transitive-incompatible-with-transitive-a8c0caa6
      └── a-1.0.0
          ├── requires b
          │   └── satisfied by b-1.0.0
          │       └── requires d==1.0.0
          │           ├── satisfied by d-1.0.0
          └── requires c
              └── satisfied by c-1.0.0
                  └── requires d==2.0.0
                      └── satisfied by d-2.0.0
      └── b-1.0.0
          └── requires d==1.0.0
              ├── satisfied by d-1.0.0
      └── c-1.0.0
          └── requires d==2.0.0
              └── satisfied by d-2.0.0
      ├── d-1.0.0
      └── d-2.0.0

satisfied by d-1.0.0 should have a "last" prefix instead of a "tee" prefix and b-1.0.0, c-1.0.0 should be connected, perhaps by a "branch" prefix.

Convert scenarios to TOML

It'd be nice to move the existing scenarios to TOML instead of JSON for readability and comments?

Preferably one file at a time so we can reorganize as discussed in #173 and #174

Scenario: Yanked versions

Is it even possible to programmatically yank versions?

Split snapshots into separate files

I really don't like multiple snapshots per file

Trying to understand "local-not-used-with-sdist" scenario

I am looking as to why pip does not agree with the expected output of "local-not-used-with-sdist" scenario: https://github.com/astral-sh/packse/blob/0.3.12/scenarios/local.json#L25

Pip ends with "Successfully installed local-not-used-with-sdist-a-207c9df5-1.2.3+foo", which looks correct to me? But expected is listed as "a": "1.2.3"

I think "1.2.3" is exlcluded by pip because of the wheel tag "py3-any-macosx_10_0_ppc64", which I don't think is a very common compatible tag? Is there somewhere listed what compatible tags the installer is supposed to be using?

Apologies if I'm missing something very obvious.

astral-sh / packse Goto Github PK

packse's People

Contributors

Stargazers

Watchers

Forkers

packse's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs