Installing dependencies from `pyproject.toml` in the Docker build phase takes very long about pip

[pfmoore]

Can this be reproduced without docker? If not, it seems like it's a docker issue, not a pip issue.

Having said that, you're not comparing the same things. A requirements file just installs the packages listed. The pyproject.toml file is building your project (which TBH looks broken, as pyproject.toml doesn't include a name or version, and you don't seem to be providing any code for your project) - which will require creating a build environment, installing setuptools in it, and then running the build step and installing the project, as well as installing the named dependencies.

[JamesParrott]

This issue should be closed, it is nothing to do with pip.

Lennart, take the time to read up on the packaging basics:
https://packaging.python.org/en/latest/tutorials/packaging-projects/

A pyproject.toml such as in your example (repeated below), is woefully incomplete, and will not build.

[project]
dependencies = ["flask", "flask_restful", "neo4j"]

To reproduce, install hatch, build and twine and create a new framework:

/Lennart>hatch new foo
foo
├── src
│   └── foo
│       ├── __about__.py
│       └── __init__.py
├── tests
│   └── __init__.py
├── LICENSE.txt
├── README.md
└── pyproject.toml

Try to build and verify:

\Lennart>cd foo

\Lennart\foo>python -m build .
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - hatchling
* Getting build dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - hatchling
* Getting build dependencies for wheel...
* Building wheel...
Successfully built foo-0.0.1.tar.gz and foo-0.0.1-py3-none-any.whl

\Lennart\foo>twine check --strict dist\*
Checking dist\foo-0.0.1-py3-none-any.whl: PASSED
Checking dist\foo-0.0.1.tar.gz: PASSED

After copying your pyproject.toml and pasting it over the top of the one hatch creates, the build fails:

python -m build .
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools >= 40.8.0
* Getting build dependencies for sdist...
Traceback (most recent call last):

...

    raise ValueError(f"{error}\n{summary}") from None
ValueError: invalid pyproject.toml config: `project`.
configuration error: `project` must contain ['name'] properties

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_sdist

[LennartCode]

Thank you for the feedback!

I am fairly new to both Docker and Python development and thus just tried to figure out where this thing breaks (docker or pip). It could well be that this is a Docker issue.

Regarding the pyproject.toml, I stripped the unnecessary stuff (bad practice, I guess). The whole file looks like this:

[project]
name = "project name"
description = "project description"
version = "0.0.1"
readme = "README.md"
dependencies = ["flask", "flask_restful", "neo4j"]

I am sorry to have caused confusion.

Also, building the project by hand (not docker) works with both requirements.txt and pyproject.toml using the exact same commands as the Dockerfile.

I am just posting this information, as it could be useful (maybe?), but feel free to close the issue if you still deem it to not be a pip issue.

Thank you for your work! <3

[pfmoore]

The extra time you're seeing is because of the build of the project. That shouldn't be particularly slow, but it does require internet access, and if your container isn't set up properly to allow internet access, maybe that's why the build is taking so long.

Your project appears to not actually be a "real" project, just a way of declaring dependencies. If that's the case, I suggest you should learn a little more about Python packaging (packaging.python.org is a good place to start) as you need a basic understanding if you're not going to make further mistakes like this.

[LennartCode]

Well, the project is a starter project and it does in fact include code (which is not relevant here). Also, the container has internet access, as using the requirements.txt instead of pyproject.toml builds the project fine in seconds.

"Mistakes like this" have actually nothing to do with the problem I reported but I thank you for your assessment of the situation. I will talk to some more docker people to triage this issue further.

[pfmoore]

Apologies - I was going off limited information and misread the situation.

It remains the case that python -m pip install -r requirements.txt doesn't build the project, whereas python -m pip install . does. That's where the performance difference lies. And if you don't understand why the two commands do different things, that's definitely something you should find out.