Improve exprience for environments that mix multiple Conda channels and PyPI about pixi

[tdejager]

I think step 7) is merged into main and will be available in the next release

[tdejager]

You are right that ideally we do not install the same pypi and conda package. However, the case being that because of different naming conda <-> pipy it can be hard to know which packages are equivalent. Pandoc being an example of having the same name but being a different package. There are also examples where the name of the package is x but the wheel in installing in namespace xy. A lot of intricacies.

That's why we create the parselmout mapping: https://github.com/prefix-dev/parselmouth. The problem being here that we do not have a mapping for the PyTorch channel (right @nichmor?), so we rely on a name to name mapping which fails.

The solution is that we should create a mapping for these channels as well. However, even when we have the mapping errors like these sometimes happen, that's why we have the clobber warning :)

---

W.r.t to your last point, specifically in the case of pypi, we now always prefer conda packages over pypi packages, like in your case its should've automatically picked the .conda version. We do this because wheels can vendor in whatever they like, while with conda the link to (system-level) packages like gcc or sdl2 is a lot more explicit.

[tdejager]

Just with regards to step 3). Is there something we could change in the explanation to make it clearer for first time users to understand whats happening :)?

[Yura52]

Just with regards to step 3). Is there something we could change in the explanation to make it clearer for first time users to understand whats happening :)?

I think the explanation is not a problem as such. To me, the problem was that I lost understanding of the state of my environment. I did not understand (and, I should admit, I still don't understand):

• how it was possible to have pytorch::pytorch and pypi::torch to be installed at the same time
• how I could actually go and check that indeed pypi::torch was used when I did import torch

Overall, I now realise that sometimes I implicitly expect that PyPI behaves just like another Conda channel. For example:

• I would be suprised if one package was installed simultaneously from two conda channels (but this is what effectively happened at Steps 2-3)
• For conda channels, if I install a package from a non-default channel (here, PyPI), I would expect that:
  • if a dependency is presented only in the default channel (conda-forge), take it from there.
  • if a dependecy is already explicitly provided in the manifest file with a channel (in my example, it is pytorch), follow the manifest file.
  • if a dependecy is presented in multiple channels, make user explicitly choose the channel (at least this is the current behavior as described here: #1532 (comment))

[Yura52]

Thank you for the detailed reply. Could you clarify, when I have both pytorch::pytorch and pypi::torch installed, what can I do to understand who is the provider of the torch package in .pixi/envs/envname/lib/python3.11/site-packages (i.e. what version: pypi:: or pytorch:: of torch I use when I do import torch)?

[tdejager]

So it isn't foolproof but you could check the INSTALLER file for the .dist-info of the package in the site-packages folder. It's conda when it is installed as a conda package and pixi-uv I think when it's pypi installed.

[Yura52]

Just in case, sharing my experience: for pypi::torch there is indeed the .dist-info thing and INSTALLER says uv-pixi. But for pytorch::pytorch there is .egg-info instead of .dist-info, seemingly without any information about the package provider:

$ ls -1 .pixi/envs/default/lib/python3.11/site-packages/torch-2.3.1-py3.11.egg-info
PKG-INFO
SOURCES.txt
dependency_links.txt
entry_points.txt
requires.txt
top_level.txt

[tdejager]

You are right, checking out: https://anaconda.org/pytorch/pytorch/2.3.1/download/osx-arm64/pytorch-2.3.1-py3.11_0.tar.bz2
seems to be packaged as a source distribution. Only wheels contain a .dist-info