Make pixi create envs somewhere else than `$PWD/.pixi` about pixi

[ruben-arts]

The plan we currently have:

• Support the global config entry `target_pixi_environment_directory = "/home/user/path/to/dir"
• Then installing an environment will directly install it in to /home/user/path/to/dir/package_name-{pwd hash}/env/default
• Symlink the dir to the .pixi in the project. But only when possible, e.g. this doesn't work on windows.
• Add it to the pixi info output.
• Add clear documentation when to use this feature and when this is bad practice.

[moritzwilksch]

to make a Pixi env show up among Python environments that I can use as notebook kernels, I need to manually symlink it to the directory with my Conda environments.

this should not be necessary. You can chose .pixi/envs/default/bin/python as your interpreter with the "select interpreter" menu. Then - if you have ipykernel installed - the VSC jupyter extension can pick that up and work without symlinking.

I agree that this would be a great feature for the other reasons mentioned above, though.

[0xbe7a]

Then installing an environment will directly install it in to /home/user/path/to/dir/package_name-{pwd hash}/env/default

/home/user/path/to/dir/package_name is this supposed to be project.name from pixi.toml @ruben-arts ?

[ruben-arts]

That is an very interesting idea! We should discuss this. I love the idea for the faster disks.

[chawyehsu]

This may also help to resolve #777.

in some cases you can then even use hardlinks for your packages instead of copies

Would be so much helpful if this could be implemented.

[pavelzw]

You get the ncurses issue (#234) when trying to run pixi in a VSCode devcontainer on macOS. When this is solved, we could also properly using devcontainers. My current workaround:

.devcontainer/devcontainer.json

    // https://github.com/prefix-dev/pixi/issues/997
    "postCreateCommand": "mkdir -p /tmp/pixienv && ln -s /tmp/pixienv .pixi && pixi run postinstall && git config --global --add safe.directory ${containerWorkspaceFolder} && pixi run pre-commit-install",

This is not optimal because then the macOS installation wouldn't work (unless /tmp/pixienv exists locally too, otherwise i get os error 17)

EDIT:
This seems to be a better solution

    "mounts": ["source=${localWorkspaceFolderBasename}-pixi,target=${containerWorkspaceFolder}/.pixi,type=volume"],
    "postCreateCommand": "sudo chown vscode .pixi && pixi run postinstall && git config --global --add safe.directory ${containerWorkspaceFolder} && pixi run pre-commit-install",

[jaimergp]

I remember reading some concerns about activating symlinked environments. Something about some old compiled libraries resolving their own location in the wrong way or something like that. Pixi would have to resolve the symlink before activating/running/opening a shell, and that should be enough, I think?

Edit, here they are:

• conda/conda#7867
• conda/conda#3308

[kxmh42]

This would also help with two other issues I have:

1. I want to use Pixi envs in VS Code, the same way I use Conda envs. To make a Pixi env show up among Python environments that I can use as notebook kernels, I need to manually symlink it to the directory with my Conda environments. It would be nice if the Pixi env was automatically put there instead and symlinked from my project directory.
   Just one note: It would be nice to have readable env names, so please don't use sha-of-$PWD suggested above. My suggestion would be to symlink .pixi/envs/default -> /path/to/envs/<pixi-project-name> and symlink non-default envs, e.g. .pixi/envs/dev -> /path/to/envs/dev@<pixi-project-name>. Of course the problem is that there might already be an existing environment with the name equal to current Pixi project name, but that could simply result in an error indicating that either the exisitng env or the Pixi project name should be renamed, or perhaps it could automatically add a short suffix to the env name.

2. On the HPC platform I use, there is a scratch partition with no backup and much larger quota, and we are asked to put our environments there. I've tried moving the rattler cache to the scratch partition, but the files from there are then copied to my pixi envs, taking up the quota on the default partition and getting unnecessarily backed up. If Pixi envs were also on the scratch partition, the files would be hardlinked from the rattler cache, saving disk space.

[pavelzw]

Sounds great!

[ruben-arts]

@0xbe7a Yep, that is the idea. I would love to support some kind of full path things but that would make the path to long.

[kxmh42]

to make a Pixi env show up among Python environments that I can use as notebook kernels, I need to manually symlink it to the directory with my Conda environments.

this should not be necessary. You can chose .pixi/envs/default/bin/python as your interpreter with the "select interpreter" menu. Then - if you have ipykernel installed - the VSC jupyter extension can pick that up and work without symlinking.

I agree that this would be a great feature for the other reasons mentioned above, though.

Thanks for the tip! It works as a workaround, but it's a bit cumbersome when I have more than one pixi env in a project and want to switch between them. E.g. with .pixi/envs containing prod and dev, I don't see prod and dev in the list of Python environments, and I have to type/paste the interpreter path (.pixi/envs/prod/bin/python or .pixi/envs/dev/bin/python), and only then select the environment.

Am I correct in thinking that microsoft/vscode-python#22978 will make it easier?

[pavelzw]

Am I correct in thinking that microsoft/vscode-python#22978 will make it easier?

yes, that should make it easier. i also thought that it might be interesting to implement some pixi detection in the jupyter plugin, i.e., that vscode can pixi add ipykernel if it's not available yet