[FEA] Enable cuda-version dependency resolution for packages that depend on cucim or other rapids packages about cucim

[vyasr]

Hey Matt, a cuda-version metapackage for pip is definitely something we'd like to implement, it just hasn't really risen that high on the priority list so far.

Your proposal with extras is interesting. There is some benefit to using extras to specify the version via a cucim metapackage as opposed to using separate packages the way we currently have (assuming the cuda-version metapackage were introduced), but not a big one The way I see it, there are three problems we want to solve.

Ensure consistent package installation

Consider the following two packages:

[project]
name = "cucim-cu11"
dependencies = [
  "cuda-version~=11",
]

[project]
name = "cucim-cu12"
dependencies = [
  "cuda-version~=12",
]

If we wanted to install cucim along with another RAPIDS package that was also CUDA-dependent and structured the same way, say cuml, the cuda-version metapackage would prevent us from doing something like pip install cucim-cu11 cuml-cu12 because those CUDA versions would be mismatched. That works just as well using two separate packages as via a package with extras.

Avoid dependent packages from needing to update the dependency name

If you have a package depending on cucim, then the cucim[cudefault] approach is a nice way to save you from needing to periodically bump your dependency list from cucim-cuX to cucim-cu{X+1}.

Allow version selection solely via cuda-version

With our conda packages, you can do something quite nice with cuda-version:

conda install cucim cuml cuda-version=12.0

Rather than specifying a version for every package, you can just specify the cuda-version package version and the conda solver will correctly figure out the versions of cuda-cudart and cuda-cccl to include. Ideally we'd like to replicate something like that for pip, but that will require a bit more thought. I also don't think that would benefit much from the extras approach.

Does this summary seem right to you? Basically the second item is the main one where extras give you a benefit AFAICT.

[jakirkham]

Related discussion in issue: rapidsai/build-planning#28

[thewtex]

Hi Vyas,

@vyasr Yes, you a right, the second point is the main benefit which could be solved with the current extras support in the ecosystem and the proposed extras usage:

Avoid dependent packages from needing to update the dependency name

but not a big one The way I see it,

It is a big one, and this importance comes from taking the perspective of someone who is developing a package that would like to depend on a RAPIDS package. Or, from someone who just wants to use that package and needing to know the details of the CUDA version or encountering conflicts is a major pain point.

[thewtex]

It seems it could be this extras approach or this proposed selector package feature?

CC @msarahan