conda-forge / dask-gateway-feedstock

A conda-smithy repository for dask-gateway.

About dask-gateway-feedstock

Feedstock license: BSD-3-Clause

Home: http://github.com/dask/dask-gateway

Package license: BSD-3-Clause

Summary: A multi-tenant server for securely deploying and managing Dask clusters.

Development: http://github.com/dask/dask-gateway

Documentation: https://gateway.dask.org

A multi-tenant server for securely deploying and managing Dask clusters.

Current build status

Azure

Variant Status linux_64 linux_aarch64_python3.10 linux_aarch64_python3.11 linux_aarch64_python3.12 linux_aarch64_python3.9 osx_64_python3.10 osx_64_python3.11 osx_64_python3.12 osx_64_python3.9 win_64

Current release info

Name	Downloads	Version	Platforms

Installing dask-gateway

Installing dask-gateway from the conda-forge channel can be achieved by adding conda-forge to your channels with:

conda config --add channels conda-forge
conda config --set channel_priority strict

Once the conda-forge channel has been enabled, dask-gateway, dask-gateway-kerberos, dask-gateway-server, dask-gateway-server-jobqueue, dask-gateway-server-kerberos, dask-gateway-server-kubernetes, dask-gateway-server-local, dask-gateway-server-yarn can be installed with conda:

conda install dask-gateway dask-gateway-kerberos dask-gateway-server dask-gateway-server-jobqueue dask-gateway-server-kerberos dask-gateway-server-kubernetes dask-gateway-server-local dask-gateway-server-yarn

or with mamba:

mamba install dask-gateway dask-gateway-kerberos dask-gateway-server dask-gateway-server-jobqueue dask-gateway-server-kerberos dask-gateway-server-kubernetes dask-gateway-server-local dask-gateway-server-yarn

It is possible to list all of the versions of dask-gateway available on your platform with conda:

conda search dask-gateway --channel conda-forge

or with mamba:

mamba search dask-gateway --channel conda-forge

Alternatively, mamba repoquery may provide more information:

# Search all versions available on your platform:
mamba repoquery search dask-gateway --channel conda-forge

# List packages depending on `dask-gateway`:
mamba repoquery whoneeds dask-gateway --channel conda-forge

# List dependencies of `dask-gateway`:
mamba repoquery depends dask-gateway --channel conda-forge

About conda-forge

conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.

A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by Azure, GitHub, CircleCI, AppVeyor, Drone, and TravisCI it is possible to build and upload installable packages to the conda-forge anaconda.org channel for Linux, Windows and OSX respectively.

To manage the continuous integration and simplify feedstock maintenance conda-smithy has been developed. Using the conda-forge.yml within this repository, it is possible to re-render all of this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender.

For more information please check the conda-forge documentation.

Terminology

feedstock - the conda recipe (raw material), supporting scripts and CI configuration.

conda-smithy - the tool which helps orchestrate the feedstock. Its primary use is in the construction of the CI .yml files and simplify the management of many feedstocks.

conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)

Updating dask-gateway-feedstock

If you would like to improve the dask-gateway recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build. Once merged, the recipe will be re-built and uploaded automatically to the conda-forge channel, whereupon the built conda packages will be available for everybody to install and use from the conda-forge channel. Note that all branches in the conda-forge/dask-gateway-feedstock are immediately built and any created packages are uploaded, so PRs should be based on branches in forks and branches in the main repository should only be used to build distinct package versions.

In order to produce a uniquely identifiable distribution:

• If the version of a package is not being increased, please add or increase the build/number.
• If the version of a package is being increased, please remember to return the build/number back to 0.

Feedstock Maintainers

• @TomAugspurger
• @consideRatio
• @jacobtomlinson
• @jcrist
• @martindurant

dask-gateway-feedstock's Issues

CircleCI showing up on status

On commit ( 6b61b69 ), CircleCI is showing up noting the config is missing (related CircleCI job).

python 3.11 support

Comment:

Looks like the newest build on conda-forge only supports python 3.10:

- package dask-gateway-server-2023.1.1-py310ha8f183a_0 requires python_abi 3.10.* *_cp310, but none of the providers can be installed

Do these builds need to be tied to a particular python version? It seems like the pypi version isn't: https://pypi.org/project/dask-gateway-server/#files

@conda-forge-admin, please add user @consideratio

Based on this documentation in https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-maintainer-list on how to add maintainers, I opened this issue.

I figure its safe to merge the PR that is being created as its designed just to add a maintainer. From the looks of it, it seems like [ci skip] being added to the title does the trick.

[warning] failed package validation and/or copy for commit 6b61b6980b5d798859afddc7c911d484a41c217f

Hi @conda-forge/dask-gateway! This is the friendly automated conda-forge-webservice!

It appears that one or more of your feedstock's outputs did not copy from the
staging channel (cf-staging) to the production channel (conda-forge). :(

This failure can happen for a lot of reasons, including an outdated feedstock
token. Below we have put some information about the failure to help you debug it.

Rerendering the feedstock will usually fix these problems.

If you have any issues or questions, you can find us on gitter in the
community chat room or you can bump us right here.

error messages:

• invalid feedstock token

`conda install -c conda-forge "dask-gateway==2022.4.0"` - failure to find new version

What makes conda-forge/pangeo-dask-feedstock#83 conclude there is a new version of dask-gateway, while at the same time, the conda and mamba solvers conclude there isn't one? I've tried figuring out what goes on but failed for soon an hour now.

conda-forge/pangeo-dask-feedstock#83 fails like..

The reported errors are:
- Encountered problems while solving:
-   - nothing provides requested dask-gateway 2022.4.0.*

Locally with py39 I fail like...

$ conda install -c conda-forge "dask-gateway-server>=2022.4.0"

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - dask-gateway-server[version='>=2022.4.0']

Current channels:

  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch

Multiple issues to resolve for the 2022.4.0 release

As I've failed to resolve this myself, I've aimed to summarize known issues we have to fix leading to a release here in conda-forge. PRs to resolve things in this feedstock very much appreciated.

Goal definition

• Ensure we are multiple maintainers of this feedstock (#22).
• Feedstock's CI to go green for dask-gateway and dask-gateway-server version 2022.4.0 on PyPI.
• Build base and extras packages:
  • base packages
    • dask-gateway
      Published as py3-none-any on PyPI
    • dask-gateway-server
      Published as py3-none-[...] on PyPI where [...] is the four variations of combining the choices of linux/osx and amd64/arm64.
  • extras packages (related to extras_require in respective setup.py file)
    • dask-gateway-kerberos
    • dask-gateway-server-[jobqueue|local|slurm|yarn]
      Note that getting these packages built may not be essential and a possible compromise.

Notes

Python 3.10 / Py 38+ / No CPython / Included golang binary

Right now we publish one version per Python version (but not for Python 3.10). Preferably we would stop doing variations based on Python version as there is no need. But, we still need variations for dask-gateway-server which bundles a Golang executable binary that is to be made available for linux-amd64 / linux-arm64 / osx-amd64 / osx-arm64.

Arm64 support

In PyPI, we have with the 2022.4.0 release created wheels for dask-gateway-server built for Linux and Mac, both for amd64 and arm64. Previously, we didn't have wheels for arm64. I consider it an optional success if we manage to package for that as well, but I think osx-arm64 requires cross-compiling which may make the configuration need to adjust significantly.

Cross compiling for dask-gateway-server's golang binary can be done with GOOS and GOARCH, see https://github.com/dask/dask-gateway/blob/d0cb34d5a21d50998d157f7ede432eb9f0726d9b/dask-gateway-server/setup.py#L140-L156.

Decision about the extras packages

Would we be willing to stop providing variations for these? I think for dask-gateway we could simply declare build.noarch=python and we would be fine for all architectures, but by adjusting the dependencies based on windows or not, we may not be able to as seen in dask-gateway-kerberos. Similarly for dask-gateway-server, should we really provide all kinds of variations based on extras_require? I've failed to find documentation on this practice or if there are options on mapping extras_require to a conda-forge feedstock.