qiime2 / package-integration

qiime2 (the QIIME 2 framework)

Source code repository for the QIIME 2 framework.

QIIME 2™ is a powerful, extensible, and decentralized microbiome bioinformatics platform that is free, open source, and community developed. With a focus on data and analysis transparency, QIIME 2 enables researchers to start an analysis with raw DNA sequence data and finish with publication-quality figures and statistical results.

Visit https://qiime2.org to learn more about the QIIME 2 project.

Installation

Detailed instructions are available in the documentation.

Users

Head to the user docs for help getting started, core concepts, tutorials, and other resources.

Just have a question? Please ask it in our forum.

Developers

Please visit the contributing page for more information on contributions, documentation links, and more.

Citing QIIME 2

If you use QIIME 2 for any published research, please include the following citation:

Bolyen E, Rideout JR, Dillon MR, Bokulich NA, Abnet CC, Al-Ghalith GA, Alexander H, Alm EJ, Arumugam M, Asnicar F, Bai Y, Bisanz JE, Bittinger K, Brejnrod A, Brislawn CJ, Brown CT, Callahan BJ, Caraballo-Rodríguez AM, Chase J, Cope EK, Da Silva R, Diener C, Dorrestein PC, Douglas GM, Durall DM, Duvallet C, Edwardson CF, Ernst M, Estaki M, Fouquier J, Gauglitz JM, Gibbons SM, Gibson DL, Gonzalez A, Gorlick K, Guo J, Hillmann B, Holmes S, Holste H, Huttenhower C, Huttley GA, Janssen S, Jarmusch AK, Jiang L, Kaehler BD, Kang KB, Keefe CR, Keim P, Kelley ST, Knights D, Koester I, Kosciolek T, Kreps J, Langille MGI, Lee J, Ley R, Liu YX, Loftfield E, Lozupone C, Maher M, Marotz C, Martin BD, McDonald D, McIver LJ, Melnik AV, Metcalf JL, Morgan SC, Morton JT, Naimey AT, Navas-Molina JA, Nothias LF, Orchanian SB, Pearson T, Peoples SL, Petras D, Preuss ML, Pruesse E, Rasmussen LB, Rivers A, Robeson MS, Rosenthal P, Segata N, Shaffer M, Shiffer A, Sinha R, Song SJ, Spear JR, Swafford AD, Thompson LR, Torres PJ, Trinh P, Tripathi A, Turnbaugh PJ, Ul-Hasan S, van der Hooft JJJ, Vargas F, Vázquez-Baeza Y, Vogtmann E, von Hippel M, Walters W, Wan Y, Wang M, Warren J, Weber KC, Williamson CHD, Willis AD, Xu ZZ, Zaneveld JR, Zhang Y, Zhu Q, Knight R, and Caporaso JG. 2019. Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2. Nature Biotechnology 37:852–857. https://doi.org/10.1038/s41587-019-0209-9

package-integration's Issues

cron failures

rev-dep test cron job: build against latest distro metapackage

Right now we're building a distro metapackage de novo for the rev-dep tests - this should actually just use the latest distro metapackage. One way to do that pretty easily is to adapt the current distro recipe by splitting into two recipes - one for building the metapackage, and another for building a meta-metapackage.

meta todo: move 'additional-tests' into recipes

Right now we have our tests specified as alp "additional-tests" - we should move those into the recipe test section to reap the benefits of the revdep testing cron job.

gh workflows: template them

Right now the ci and cron workflows are mostly identical - setting up a templating script might help make our lives easier.

imp: add auto retry for 'invalidchunklength' and 'cancelled' failures

It would be nice if we could add an automatic job re-run trigger for the following types of GHA failures:

requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

CondaHTTPError: HTTP 530 CONNECTION FAILED for url <https://conda.anaconda.org/conda-forge/osx-64/cryptography-39.0.0-py38ha6c3189_0.conda>
Elapsed: 00:09.195701
CF-RAY: 79d1a4b35f567ecb-LAX

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

meta todo: update recipes to use cbc pins for common deps

Like pandas, scipy, numpy, etc.

imp: add input param for dev/rel epoch to update $Q2_CHANNEL variable

We need to add an input parameter that differentiates the development vs. release epoch. This will be used in another new parameter that we will call the 'gate' - this will be for the package channel that each metapackage will be looking at (either staged or passed for dev/rel epochs, respectively).

This is where those parameters will be used:

package-integration/.github/actions/destructure-metapackage/action.yml

Line 29 in 7e9c493

echo "Q2_CHANNEL=https://packages.qiime2.org/qiime2/${{ inputs.epoch }}/staged/${{ inputs.distro }}" >> $GITHUB_ENV