isabella232 / corpora-data-portal

The data portal supporting the submission, exploration, and management of projects and datasets to Corpora.

Corpora Data Portal

The Corpora Data Portal (CDP) enables the publication, discovery and exploration of interoperable single-cell datasets. Data contributors can upload, review and publish datasets for private or public use. Via the portal, data consumers are able to discover, download and connect data to visualization tools such as cellxgene to perform further analysis. The goal of the CDP is to catalyze distributed collaboration of single-cell research by providing a large, well-labeled repository of interoperable datasets.

Developers

Development quickstart

See DEV_ENV.md for the local development guide.

Pre-requisites

1. Install and configure awscli
2. Configure ssh access

Environment variables

Name	Description	Values
DEPLOYMENT_STAGE	Specifies an app deployment stage for tasks such as deployments and functional tests.	dev, staging, prod
AWS_PROFILE	Specifies the profile used to interact with AWS resources via awscli.	single-cell-dev, single-cell-prod
CORPORA_LOCAL_DEV	If this variable is set to any value, the Corpora app will look for the database on localhost:5432 and will use the aws secret corpora/backend/${DEPLOYMENT_STAGE}/database_local.	Any

Commands

Command	Description	Notes
make fmt	Auto-format codebase using black.	This should be run before merging in any changes.
make lint	Perform lint checks on codebase using flake8.	This should be run before merging in any changes.
make unit-test	Run all unit tests.
make functional-tests	Run all functional tests.	These tests run against a deployed environment which is selected by the value of DEPLOYMENT_STAGE.

Deployment

1. Set DEPLOYMENT_STAGE and AWS_PROFILE according to the environment to be deployed.
2. Deploy Backend
3. Deploy Cloudfront-invalidator
4. Deploy Frontend

Running unittests

1. Set AWS_PROFILE
2. Run the tests $ make unit-test

Running functional tests

1. Set DEPLOYMENT_STAGE and AWS_PROFILE according to the environment to be deployed.
2. In another terminal run make functional-test

Running local functional tests

1. Set DEPLOYMENT_STAGE and AWS_PROFILE according to the environment to be deployed.
2. Run make local-database to setup and populate a locally hosted database.
3. Run make local-backend to setup the api on a locally.
4. In another terminal run make functional-test