The overriden ``create`` method in the plugin serializer shouldn't save a plugin until all the plugin parameters and its default values are validated.

ChRIS_store

Backend for the ChRIS store. This is a Django-PostgreSQL project that houses descriptions of ChRIS plugin-apps and workflows for registering to a ChRIS CUBE instance.

ChRIS store development, testing and deployment

Abstract

This page describes how to quickly get the set of services comprising the backend up and running for development and how to run the automated tests. A production deployment of the ChRIS store backend services is also explained.

Preconditions

Install latest Docker and Docker Compose

Currently tested platforms:

Ubuntu 18.04+ and MAC OS X 10.14+ and Fedora 31+ (Additional instructions for Fedora)
Docker 18.06.0+
Docker Compose 1.27.0+

On a Linux machine make sure to add your computer user to the `docker` group

Consult this page: https://docs.docker.com/engine/install/linux-postinstall/

TL;DR

If you read nothing else on this page, and just want to get an instance of the ChRIS store backend services up and running with no mess, no fuss:

git clone https://github.com/FNNDSC/ChRIS_store.git
cd ChRIS_store
./make.sh down ; ./make.sh up

The resulting instance uses the default Django development server and therefore is not suitable for production.

Production deployment on a single-machine Docker Swarm cluster

To get the production system up:

Start a local Docker Swarm cluster if not already started:

docker swarm init --advertise-addr 127.0.0.1

Fetch source code:

git clone https://github.com/FNNDSC/ChRIS_store
cd ChRIS_store

Create secrets directory:

mkdir swarm/prod/secrets

Now copy all the required secret configuration files into the secrets directory, please take a look at this wiki page to learn more about these files

Deploy ChRIS store backend containers:

./deploy.sh up

To tear down:

Remove ChRIS store backend containers:

cd ChRIS_store
./deploy.sh down

Remove the local Docker Swarm cluster if desired:

docker swarm leave --force

Development

Instantiate ChRIS Store dev environment

Start ChRIS Store services by running the make bash script from the repository source directory

git clone https://github.com/FNNDSC/ChRIS_store.git
./make.sh up

All the steps performed by the above script are properly documented in the script itself.

After running this script all the automated tests should have successfully run and a Django development server should be running in interactive mode in this terminal.

Rerun automated tests after modifying source code

Open another terminal and run the Unit and Integration tests within the container running the Django server:

To run only the Unit tests:

cd ChRIS_store
docker-compose -f docker-compose_dev.yml exec chris_store_dev python manage.py test --exclude-tag integration

To run only the Integration tests:

docker-compose -f docker-compose_dev.yml exec chris_store_dev python manage.py test --tag integration

To run all the tests:

docker-compose -f docker-compose_dev.yml exec chris_store_dev python manage.py test

Check code coverage of the automated tests

Make sure the store_backend/ dir is world writable. Then type:

docker-compose -f docker-compose_dev.yml exec chris_store_dev coverage run --source=plugins,pipelines,users manage.py test
docker-compose -f docker-compose_dev.yml exec chris_store_dev coverage report

Using HTTPie to play with the REST API

A simple GET request:

http http://localhost:8010/api/v1/

A simple POST request to register a new plugin app in the store:

First save the plugin representation json file by running the plugin with the --savejson flag:

docker run --rm -v /tmp/json:/json fnndsc/pl-simplefsapp simplefsapp --savejson /json

Then upload the plugin representation json file to the ChRIS Store as part of the POST request:

http -a cubeadmin:cubeadmin1234 -f POST http://localhost:8010/api/v1/plugins/ dock_image=fnndsc/pl-simplefsapp descriptor_file@/tmp/json/SimpleFSApp.json public_repo=https://github.com/FNNDSC/pl-simplefsapp name=pl-simplefsapp

An unauthenticated POST request to create a new ChRIS store user account:

http POST http://localhost:8010/api/v1/users/ Content-Type:application/vnd.collection+json Accept:application/vnd.collection+json template:='{"data":[{"name":"email","value":"[email protected]"}, {"name":"password","value":"newstoreuser1234"}, {"name":"username","value":"newstoreuser"}]}'

Destroy ChRIS Store dev environment

Stop and remove ChRIS Store services by running the make bash script from the repository source directory

./make.sh down

REST API Documentation

Available here.

Install Sphinx and the http extension (useful to document the REST API)

pip install Sphinx
pip install sphinxcontrib-httpdomain

Build the html documentation

cd docs/
make html

Learn More

If you are interested in contributing or joining us, Check here.

	class PluginFilter(FilterSet):
	"""
	Filter class for the Plugin model.
	"""
	min_creation_date = django_filters.IsoDateTimeFilter(field_name='creation_date',
	lookup_expr='gte')
	max_creation_date = django_filters.IsoDateTimeFilter(field_name='creation_date',
	lookup_expr='lte')
	owner_username = django_filters.CharFilter(
	field_name='meta__owner__username', lookup_expr='exact')
	name = django_filters.CharFilter(field_name='meta__name', lookup_expr='icontains')
	name_exact = django_filters.CharFilter(field_name='meta__name', lookup_expr='exact')
	title = django_filters.CharFilter(field_name='meta__title', lookup_expr='icontains')
	category = django_filters.CharFilter(field_name='meta__category',
	lookup_expr='icontains')
	type = django_filters.CharFilter(field_name='meta__type', lookup_expr='exact')
	description = django_filters.CharFilter(field_name='description',
	lookup_expr='icontains')
	name_title_category = django_filters.CharFilter(method='search_name_title_category')
	name_latest = django_filters.CharFilter(method='search_latest')
	name_exact_latest = django_filters.CharFilter(method='search_latest')

	def search_name_title_category(self, queryset, name, value):
	"""
	Custom method to get a filtered queryset with all plugins for which name or title
	or category matches the search value.
	"""
	# construct the full lookup expression.
	lookup = models.Q(meta__name__icontains=value)
	lookup = lookup \| models.Q(meta__title__icontains=value)
	lookup = lookup \| models.Q(meta__category__icontains=value)
	return queryset.filter(lookup)

	def search_latest(self, queryset, name, value):
	"""
	Custom method to get a filtered queryset with the latest version according to
	creation date of all plugins whose name matches the search value.
	"""
	if name == 'name_exact_latest':
	qs = queryset.filter(meta__name=value)
	return qs.order_by('-creation_date')[:1]
	else:
	qs = queryset.filter(meta__name__icontains=value)
	qs = qs.order_by('meta', '-creation_date')
	result_id_list = []
	meta_id = 0
	for pl in qs:
	pl_meta_id = pl.meta.id
	if pl_meta_id != meta_id:
	result_id_list.append(pl.id)
	meta_id = pl_meta_id
	return qs.filter(pk__in=result_id_list)

	class Meta:
	model = Plugin
	fields = ['id', 'name', 'name_latest', 'name_exact', 'name_exact_latest',
	'dock_image', 'type', 'category', 'owner_username', 'min_creation_date',
	'max_creation_date', 'title', 'version', 'description',
	'name_title_category']

	fields = ['id', 'name', 'name_latest', 'name_exact', 'name_exact_latest',
	'dock_image', 'type', 'category', 'owner_username', 'min_creation_date',
	'max_creation_date', 'title', 'version', 'description',
	'name_title_category']

	if default is None:
	raise serializers.ValidationError(
	{'descriptor_file': ['A default value is required for optional '
	'parameters.']})

fnndsc / chris_store Goto Github PK

chris_store's Introduction

ChRIS_store

ChRIS store development, testing and deployment

Abstract

Preconditions

Install latest Docker and Docker Compose

On a Linux machine make sure to add your computer user to the docker group

TL;DR

Production deployment on a single-machine Docker Swarm cluster

To get the production system up:

To tear down:

Development

Instantiate ChRIS Store dev environment

Rerun automated tests after modifying source code

Check code coverage of the automated tests

Using HTTPie to play with the REST API

A simple GET request:

A simple POST request to register a new plugin app in the store:

An unauthenticated POST request to create a new ChRIS store user account:

Destroy ChRIS Store dev environment

REST API Documentation

Learn More

chris_store's People

Contributors

Stargazers

Watchers

Forkers

chris_store's Issues

Current Situation

Proposal

Tasks

How It Works

Pagination

Flexible Usage

Implementation

Implementation Objectives

Description

Recommend Projects

Recommend Topics

Recommend Org

Jobs

On a Linux machine make sure to add your computer user to the `docker` group