marksmall / eoepca

EOEPCA system integration, test and deployment

EOEPCA system

EOEPCA Reference Implementation - System
Explore the docs »
View Demo · Report Bug · Request Feature

Table of Contents

• Table of Contents
• About The Project
• Getting Started
  • Hostnames and DNS
• System Documentation
• Technical Domains
  • User Management
  • Processing and Chaining
  • Resource Management
• Releases
• Issues
• License
• Contact
• Acknowledgements

About The Project

EO Exploitation Platform Common Architecture (EOEPCA)

The goal of the “Common Architecture” is to define and agree the technical interfaces for the future exploitation of Earth Observation data in a distributed environment. The Common Architecture will thus provide the interfaces to facilitate the federation of different EO resources into a “Network of EO Resources”. The “Common Architecture” will be defined using open interfaces that link the different Resource Servers (building blocks) so that a user can efficiently access and consume the disparate services of the “Network of EO Resources”.

This repository represents the system integration of the building blocks that comprise the Reference Implementation of the Common Architecture.

The system is designed for deployment to cloud infrastructure orchestrated by a Kubernetes cluster. We include here the automation required to provision, deploy and test the emerging EOEPCA system.

Getting Started

The EOEPCA system deployment comprises several steps. Instructions are provided for both cloud deployment, and local deployment for development purposes.

For the latest release (v1.1) ensure that the correct version of this README is followed.

The first step is to fork this repository into your GitHub account. Use of fork (rather than clone) is recommended to support our GitOps approach to deployment with Flux Continuous Delivery, which requires write access to your git repository for deployment configurations.
Having forked, clone the repository to your local platform...

$ git clone [email protected]:<user>/eoepca.git
$ cd eoepca
$ git checkout v1.1

NOTE that this clones the specific tag that is well tested. The develop branch should alternatively be used for the latest development.

Step	Cloud (OpenStack)	Local Developer
Infrastructure	CREODIAS	n/a (local developer platform)
Kubernetes Cluster	Rancher Kubernetes Engine	Minikube
EOEPCA System Deployment (flux)	EOEPCA GitOps	EOEPCA GitOps
EOEPCA System Deployment (Deployment Guide)	Deployment Guide	Deployment Guide
Acceptance Test	Run Test Suite	Run Test Suite

NOTE that, with release v1.1, the number of system components has been expanded to the point where it is more difficult to make a full system deployment in minikube, due to the required resource demands. Nevertheless, it is possible to make a minikube deployment to a single node with sufficient resources (4 cpu, 16GB) - as illustrated by the Deployment Guide.

NOTE also that the Deployment Guide provides a more detailed description of the deployment and configuration of the components, supported by some shell scripts that deploy the components directly using helm (rather than using flux GitOps). The Deployment Guide represents a more informative introduction, and the supporting scripts assume minikube out-of-the-box.

Hostnames and DNS

To ease development/testing, the EOEPCA deployment is configured to use host/service names that embed IP-addresses - which avoids the need to configure public nameservers, (as would be necessary for a production deployment). Our services are exposed through Kubernetes ingress rules that use name-based routing, and so simple IP-addresses are insufficient. Therefore, we exploit the services of nip.io that provides dynamic DNS in which the hostname->IP-adress mapping is embedded in the hostname.

Thus, we use host/service names of the form <service-name>.<public-ip>.nip.io, where the <public-ip> is the public-facing IP-address of the deployment. For cloud deployment the public IP is that of the cloud load-balancer, or for minikube it is the minikube ip - for example workspace.192.168.49.2.nip.io.

NOTE that we also maintain a deployment under the domain demo.eoepca.org - upon which this release has been prepared.

Our public endpoint address is baked into our deployment configuration - in particular the Kubernetes Ingress resources. To re-use our deployment configuration these Ingress values must be updated to suit your deployment environment.

System Documentation

• Use Case Analysis Document
• Master System Design Document
• Master System ICD Document

Technical Domains

User Management

Building Block	Repository	Documentation
Login Service	https://github.com/EOEPCA/um-login-service	https://eoepca.github.io/um-login-service/ https://github.com/EOEPCA/um-login-service/wiki
User Profile	https://github.com/EOEPCA/um-user-profile	https://eoepca.github.io/um-user-profile/ https://github.com/EOEPCA/um-user-profile/wiki
Policy Enforcement Point (PEP)	https://github.com/EOEPCA/um-pep-engine	https://eoepca.github.io/um-pep-engine/ https://github.com/EOEPCA/um-pep-engine/wiki
Policy Decision Point (PDP)	https://github.com/EOEPCA/um-pep-engine	https://eoepca.github.io/um-pep-engine/ https://github.com/EOEPCA/um-pdp-engine/wiki

Processing and Chaining

Building Block	Repository	Documentation
Application Deployment & Execution Service (ADES)	https://github.com/EOEPCA/proc-ades	https://eoepca.github.io/proc-ades/ https://github.com/EOEPCA/proc-ades/wiki
Processor Development Environment (PDE)	https://github.com/EOEPCA/proc-pde	https://github.com/EOEPCA/proc-pde/blob/master/README.md
Sample Application: s-expression	https://github.com/EOEPCA/app-s-expression	https://github.com/EOEPCA/app-s-expression/blob/main/README.md
Sample Application: nhi	https://github.com/EOEPCA/app-nhi	https://github.com/EOEPCA/app-nhi/blob/main/README.md

Resource Management

Building Block	Repository	Documentation
Resource Catalogue	https://github.com/geopython/pycsw	https://eoepca.github.io/rm-resource-catalogue/ https://docs.pycsw.org/en/latest/
Data Access Services	https://github.com/EOEPCA/rm-data-access/	https://eoepca.github.io/rm-data-access/
User Workspace	https://github.com/EOEPCA/rm-workspace-api/	https://eoepca.github.io/rm-workspace-api/

Releases

EOEPCA system releases are made to provide integrated deployments of the developed building blocks. The release history is as follows:

• 31/05/2022 - Release 1.1
• 24/12/2021 - Release 1.0
• 23/07/2021 - Release 0.9
• 10/03/2021 - Release 0.3
• 23/11/2020 - Release 0.2
• 13/08/2020 - Release 0.1.2
• 06/08/2020 - Release 0.1.1
• 22/06/2020 - Release 0.1

Issues

See the open issues for a list of proposed features (and known issues).

License

The EOEPCA SYSTEM is distributed under the Apache-2.0 License. See LICENSE for more information.

Building-blocks and their sub-components are individually licensed. See their respective source repositories for details.

Contact

Project Link: Project Home (https://eoepca.github.io/)

Acknowledgements

• README.md is based on this template by Othneil Drew.