00mjk / cloudbridge

A consistent interface to multiple IaaS clouds; in Python.

CloudBridge provides a consistent layer of abstraction over different Infrastructure-as-a-Service cloud providers, reducing or eliminating the need to write conditional code for each cloud.

Documentation

Detailed documentation can be found at http://cloudbridge.cloudve.org.

Build Status Tests

Provider/Environment	Python 3.8
Amazon Web Services
Google Cloud Platform
Microsoft Azure
OpenStack
Mock Provider

Installation

Install the latest release from PyPi:

pip install cloudbridge[full]

For other installation options, see the installation page in the documentation.

Usage example

To get started with CloudBridge, export your cloud access credentials (e.g., AWS_ACCESS_KEY and AWS_SECRET_KEY for your AWS credentials) and start exploring the API:

from cloudbridge.factory import CloudProviderFactory, ProviderList

provider = CloudProviderFactory().create_provider(ProviderList.AWS, {})
print(provider.security.key_pairs.list())

The exact same command (as well as any other CloudBridge method) will run with any of the supported providers: ProviderList.[AWS | AZURE | GCP | OPENSTACK]!

Citation

N. Goonasekera, A. Lonie, J. Taylor, and E. Afgan, "CloudBridge: a Simple Cross-Cloud Python Library," presented at the Proceedings of the XSEDE16 Conference on Diversity, Big Data, and Science at Scale, Miami, USA, 2016. DOI: http://dx.doi.org/10.1145/2949550.2949648

Quick Reference

The following object graph shows how to access various provider services, and the resource that they return.

Design Goals

1. Create a cloud abstraction layer which minimises or eliminates the need for cloud specific special casing (i.e., Not require clients to write if EC2 do x else if OPENSTACK do y.)
2. Have a suite of conformance tests which are comprehensive enough that goal 1 can be achieved. This would also mean that clients need not manually test against each provider to make sure their application is compatible.
3. Opt for a minimum set of features that a cloud provider will support, instead of a lowest common denominator approach. This means that reasonably mature clouds like Amazon and OpenStack are used as the benchmark against which functionality & features are determined. Therefore, there is a definite expectation that the cloud infrastructure will support a compute service with support for images and snapshots and various machine sizes. The cloud infrastructure will very likely support block storage, although this is currently optional. It may optionally support object storage.
4. Make the CloudBridge layer as thin as possible without compromising goal 1. By wrapping the cloud provider's native SDK and doing the minimal work necessary to adapt the interface, we can achieve greater development speed and reliability since the native provider SDK is most likely to have both properties.

Contributing

Community contributions for any part of the project are welcome. If you have a completely new idea or would like to bounce your idea before moving forward with the implementation, feel free to create an issue to start a discussion.

Contributions should come in the form of a pull request. We strive for 100% test coverage so code will only be accepted if it comes with appropriate tests and it does not break existing functionality. Further, the code needs to be well documented and all methods have docstrings. We are largely adhering to the PEP8 style guide with 80 character lines, 4-space indentation (spaces instead of tabs), explicit, one-per-line imports among others. Please keep the style consistent with the rest of the project.

Conceptually, the library is laid out such that there is a factory used to create a reference to a cloud provider. Each provider offers a set of services and resources. Services typically perform actions while resources offer information (and can act on itself, when appropriate). The structure of each object is defined via an abstract interface (see cloudbridge/providers/interfaces) and any object should implement the defined interface. If adding a completely new provider, take a look at the provider development page in the documentation.