isabella232 / layer-apache-bigtop-base

Overview

This is the base layer for charms that wish to take advantage of the Apache Bigtop platform for configuring and deploying services via puppet. Including this layer gives you access to a Python class called Bigtop, which contains, among other things, tools for triggering a puppet run.

Usage

To create a charm using this base layer, first you must include it in your layer.yaml file:

includes: ['layer:apache-bigtop-base']

This will fetch the layer from interfaces.juju.solutions and incorporate it into your charm. To use the Bigtop class, import it, then call its render_site_yaml and trigger_puppet methods in sequence, like so (you might notice that Bigtop also has an install method -- this is run automatically in the reactive handlers of this layer; you don't need to call it yourself):

from charms import layer

# Setup arguments to pass to .render_site_yaml
hosts = {'some_bigtop_service': <host>}
roles = ['some_bigtop_service_nodetype0', 'some_bigtop_service_nodetype1']
override = {'some_bigtop_service::somekey': <somevalue>, ...}

# Trigger a puppet run
bigtop = layer.apache_bigtop_base.Bigtop()
bigtop.render_site_yaml(hosts, roles, override)
bigtop.trigger_puppet()

This tells Bigtop to run puppet and install your service.

How does Bigtop know what to install? You tell it what to install by passing a list of roles to render_site_yaml. You may wish to consult [this list of valid roles][roles] to see what is available.

[roles]: https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/manifests/cluster.pp)

Note: Bigtop is also able to generate a list of roles from a component. The Bigtop class is theoretically able to infer components from the hosts dict that you pass to render_site_yaml. As of this writing, this code path is not well tested, so you may want to specify roles explicitly. List of valid components.

Reactive states

This layer will set the following states:

• bigtop.available: This state is set after Bigtop.install has been run. At this point, you are free to invoke Bigtop.render_site_yaml.

• bigtop.version.changed: (experimental) This state is set after a user has changed the bigtop_version config option. At this point, your charm can call Bigtop.check_bigtop_repo_package to determine if a new package version is available. See the Bigtop Versions section for more details.

Layer configuration

Bigtop actually takes care of most of the configuration details for you. You can specify overrides if needed -- either in code or in your layer.yaml -- and pass those in in an overrides dict when you call Bigtop.render_site_yaml.

Depending on your service, you'll also need to tell Bigtop where to find important hosts, by passing a dict of hosts to Bigtop.render_site_yaml.

The leadership layer may come in handy when it comes to populating that hosts dict.

Actions

reinstall (experimental)

This layer provides a generic reinstall action that will invoke Bigtop.reinstall_repo_packages. This is meant to serve as a template for writing a Bigtop charm-specific action used to upgrade packages when a new Bigtop repository has been configured. See the Bigtop Versions section for more details. For an example charm that extends this generic action, see the Spark reinstall action.

smoke-test

This layer provides a generic smoke-test action that will invoke Bigtop smoke tests. This is meant to serve as a template for writing a Bigtop charm-specific test. See the Zookeeper smoke test as an example of extending this generic action.

Bigtop Source

Bigtop charms require project source that includes the puppet manifests and modules used to deploy Bigtop software. This layer provides the ability to specify a zip archive of the Bigtop source repository. By default, the latest tagged release of the Bigtop source repository is used. You may specify a different archive when deploying a Bigtop charm. For example:

juju deploy hadoop-namenode --resource bigtop-repo=/tmp/branch-1.1.zip

Bigtop Versions

Note: Support for changing Bigtop versions at runtime should be considered experimental in the current release. Users are encouraged to try this feature in development environments. Changing the Bigtop version in a production deployment is not yet recommended.

The bigtop_version config option can be used to change the version of Bigtop at runtime:

juju config <charm> bigtop_version=1.2.1

When changed, this layer will configure a new repository, fetch the new bigtop source, patch it as needed, and set the bigtop.version.changed state. When this state is set, your charm may query package version changes using Bigtop.check_bigtop_repo_package. If a version has changed, your charm may then inform the user that an upgrade is available and direct them to use the reinstall action to upgrade each affected unit.

If a user wishes to roll back to a previous Bigtop version, they simply re-configure bigtop_version:

juju config <charm> bigtop_version=1.2.0

This layer does not perform any automatic upgrades. User intervention is always required by calling the reinstall action to update system packages.

Unit Tests

To run unit tests for this layer, change to the root directory of the layer in a terminal, and run tox -c tox_unit.ini. To tweak settings, such as making the tests more or less verbose, edit tox_unit.ini.

Contributing

This charm has been built on top of the Apache Bigtop project. Bug fixes, new features and other contributions to this charm are welcome, and encouraged!

For instructions on finding source code, formatting pull requests, and getting help, please visit the Contributing page on our wiki.

Contact Information

• [email protected]

Resources

• Apache Bigtop home page
• Apache Bigtop issue tracking
• Apache Bigtop mailing lists
• Juju Big Data
• Juju Bigtop charms
• Juju mailing list