chaostoolkit / chaostoolkit-documentation

The Chaos Toolkit documentation

Chaos Toolkit - Chaos Engineering for All Engineers

Community • Installation • Tutorials • Reference • ChangeLog

---

Chaos Toolkit - Chaos Engineering for All Engineers

The Chaos Toolkit, or as we love to call it “ctk”, is a simple CLI-driven tool who helps you write and run Chaos Engineering experiment. It supports any target platform you can think of through existing extensions or the ones you write as you need.

Chaos Toolkit is versatile and works really well in settings where other Chaos Engineering tools may not fit: cloud environments, datacenters, CI/CD, etc.

Install or Upgrade

Provided you have Python 3.8+ installed, you can install it as follows:

$ pip install -U chaostoolkit

Getting Started

Once you have installed the Chaos Toolkit you can use it through its simple command line tool.

Running an experiment is as simple as:

$ chaos run experiment.json

Get involved!

Chaos Toolkit's mission is to provide an open API to chaos engineering in all its forms. As such, we encourage and welcome you to join our open community Slack team to discuss and share your experiments and needs with the community. You can also use StackOverflow to ask any questions regarding using the Chaos Toolkit or Chaos Engineering.

If you'd prefer not to use Slack then we welcome the raising of GitHub issues on this repo for any questions, requests, or discussions around the Chaos Toolkit.

Finally you can always email [email protected] with any questions as well.

Contribute

Contributors to this project are welcome as this is an open-source effort that seeks discussions and continuous improvement.

From a code perspective, if you wish to contribute, you will need to run a Python 3.8+ environment. Please, fork this project, write unit tests to cover the proposed changes, implement the changes, ensure they meet the formatting standards set out by ruff, add an entry into CHANGELOG.md, and then raise a PR to the repository for review

The project is driven by PDM, so install it and you can run the following commands:

$ pdm install
$ pdm run test
$ pdm run format
$ pdm run lint

The Chaos Toolkit projects require all contributors must sign a Developer Certificate of Origin on each commit they would like to merge into the master branch of the repository. Please, make sure you can abide by the rules of the DCO before submitting a PR.

chaostoolkit-documentation's Issues

Add chaostoolkit core developer documentation to reference, and link from contributing page.

Add full documentation on how to extend the Chaos Toolkit for your own probes and actions

Add documentation on how to extend an experiment with locally available python code

Provides a lightweight way to extend the CTK for an experiment, or experiments, when you're not sure you need the full capabilities of a separately distributed extension module.

Clarify the status of a python probe tolerance

The spec isn't quite clear on what is expected to be returned by a Python probe tolerance when it's successful or not.

Add documentation for creating an extension to chaos toolkit.

README.md should provide an entry point into the docs rather than being about the docs builder framework

Or at least have a link to the gh-pages or wherever the docs are live

Typo in the docs

Hey. Just spying on you both ;-) and noticed a typo.

The chaostoolkit-documehtation is the

On this page: "http://chaostoolkit.org/technology/"

Write tutorial on tolerances

The current doc isn't clear enough on the variety nor how to use them.

Add instructions to kill daemon services in tutorial to tidy-up after the tutorial is completed

Add resources and timeline pages to docs

Ensure Creative Commons Attribution license is specified.

Add overview diagram and explanation to associated GitHub orgs, community projects

Suggest adding to the "Contribute" section a diagram and explanation to help people navigate the various core and external projects, that we're aware of, that relate to the Chaos Toolkit.

Explain how to workon on an extension with discover without installing everytime

When one works on an extension, the discover method is a pain in its default behavior because it keeps installing the existing official distribution package instead of picking up the developed one.

We should document how to get that worked around in http://chaostoolkit.org/reference/developing/guidelines/#creating-an-extension

First, run python setup.py develop from your extension workspace. Then run chaos discover --no-install ext-name so that discover does not override the installation everytime.

Add introductory Chaos Toolkit workflow video

Clarify that configuration and secrets are all made available automatically to drivers

Add introduction page to drivers reference

Inconsistency between experiment example and the documentation.

The example for 'Minimal Experiment' is not consistent with what documentation says. In the document, 'type' for the PROBE must be 'probe', and 'type' for the PROVIDER must be 'python', 'http', or 'process', but in the example, 'type' for the PROBE is 'python', and PROVIDER is missing the 'type'. Here's that example. https://docs.chaostoolkit.org/reference/api/experiment/#minimal-experiment

Document how to add time constraints to an experiment

It is common, to run an experiment for a certain period of time and not indefinitely. In addition, sometimes, the operator may not be on the keyboard to stop the fault injection. Therefore, a timer or time controls would be useful when running an experiment. It would be very helpful to include a section mentioning all the available options.

Provide a full experiment JSON description as part of the open API for chaos engineering section of the docs

Currently we do have a concepts section but now that the experiment JSON description is stabilising it's time to document this as part of the open API, which is this project's purpose.

Create tutorial to address creating a new extension with probes for observability

Goals are to explain how to build an extension from scratch, to explain the role of probes for observability, and to encourage contributions to the Prometheus extension as well as further extensions.

Missing info on Settings page

The Settings page is missing information:
https://docs.chaostoolkit.org/reference/usage/settings/

It might be somewhere else under Documentation.

The 404 page for the documentation site looks super odd with auto-generated text.

As originally reported here: https://github.com/chaostoolkit/chaostoolkit-lib/tree/issue83

Create tutorial on how to simply extend the toolkit for a simple process-based custom action and probe

Add more concrete examples of using the different styles of tolerance

In particular json path tolerances.

Documentation wrong link

In the readme, the documentation link ("Get to the documentation of the Chaos Toolkit.") is pointing to the chaostoolkit home page https://chaostoolkit.org/ instead of the doc subdomain https://docs.chaostoolkit.org/

Fix typo in pip command for cookiecutter

Current incorrect command:

> pip install - U cookiecutter

Should be changed to:

> pip install -U cookiecutter

The documentation is missing a page on the settings, currently returning a 404

Access https://docs.chaostoolkit.org/reference/usage/settings/ to see the 404

Is the schema valid?

I'm trying to use the schema included in this repo to validate my JSON creations -- I notice the app itself is just checking keys:

https://github.com/chaostoolkit/chaostoolkit-lib/blob/b14b122777b5b1d580ef0d4c34d7e55587b27fbf/chaoslib/hypothesis.py#L35

I'd suggest switching to a schema that the app validates when running a command -- it's a very nice way of sanitizing your data, but it can also be re-used when other devs try to do something like generating these json experiment files.

schema link: https://github.com/chaostoolkit/chaostoolkit-documentation/blob/master/schema.json

Refine definition of steady state hypothesis concept

Get Started with the Chaos Toolkit (windows)

Add merchandise page to docs resources

Add some clarification to the section Deploy Chaos Toolkit as a Kubernetes Operator

Under https://docs.chaostoolkit.org/deployment/k8s/operator/#deploy-the-operator

Should there be a note before: "Next, simply run the following:" explaining you will need to be connected to a cluster prior to running the kustomize command. and perhaps a link to https://cloud.google.com/kubernetes-engine/docs/quickstart#create_cluster

After the section: "Look at the Chaos Toolkit running:" - should we add a not about viewing the logs:

kubectl -n chaostoolkit-run logs chaostoolkit-<pod-id>

After "They will be injected into the Chaos Toolkit’s pod as environment variables."
Add a link to the example showing the use of environment variables.

After the code section of "configMapName", add a link to the example using the config map.

At the end of "https://docs.chaostoolkit.org/deployment/k8s/operator/#load-the-experiment-from-a-url" add a link to the example that run's from a URL.

Move the chaos toolkit install instructions to below the dependency installation for the sample service

Improve http probe specification to clarify POST requests

It isn't clear what is sent based on the content-type.

Notably the content is not set to the body property when sending application/json but instead it sends the arguments payload.

Suggest moving `Usage` to second entry under `Reference`

Broken link

Theres a broken link on the documentation index page

https://github.com/chaostoolkit/chaostoolkit-documentation/blob/master/sources/index.md

Suggest refactoring the word 'bail' when defining the behaviour of SSH

In the following documentation quote

In the case of a scalar or the sequence, the tolerance validation MUST be strict. The value returned by the Probe MUST be checked against the scalar value. The experiment MUST bail when both fail to match.

The word 'bail' does not convey clearly and without any misconceptions the various outcomes which can happen during a run of an experiment based (first run of SSH - stops experiment, run after method marks deviation).

Add explanation of CLI plugins and tutorial

Update documentation to be use-case-centric

The current docs need a complete overhaul to include the new workflow, links to the new tutorials on Katacoda, and finally an emphasis on using the toolkit with target environment levels such as Kubernetes, PCF etc.

Add instructions for submodule installation when developing the docs

the verify_tls parameter is not documented

The http probe allows to disable cert validation via the verify_tls parameter. But that is not documented.

Link to Python page is broken on Chaos toolkit install page

missing action: create_service_endpoint

Why can i remove a service endpoint but i cannot create one ?
Maybe i miss something in the documentation

I'm running an experimentation where i have a springboot microservice and a mysql statefulset
I want to remove mysql service endpoint and check about my microservice resilience
But at the end of the experiment I dont have mysql service endpoint and i need to recreate it manually

Is there a way to recreate it with chaostoolkit ? # #

The drivers/azure/#usage paragraph contains outdated information

The json example contains an outdated information

{
    "type": "action",
    "name": "start-service-factory-chaos",
    "provider": {
        "type": "python",
        "module": "chaosazure.factory.actions",
        "func": "start_chaos",
        "secrets": ["azure"],
        "arguments": {
            "parameters": {
                "TimeToRunInSeconds": 45
            }
        }
    }
},
{
    "type": "action",
    "name": "stop-service-factory-chaos",
    "provider": {
        "type": "python",
        "module": "chaosazure.factory.actions",
        "func": "stop_chaos",
        "secrets": ["azure"]
    }
}

The modules point to chaosazure.factory.actions. The latest version of the Chaostoolkit has the module chaosazure.fabric.actions instead. Consider also to give new names to the declared actions, e.g. "stop-service-fabric-chaos" and "start-service-fabric-chaos".

Experiment must declare a steady state hypothesis?

https://docs.chaostoolkit.org/reference/api/experiment/

currently says experiment MUST declare

a steady-state-hypothesis property

I don't believe that is true

Change title typo on tutorial page

"Get the code" in the tutorial should be more obviously mandatory

Documentation for Azure Extensions is Empty

The docs page at https://docs.chaostoolkit.org/drivers/azure/ is blank.
Refer screenshot below.

Add virtual environment hint to sample instructions

Part of the 'Get Started with...' has JSON without an end curly brace

The section titled Declare an Experiment to Observe the Weakness has a JSON example, which lacks the end curly brace.

Allow to be explicit about which rollback(s) to play for a given activity

Currently, rollbacks are all played (or none) sequentially. But, as per chaostoolkit/chaostoolkit#177, it would be useful to let users tell the CTK how to linka rollback (or set of rollbacks) to an action in the method.

The idea would then be to say "only play rollbacks of actions that were executed successfully".

This is an addition to the specification.