fairwindsops / reckoner

Declaratively install and manage multiple Helm chart releases

Command line helper for helm.

This utility adds to the functionality of Helm in multiple ways:

• Creates a declarative syntax to manage multiple releases in one place
• Allows installation of charts from a git commit/branch/release

Documentation

Check out the documentation at docs.fairwinds.com

Requirements

• helm (>= 3.0.0), installed and initialized

Helm2 is untested as of v4.3.0. The maintainers of helm have deprecated helm2.

Quickstart

In course.yml, write:

namespace: default
charts:
  grafana:
    namespace: grafana
    values:
      image:
        tag: "6.2.5"
  polaris-dashboard:
    namespace: polaris-dashboard
    repository:
      git: https://github.com/FairwindsOps/charts
      path: stable
    chart: polaris

Then run:

reckoner plot course.yml --run-all

Grafana and Polaris should now be installed on your cluster!

Importing Existing Releases

Importing existing releases is experimental and the result should be reviewed.

If you're already using Helm but want to start using reckoner, you can use reckoner import to facilitate your migration.

We recommend carefully examining the output of a reckoner diff before relying on any imported course.yml definitions.

Join the Fairwinds Open Source Community

The goal of the Fairwinds Community is to exchange ideas, influence the open source roadmap, and network with fellow Kubernetes users. Chat with us on Slack or join the user group to get involved!

Other Projects from Fairwinds

Enjoying Reckoner? Check out some of our other projects:

• Polaris - Audit, enforce, and build policies for Kubernetes resources, including over 20 built-in checks for best practices
• Goldilocks - Right-size your Kubernetes Deployments by compare your memory and CPU settings against actual usage
• Pluto - Detect Kubernetes resources that have been deprecated or removed in future versions
• Nova - Check to see if any of your Helm charts have updates available
• rbac-manager - Simplify the management of RBAC in your Kubernetes clusters

Or check out the full list

reckoner's Issues

Values file that works with helm does not work with reckoner

This values file doesn't work with reckoner but works with a normal install:

prometheus:
  url: http://linkerd-prometheus.linkerd.svc

rules:
  default: false
  custom:
    - seriesQuery: 'response_latency_ms_bucket{namespace!="",pod!=""}'
      resources:
        template: '<<.Resource>>'
      name:
        matches: '^(.*)_bucket$'
        as: "${1}_50th"
      metricsQuery: 'histogram_quantile(0.50, sum(irate(<<.Series>>{<<.LabelMatchers>>, direction="inbound"}[5m])) by (le, <<.GroupBy>>))'

    - seriesQuery: 'response_latency_ms_bucket{namespace!="",pod!=""}'
      resources:
        template: '<<.Resource>>'
      name:
        matches: '^(.*)_bucket$'
        as: "${1}_95th"
      metricsQuery: 'histogram_quantile(0.95, sum(irate(<<.Series>>{<<.LabelMatchers>>, direction="inbound"}[5m])) by (le, <<.GroupBy>>))'

    - seriesQuery: 'response_latency_ms_bucket{namespace!="",pod!=""}'
      resources:
        template: '<<.Resource>>'
      name:
        matches: '^(.*)_bucket$'
        as: "${1}_99th"
      metricsQuery: 'histogram_quantile(0.99, sum(irate(<<.Series>>{<<.LabelMatchers>>, direction="inbound"}[5m])) by (le, <<.GroupBy>>))'

[ do-sfo2-reckoner-demo   ] [pw]  1m0s
✓ reckoner --log-level=DEBUG plot course.yaml  --heading prometheus-adapter
2019-01-08 10:24:28 root[64881]    DEBUG helm help
2019-01-08 10:24:28 root[64881]    DEBUG helm version --short --server
2019-01-08 10:24:29 root[64881]    DEBUG Installing repository: {'url': 'https://kubernetes-charts-incubator.storage.googleapis.com', 'name': 'incubator'}
2019-01-08 10:24:29 root[64881]    DEBUG helm repo list
2019-01-08 10:24:29 root[64881]    DEBUG Chart repository incubator already installed
2019-01-08 10:24:29 root[64881]    DEBUG Installing repository: {'url': 'https://kubernetes-charts.storage.googleapis.com', 'name': 'stable'}
2019-01-08 10:24:29 root[64881]    DEBUG helm repo list
2019-01-08 10:24:30 root[64881]    DEBUG Chart repository stable already installed
2019-01-08 10:24:30 root[64881]    DEBUG Installing repository: {'url': 'http://influx-charts.storage.googleapis.com', 'name': 'influx'}
2019-01-08 10:24:30 root[64881]    DEBUG helm repo list
2019-01-08 10:24:30 root[64881]    DEBUG Chart repository influx already installed
2019-01-08 10:24:30 root[64881]    DEBUG helm repo update
2019-01-08 10:24:32 root[64881]    DEBUG Helm Minimum Version is: 2.12.1
2019-01-08 10:24:32 root[64881]    DEBUG helm version --short --client
2019-01-08 10:24:32 root[64881]    DEBUG Helm Installed Version is 2.12.1
2019-01-08 10:24:32 root[64881]    DEBUG Reckoner Minimum Version is 0.10.1
2019-01-08 10:24:32 root[64881]    DEBUG Reckoner Installed Version is 0.10.3
2019-01-08 10:24:32 root[64881]    DEBUG helm version --short --client
2019-01-08 10:24:32 root[64881]     INFO Installing prometheus-adapter
2019-01-08 10:24:32 root[64881]    DEBUG helm repo list
2019-01-08 10:24:32 root[64881]    DEBUG Chart repository stable already installed
2019-01-08 10:24:32 root[64881]    DEBUG Updating chart dependencies: stable/prometheus-adapter
2019-01-08 10:24:32 root[64881]    ERROR Helm upgrade failed. Rolling back prometheus-adapter
2019-01-08 10:24:32 root[64881]    DEBUG Traceback (most recent call last):
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/course.py", line 122, in plot
    chart.install(namespace=self.namespace, context=self.context)
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/chart.py", line 214, in install
    for file in self.files:
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/chart.py", line 87, in files
    return dict(self._chart.get('files', []))
ValueError: dictionary update sequence element #0 has length 31; 2 is required

2019-01-08 10:24:32 root[64881]    ERROR ERROR: Some charts failed to install and were rolled back
2019-01-08 10:24:32 root[64881]    ERROR  - prometheus-adapter
[ do-sfo2-reckoner-demo   ] [pw]
✓ helm delete --purge prometheus-adapter
Error: release: "prometheus-adapter" not found
[ do-sfo2-reckoner-demo   ] [pw]
 1 ✘ helm install -n^C
[ do-sfo2-reckoner-demo   ] [pw]
 130 ✘ helm install stable/prometheus-adapter --name prometheus-adapter -f prometheus-adapter.values.yml --namespace linkerd
NAME:   prometheus-adapter
LAST DEPLOYED: Tue Jan  8 10:25:08 2019
NAMESPACE: linkerd
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRole
NAME                                 AGE
prometheus-adapter-server-resources  2s
prometheus-adapter-resource-reader   2s

==> v1/Service
NAME                TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)  AGE
prometheus-adapter  ClusterIP  10.245.111.139  <none>       443/TCP  2s

==> v1/Deployment
NAME                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
prometheus-adapter  1        1        1           0          2s

==> v1/ConfigMap
NAME                DATA  AGE
prometheus-adapter  1     2s

==> v1/ServiceAccount
NAME                SECRETS  AGE
prometheus-adapter  1        2s

==> v1/ClusterRoleBinding
NAME                                      AGE
prometheus-adapter:system:auth-delegator  2s
prometheus-adapter-resource-reader        2s
prometheus-adapter-hpa-controller         2s

==> v1/RoleBinding
NAME                            AGE
prometheus-adapter-auth-reader  2s

==> v1beta1/APIService
NAME                           AGE
v1beta1.custom.metrics.k8s.io  2s

==> v1/Pod(related)
NAME                                READY  STATUS             RESTARTS  AGE
prometheus-adapter-9876c8dc6-q8d6r  0/1    ContainerCreating  0         2s


NOTES:
prometheus-adapter has been deployed.
In a few minutes you should be able to list metrics using the following command:

  kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1

Output manifest from Reckoner

CLI command to output reckoner courses to output manifest so it can be fed to other CD tools like spinnaker.

https://helm.sh/docs/helm/#helm-template

Feature Request: Multiple cluster support

It would be great to apply a course to multiple clusters at once. For example, maybe a list of kubectl contexts could be defined and reckoner would apply the file to each of them:

namespace: kube-system
repository: stable
contexts:
  - cluster-1
  - cluster-2
  - cluster-3

`values_strings` is not using `--set-string`

values_strings from the course.yaml is not being converted to --set-string on the command line args

Add a reason to the chart failure errors

As per discussion in #77, it would be nice if there was a reason attached to a chart install failure. Perhaps an additional property to contain the reason?

Reckoner 0.8.2 ignores the context field

When setting context to pick a kube context, reckoner ignores this field entirely and just uses the current context.

MinimumVersion Error Handling Broken

I would expect to see a logged error rather than a stacktrace.

[ do-sfo2-sudermanjr   ] [pw]  1m25s
✓ reckoner plot course.yaml --heading external-dns
Traceback (most recent call last):
  File "/Users/asuderma/.pentagon/pw/bin/reckoner", line 11, in <module>
    load_entry_point('reckoner', 'console_scripts', 'reckoner')()
  File "/Users/asuderma/.pentagon/pw/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/asuderma/.pentagon/pw/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/asuderma/.pentagon/pw/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/asuderma/.pentagon/pw/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/asuderma/.pentagon/pw/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/asuderma/.pentagon/pw/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/cli.py", line 53, in plot
    h = Reckoner(file=file, dryrun=dry_run, debug=debug, helm_args=helm_args, local_development=local_development)
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/reckoner.py", line 72, in __init__
    self.course = Course(file)
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/course.py", line 77, in __init__
    self._compare_required_versions()
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/course.py", line 159, in _compare_required_versions
    raise MinimumVersionException("helm Minimum Version {} not met.".format(helm_minimum_version))
reckoner.exception.MinimumVersionException: helm Minimum Version 2.12.1 not met.

Post Install Hooks Should Run Relative to Course.yml Path

This would help make reckoner behavior more consistent for teams that may not consistently be running a reckoner plot from the same path.

Container Spec env Var interpolation issues

Currently if you try to use "variable reference" in the set-values: {} block of the course.yml, you will get an unexpected exit due to Template(arg).substitute(os.environ) failing on ValueError. The substitution picks up $(OTHER_ENV_VAR) as something it should act upon, but returns a ValueError due to not conforming to an expected format.

The behavior could potentially be:

• Pass through an warn the user of a strange interpolation
• Completely ignore the error and leave the arg as "uninterpolated"
• Something else?

To reproduce try to add $(SOMETHING) to a value processed in the course.yaml.

Provide meaningful error message for unset Environment Variables

Current sample debug logging shows that the only items that log by default are not very helpful (Helm upgrade failed. Rolling back datadog).

reckoner --log-level=DEBUG plot --dry-run --debug --heading datadog course.yml
2019-01-07 12:37:15 robs-mbp.lan root[29719] WARNING The --debug flag will be deprecated.  Please use --helm-args or --dry-run instead.
2019-01-07 12:37:15 robs-mbp.lan root[29719] DEBUG helm help
2019-01-07 12:37:15 robs-mbp.lan root[29719] DEBUG helm version --short --server
2019-01-07 12:37:16 robs-mbp.lan root[29719] DEBUG Installing repository: {'url': 'https://kubernetes-charts-incubator.storage.googleapis.com', 'name': 'incubator'}
2019-01-07 12:37:16 robs-mbp.lan root[29719] DEBUG helm repo list
2019-01-07 12:37:16 robs-mbp.lan root[29719] DEBUG Chart repository incubator already installed
2019-01-07 12:37:16 robs-mbp.lan root[29719] DEBUG Installing repository: {'url': 'https://kubernetes-charts.storage.googleapis.com', 'name': 'stable'}
2019-01-07 12:37:16 robs-mbp.lan root[29719] DEBUG helm repo list
2019-01-07 12:37:17 robs-mbp.lan root[29719] DEBUG Chart repository stable already installed
2019-01-07 12:37:17 robs-mbp.lan root[29719] DEBUG Installing repository: {'url': 'https://charts.gitlab.io', 'name': 'gitlab'}
2019-01-07 12:37:17 robs-mbp.lan root[29719] DEBUG helm repo list
2019-01-07 12:37:17 robs-mbp.lan root[29719] DEBUG Chart repository gitlab already installed
2019-01-07 12:37:17 robs-mbp.lan root[29719] DEBUG helm repo update
2019-01-07 12:37:18 robs-mbp.lan root[29719] INFO Installing datadog
2019-01-07 12:37:18 robs-mbp.lan root[29719] DEBUG helm repo list
2019-01-07 12:37:18 robs-mbp.lan root[29719] DEBUG Chart repository stable already installed
2019-01-07 12:37:18 robs-mbp.lan root[29719] ERROR Helm upgrade failed. Rolling back datadog
2019-01-07 12:37:18 robs-mbp.lan root[29719] DEBUG Traceback (most recent call last):
  File "/Users/rob/.pentagon/venv/lib/python2.7/site-packages/reckoner/course.py", line 118, in plot
    chart.install(namespace=self.namespace, context=self.context)
  File "/Users/rob/.pentagon/venv/lib/python2.7/site-packages/reckoner/chart.py", line 226, in install
    self.__check_env_vars()
  File "/Users/rob/.pentagon/venv/lib/python2.7/site-packages/reckoner/chart.py", line 109, in __check_env_vars
    raise Exception("Missing requirement environment variable: {}".format(e.args[0]))
Exception: Missing requirement environment variable: RABBIT_DEV_PASS

--recreate-pods set twice in tests for chart installation

When fixing #34 and re-enabling the chart installation tests, I had to add --recreate-pods twice to the list of expected arguments in order for the tests to pass.

https://github.com/reactiveops/reckoner/blob/a4452651e11d9de533ca2d503d26cd3f18dbe09f/tests/test_reckoner.py#L374
https://github.com/reactiveops/reckoner/blob/a4452651e11d9de533ca2d503d26cd3f18dbe09f/tests/test_reckoner.py#L393

README Fixes

The README still suggests installing with pip directly from git, and it still has the helm plugin as an option. We should remove these and stick to just pip install reckoner

Add capability for additional args

Carried over from: FairwindsOps/autohelm#55

All in all, I think we could add a new multiple command line arg --helm-args, add a helm_args section to the course.yml spec and also per chart. As part of that, also deprecate the other command line args that are essentially just --helm-args but leaving the ones that require special logic. This would provide more than intended functionality, but also reduce the amount of changes required when new Helm args are released.

• add --helm-args to command line : FairwindsOps/autohelm#54
• add helm_args section to course.yml
• add deprecation warnings to --debug flag
• remove --debug flag

Keep the existing reckoner --dry-run flag that will imply a helm dry-run and skip hooks.

Errors in Post Install Hook Are Not Logged

It seems like whatever output comes from a post install hook doesn't actually make it to stdout. That means if you happening to be referencing an invalid path, there will be nothing in the Reckoner output to indicate there's a problem with your config.

Reference external default values files

Feature request to be able to reference an external value file for a chart. This concept is to be able to dry up repetitive base configurations across course.yml in different environments. These files could be layered, with deep merging preferring the latest item defined.

Example

charts:
  my-release-cert-manager:
    chart: cert-manager
    version: 0.6.0
    remote-default-values:
    - type: <type> # default to detect type if `type` key isn't defined
      source: <url> # utilized by the type implementation provider (with protocol)
      path: <path> # path to get/clone to a temp file locally

I'm envisioning supported values in the remote-default-values key would be:

remote-default-values:
- source: https://my.domain/
  path: /path/to/file
- type: git
  source: ssh://[email protected]:org/my-repo.git
  path: /path/to/default.yml
- type: git
  source: [email protected]:organiz/my-repo.git
  path: /my-default.yml
# Maybe if we're feeling fancy, use the AWS API to pull this object
- type: s3
  source: s3://my-bucket-of-defaults/
  path: /path/to/default/object.yml

Protocol detection would be based on git urls as well as protocol support for standard HTTP(s). If a protocol is tried and failed we should bubble up that error.

Thoughts and Considerations

• Deep merging of values is handled with helm's default behavior (we don't deserialize the loaded remote yaml)
• Helm command definition ordering priority of values
  1. set-values
  2. values
  3. files (values files locally)
  4. remote-default-values
  5. remote chart values.yaml defaults
• Not included in MVP
  • variable interpolation in the default templates

return code 0 on failed deployment

hi guys,
when im installing my charts (happens even if i use the '--only' flag) ive noticed that if the helm command isnt able to install the chart reckoner still give back a return code 0.

is it on purpose? theres a way to make the installation strict?

Getting an OrderedDict in output

2018-12-05 10:25:24 root[91265]    ERROR Error with subprocess call: helm upgrade --install aws-iam-authenticator reactiveops/aws-iam-authenticator --namespace=infra --version=v1.1.1 --set=cluster_id=REDACTED --set=users[0].name=reactiveops --set=users[0].groups[0]=system:masters --set=roles[0]=OrderedDict([('name', 'KubeAdmin'), ('groups', ['system:masters'])]) --set=roles[1].name=KubeEngineer --set=roles[1].groups[0]=engineers --set=account_id=REDACTED

certain subcommands don't work

Might suggest removing them as they do not work today.

reckoner generate Stacktrace -- see below:

reckoner generate
2019-01-04 14:23:03 xps13-9370 root[16862] INFO Generating example course as course.yml
Traceback (most recent call last):
  File "/home/user/.pentagon/venv/bin/reckoner", line 11, in <module>
    sys.exit(cli())
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/reckoner/cli.py", line 62, in generate
    src = pkg_resources.resource_string("reckoner", "example-course.yml")
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1152, in resource_string
    self, resource_name
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1396, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/home/user/.pentagon/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1519, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: '/home/user/.pentagon/venv/local/lib/python2.7/site-packages/reckoner/example-course.yml'

reckoner version

Traceback (most recent call last):
  File "/home/nhuanca/.pentagon/venv/bin/reckoner", line 11, in <module>
    sys.exit(cli())
  File "/path/to/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/path/to/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/path/to/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/path/to/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/path/to/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/path/to/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/path/to/reckoner/cli.py", line 72, in version
    print(reckoner.__version__)
NameError: global name 'reckoner' is not defined

Reckoner import feature

It would be awesome if reckoner had a command to import a repository chart and build the correct yaml format for me.

Example

# all params used
reckoner import foo/datadog --version v0.0.1 --repository foo/https://your.repo/charts --name my-datadog <course.yml>

# defaults to whatever repo is in course.yml for "stable" and loads the latest version as --version
reckoner import stable/cert-manager

# expected output
  my-datadog:
    chart: datadog
    repository: stable
    version: v0.0.1

Features

• Validate the repo, chart and version exist
• Leverage the course.yml

MVP

• Instead of modifying your course.yml maybe stdout the yaml sections
• If we could save comments and alignment then we could just write to the course.yml directly

Stretch

• Leverage defaults for desired default values settings reckoner import stable/cert-manager --default-values=https://my.url.com/charts/_my_defaults.yml

Feature Request: Tab completion

Carried over from: FairwindsOps/autohelm#38

Detect failed deployments and notify or force

There is a known helm issue: helm/helm#3208 where as sometimes a failed release causes the install to fail. An easy work around exist with helm delete --purge <FAILED RELEASE then helm upgrade --install or just helm upgrade --install --force. Reckoner should detect this issue and either notify the user or force the install.

Publish PyInstaller Binaries

Augment the pipeline to push binaries to the github release page. Include both Darwin (OS X) and Linux binaries.

FEATURE - Display a message when no chart matches --heading

When specifying a --heading, it would be nice if Reckoner told you that there was no release matching that name.

Minimum version is still referred to as autohelm and appears to be broken

2018-12-28 09:21:47 root[26618]    DEBUG Autohelm Minimum Version is 0.0.0
2018-12-28 09:21:47 root[26618]    DEBUG Autohelm Installed Version is 0.8.3

And the course.yml specifies a later version

Unable to install charts in a subdirectory

I have tried to use reckoner to install the RO autospotting chart and I encounter this error:

2018-12-19 10:05:12 Kims-MacBook-Pro.local root[7282] INFO Installing autospotting
2018-12-19 10:05:18 Kims-MacBook-Pro.local root[7282] ERROR Error with subprocess call: helm upgrade --install autospotting /Users/kimschles/workspace/projects/redcanary/redcanary-infrastructure/helm/cache/archive/git@github_com:reactiveops_autospotting-ci_git/ --namespace=kube-system --version=0.1.3 --set=image=quay.io/reactiveops/autospotting:0.1.3 --set=environment.REGIONS=us-east-2 --set=environment.ALLOWED_INSTANCE_TYPES=m5.*
2018-12-19 10:05:18 Kims-MacBook-Pro.local root[7282] ERROR Error: no Chart.yaml exists in directory "/Users/kimschles/workspace/projects/redcanary/redcanary-infrastructure/helm/cache/archive/git@github_com:reactiveops_autospotting-ci_git"

2018-12-19 10:05:18 Kims-MacBook-Pro.local root[7282] ERROR Error: no Chart.yaml exists in directory "/Users/kimschles/workspace/projects/redcanary/redcanary-infrastructure/helm/cache/archive/git@github_com:reactiveops_autospotting-ci_git"

2018-12-19 10:05:18 Kims-MacBook-Pro.local root[7282] ERROR Helm upgrade failed. Rolling back autospotting
2018-12-19 10:05:18 Kims-MacBook-Pro.local root[7282] ERROR ERROR: Some charts failed to install and were rolled back
2018-12-19 10:05:18 Kims-MacBook-Pro.local root[7282] ERROR  - autospotting 

When I cd $HELM_HOME I can see the different helm charts. The charts maintained by other folks are tar files, and our charts are directories named their git remote. The autospotting directory has a Dockerfile, README.md and quay.config. There's also an autospotting directory that is empty, and that's where the I'm expecting the Chart.yaml but it's not there.

fix releases CircleCI

The release script isn't working as expected 🤕 - need to adjust so there's less manual steps for a release.

--dry-run flag does nothing

Used to work.

Reckoner Errors are Incorrect

Carried over from: FairwindsOps/autohelm#39

Local file charts should not try to `helm repo add`

If a path is specified as the repository, we see this warning which is confusing:

✔ ± reckoner plot course.yml --heading ip-masq-agent --dry-run
2019-01-25 09:36:19 root[36831]     INFO Installing ip-masq-agent
2019-01-25 09:36:20 root[36831]  WARNING Unable to install repository ../../../../../charts: Command Failed with output below:
STDOUT:
STDERR: Error: Could not find protocol handler for:

COMMAND: helm repo add ../../../../../charts None
2019-01-25 09:36:21 root[36831]     INFO [debug] Created tunnel using local port: '51041'

[debug] SERVER: "127.0.0.1:51041"

I believe we shouldn't be trying to do a helm repo add if the "repository" is a local file.

[FEATURE REQUEST] - Chart Dependencies

Carried over from: FairwindsOps/autohelm#47

Reckoner should diff before applying charts with no changes

Carried over from: FairwindsOps/autohelm#50

Feature request "--strict" mode

I notice that Reckoner when it fails to install/upgrade a chart, just continues on. I'd really like a strict mode that bails out the whole run and returns an exit code 1 (or something non-zero) if any chart fails. My use case here is a CI/CD flow, since my jobs all pass even if some of the charts fail (which means errors aren't surfaced).

Thanks!

A Failing Pre-Install hooks should block chart install

Currently the behavior of reckoner is to continue on if any of the pre or post installation hooks fail. This can be a confusing user experience, especially if you've fired 20 chart installs and the first one had a failure. This signal can be lost in the noise of output from Reckoner.

Feature Request

• Could Reckoner, by default, stop after the first failed pre-install hook and notify the customer.
• Could we add a quick summary of failed chart installations at the end of the run (failed chart install N shouldn't block chart N+1)
• Could we add an override to ignore all failed pre-install-hooks
• Failed post-install hooks should also be added to the summary list of failed chart installs as "Failed Post-Install"
• All hooks should stop running after the first one fails (example below)

charts:
  my-chart:
    set-values:
      enabled: true
    hooks:
      pre_install:
        - echo true
        - curl non*possible().domain
        - echo true   #<--- this should never fire if the curl above fails & chart will also not be installed
      post_install:
        - echo true
        - cat fileno-existy
        - echo this-never-runs-in-post

Multiline output of hooks needs better formatting

Currently, multiline output on a hook looks like:

2019-02-19 14:27:33 root[54531]     INFO Returned stdout: customresourcedefinition.apiextensions.k8s.io/certificates.certmanager.k8s.io unchanged
customresourcedefinition.apiextensions.k8s.io/issuers.certmanager.k8s.io unchanged
customresourcedefinition.apiextensions.k8s.io/clusterissuers.certmanager.k8s.io unchanged
customresourcedefinition.apiextensions.k8s.io/orders.certmanager.k8s.io unchanged
customresourcedefinition.apiextensions.k8s.io/challenges.certmanager.k8s.io unchanged

This would be much easier to read if it was indented. Mostly just a nice-to-have.

Detect TILLER_NAMESPACE and Helm version in cluster

Detecting server-side cluster details would be a nice addition. Warnings for helm versions and automatically setting tiller-namespace

Support for float values

Floating point values get cast into strings. This breaks some helm charts that expect them to be floats.

Example reckoner settings:

kubespawner_override:
   cpu_limit: 0.5
   mem_limit: 512

Error from app on the cpu_limit type:
Error: the 'cpu_limit' trait of KubeSpawner instance must be a float, but a value of '0.5' <class 'str'> was specified

Something broken in external repo handling

when adding an external chart repo, install fails on downloading the chart. This works outside of reckoner with these commands:

helm repo add influx http://influx-charts.storage.googleapis.com
helm install influx/telegraf-ds

Here's the debug output:

[ do-sfo2-sudermanjr   ] [pw]
 130 ✘ reckoner --log-level=DEBUG plot course.yaml --heading data
2019-01-08 08:40:53 root[36803]    DEBUG helm help
2019-01-08 08:40:53 root[36803]    DEBUG helm version --short --server
2019-01-08 08:40:54 root[36803]    DEBUG Installing repository: {'url': 'https://kubernetes-charts-incubator.storage.googleapis.com', 'name': 'incubator'}
2019-01-08 08:40:54 root[36803]    DEBUG helm repo list
2019-01-08 08:40:54 root[36803]    DEBUG Chart repository incubator already installed
2019-01-08 08:40:54 root[36803]    DEBUG Installing repository: {'url': 'https://kubernetes-charts.storage.googleapis.com', 'name': 'stable'}
2019-01-08 08:40:54 root[36803]    DEBUG helm repo list
2019-01-08 08:40:54 root[36803]    DEBUG Chart repository stable already installed
2019-01-08 08:40:54 root[36803]    DEBUG Installing repository: {'url': 'http://influx-charts.storage.googleapis.com', 'name': 'influx'}
2019-01-08 08:40:54 root[36803]    DEBUG helm repo list
2019-01-08 08:40:54 root[36803]    DEBUG Chart repository influx already installed
2019-01-08 08:40:54 root[36803]    DEBUG helm repo update
2019-01-08 08:40:56 root[36803]    DEBUG Helm Minimum Version is: 2.12.1
2019-01-08 08:40:56 root[36803]    DEBUG helm version --short --client
2019-01-08 08:40:56 root[36803]    DEBUG Helm Installed Version is 2.12.1
2019-01-08 08:40:56 root[36803]    DEBUG Reckoner Minimum Version is 0.10.1
2019-01-08 08:40:56 root[36803]    DEBUG Reckoner Installed Version is 0.10.3
2019-01-08 08:40:56 root[36803]    DEBUG helm version --short --client
2019-01-08 08:40:56 root[36803]     INFO Installing data
2019-01-08 08:40:56 root[36803]    DEBUG helm repo list
2019-01-08 08:40:56 root[36803]    DEBUG Chart repository influx already installed
2019-01-08 08:40:56 root[36803]    DEBUG Updating chart dependencies: influx/influxdb
2019-01-08 08:40:56 root[36803]    DEBUG helm upgrade --install data influx/influxdb --namespace=tick --version=1.7.1
2019-01-08 08:40:57 root[36803]    ERROR Command Failed with output below:
STDOUT:
STDERR: Error: failed to download "influx/influxdb" (hint: running `helm repo update` may help)

COMMAND: helm upgrade --install data influx/influxdb --namespace=tick --version=1.7.1
2019-01-08 08:40:57 root[36803]    ERROR Helm upgrade failed. Rolling back data
2019-01-08 08:40:57 root[36803]    DEBUG Traceback (most recent call last):
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/course.py", line 122, in plot
    chart.install(namespace=self.namespace, context=self.context)
  File "/Users/asuderma/repos/reactiveops/reckoner/reckoner/chart.py", line 232, in install
    raise e
HelmClientException: Command Failed with output below:
STDOUT:
STDERR: Error: failed to download "influx/influxdb" (hint: running `helm repo update` may help)

COMMAND: helm upgrade --install data influx/influxdb --namespace=tick --version=1.7.1

2019-01-08 08:40:57 root[36803]    ERROR ERROR: Some charts failed to install and were rolled back
2019-01-08 08:40:57 root[36803]    ERROR  - data

Snippet of course.yml

namespace: infra
repository: stable
minimum_versions:
  helm: 2.12.1
  reckoner: 0.10.1
repositories:
  incubator:
    url: https://kubernetes-charts-incubator.storage.googleapis.com
  stable:
    url: https://kubernetes-charts.storage.googleapis.com
  influx:
    url: http://influx-charts.storage.googleapis.com
charts:
  data:
    namespace: tick
    chart: influxdb
    version: 1.7.1
    repository: influx

Reckoner v0.10.2
Helm v2.12.1

End to End Test Course Yamls

It would be great to have some tests for specific features in course.yml or at least basic and complex course.yml files for running blackbox end-to-end testing.

These would also be useful for testing the CLI contracts of providing files and command line arguments.

Vision:

• test_basic_course.yml (installs one chart with only variables and no helm default args)
• test_advanced_course.yml (installs two+ charts with values, yaml files, string-values, etc)

Deliverable

• Just the course yaml files that can be run locally for manual testing (figure out automation later)

Feature: Reckoner should check for the namespace before pre_install hooks

Generally speaking, if you've got pre_install hooks on a chart definition, you're probably adding or changing something within the configured namespace of the chart. Since the helm command isn't run yet at the time of firing the webhook, we can assume that the namespace may not be there.

Potential directions to go:

• If namespace doesn't exist, add a flag to be able to autocreate it before pre-install hooks
• If a namespace doesn't exist, fail and don't run the pre-install hooks (skip the chart install)
• If the namespace doesn't exist, just autocreate it (could have un-desireable results or unclear changes)

--local-development flag does not work

The --local-development flag suggests that it does not need Tiller access to work, however when using it with --dry-run I see this error:

2019-05-13 14:41:27 root[16325]    ERROR Command Failed with output below:
STDOUT:
STDERR: Error: Get http://localhost:8080/api/v1/namespaces/kube-system/pods?labelSelector=app%3Dhelm%2Cname%3Dtiller: dial tcp [::1]:8080: connect: connection refused

COMMAND: helm upgrade --install metrics-server stable/metrics-server --namespace=metrics-server --dry-run --debug --version=2.4.0 --set=args[0]=--kubelet-insecure-tls --set=args[1]=--kubelet-preferred-address-types=InternalIP --set=args[2]=--metric-resolution=30s --set=args[3]=--v=3

Documentation around chart location in git repos is misleading

https://github.com/reactiveops/reckoner/blob/master/README.md#path
isn't specific enough and is misleading where it describes the 'path' and 'chart path'

Maybe add a small example like this one since the concept doesn't it well in words:
https://github.com/reactiveops/autohelm/blob/master/examples.md#rbac-manager

Schema validation

Reckoner should always evaluate the course.yaml with a true schema validation before processing any actions.

• Would lead to less half baked runs
• Have less burden on the user to constantly try syntax
• Better user experience for unused blocks of yaml
• Hopefully could provide a reference for users wanting to know the full breadth of options in a course.yaml

Install docs don't mention the binary release installation

The current readme only mentions the pip install versions instead of the, now recommended I think, Binary release.

Truthy/Falsey values don't do what you want them to

Carried over from: FairwindsOps/autohelm#7

Version 0.10.1 has re-broken `values-strings`

course.yml snippet

charts:
  cert-manager:
    version: v0.5.2
    hooks:
      post_install:
        - kubectl apply -f manifests/reckoner/letsencrypt-staging.json
        - kubectl apply -f manifests/reckoner/letsencrypt-prod.json
    values:
      ingressShim:
        defaultIssuerName: letsencrypt-prod
        defaultIssuerKind: ClusterIssuer
        resources:
          requests.cpu: 10m
          requests.memory: 32Mi
          limits.cpu: 15m
          limits.memory: 64Mi
      resources:
        requests.cpu: 10m
        requests.memory: 50Mi
        limits.cpu: 15m
        limits.memory: 100Mi
    values-strings:
      extraArgs:
        - '--dns01-self-check-nameservers=8.8.8.8\:53'

Helm Command output:

2019-01-07 15:51:32 root[20781]    DEBUG helm upgrade --install cert-manager stable/cert-manager --namespace=infra --dry-run --debug --version=v0.5.2 --set=ingressShim.defaultIssuerName=letsencrypt-prod -
-set=ingressShim.defaultIssuerKind=ClusterIssuer --set=ingressShim.resources.requests.cpu=10m --set=ingressShim.resources.requests.memory=32Mi --set=ingressShim.resources.limits.cpu=15m --set=ingressShim.
resources.limits.memory=64Mi --set=resources.requests.cpu=10m --set=resources.requests.memory=50Mi --set=resources.limits.cpu=15m --set=resources.limits.memory=100Mi --set=extraArgs[0]=--dns01-self-check-
nameservers=8.8.8.8\:53

I would expect the last arg in the command to have --set-string instead of --set

Not parsing Environment variables properly

When I run reckoner on a course.yml that containers environment variables, it just passes the literal string rather than substituting.

Clean Up readme and Usage Documentation

The documentation is missing reckoner flags and functionality that would be useful to outline in the Getting Started section.

Things to document (not exhaustive):

1. env var interpolation in settings
2. Nice quality "Getting Started" section with a few examples and workflows
3. Working with different environments / clusters
4. Setting Kubecontext for apply safety
5. Using --only flag for plots
6. Behavioral documentation (when a chart or pre_install or post_install hook fails, what happens)
7. Can Reckoner be used in CI/CD capacity? (what's the current user story there)

The road to python 3

Start migration to python three and version bump to v2.0.0

Documentation Needed

• Docs on releasing with new dynamic version process
• Docs on how to setup development environment with new requirements.txt
• Docs on differences between development-requirements.txt and the setup.py requirements

Install horribly broken in latest release

The refactor left the install capability non-existent. Critical bug!