GithubHelp home page GithubHelp logo

runatlantis / atlantis Goto Github PK

View Code? Open in Web Editor NEW
7.5K 79.0 1.0K 54.92 MB

Terraform Pull Request Automation

Home Page: https://www.runatlantis.io

License: Other

Makefile 0.17% Go 97.31% Shell 0.36% CSS 0.57% JavaScript 0.25% HCL 0.36% Dockerfile 0.41% TypeScript 0.49% Vue 0.07%
terraform devops atlantis golang go sre hacktoberfest automation tacos

atlantis's Introduction

Atlantis

Latest Release SuperDopeBadge Go Report Card Go Reference CircleCI Slack

Atlantis Logo

Terraform Pull Request Automation

Resources

What is Atlantis?

A self-hosted golang application that listens for Terraform pull request events via webhooks.

What does it do?

Runs terraform plan, import, apply remotely and comments back on the pull request with the output.

Why should you use it?

  • Make Terraform changes visible to your whole team.
  • Enable non-operations engineers to collaborate on Terraform.
  • Standardize your Terraform workflows.

Stargazers over time

Stargazers over time

atlantis's People

Contributors

aayyush avatar anubhavmishra avatar chenrui333 avatar cket avatar dependabot[bot] avatar fauzyy avatar finnag avatar genpage avatar gezb avatar inkel avatar jamengual avatar jippi avatar johananl avatar jpreese avatar krrrr38 avatar lilincmu avatar lkysow avatar lukemassa avatar mcdafydd avatar msarvar avatar nickwu241 avatar nishkrishnan avatar nitrocode avatar pseudomorph avatar renovate[bot] avatar shouichi avatar tenzer avatar unrob avatar veverkap avatar x-guardian avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

atlantis's Issues

Options not being parsed properly

I'm using docker atlantisnorth/atlantis:latest and I'm trying to run a plan in a specific directory and it's not working. When I pass the -d option it passes it to terraform and I get this error from terraform:

flag provided but not defined: -d

The command I use is atlantis plan -d . -w <my_env>

Really though this is kinda secondary to my root issue, I have other directories holding spec tests and templates and I'm trying to just force the plan to run in the root directory instead of walking the tree.

Maybe this could just be a feature request? Maybe make an option in the atlantis.yaml to ignore specific directories? It looks like it already ignores a few directories by default, maybe make that configurable?

Add newlines to flags to make easier to read

When spf13/pflag#155 is merged we can just add a newline to the end of our descriptions.

Will change

Usage:
  atlantis server [flags]

Flags:
      --allow-fork-prs                 > Allow Atlantis to run on pull requests from forks. A security issue for
                                       public repos.
      --atlantis-url string            > URL that Atlantis can be reached at. Defaults to http://$(hostname):$port
                                       where $port is from --port.
      --config string                  > Path to config file.
      --data-dir string                > Path to directory to store Atlantis data. (default "~/.atlantis")
      --gh-hostname string             > Hostname of your Github Enterprise installation. If using github.com, no
                                       need to set. (default "github.com")
      --gh-token string                > GitHub token of API user. Can also be specified via the ATLANTIS_GH_TOKEN
                                       environment variable.
      --gh-user string                 > GitHub username of API user.
      --gh-webhook-secret string       > Secret used to validate GitHub webhooks (see
                                       https://developer.github.com/webhooks/securing/). SECURITY WARNING: If not
                                       specified, Atlantis won't be able to validate that the incoming webhook
                                       call came from GitHub. This means that an attacker could spoof calls to
                                       Atlantis and cause it to perform malicious actions. Should be specified via
                                       the ATLANTIS_GH_WEBHOOK_SECRET environment variable.
      --gitlab-hostname string         > Hostname of your GitLab Enterprise installation. If using gitlab.com, no
                                       need to set. (default "gitlab.com")
      --gitlab-token string            > GitLab token of API user. Can also be specified via the
                                       ATLANTIS_GITLAB_TOKEN environment variable.
      --gitlab-user string             > GitLab username of API user.
      --gitlab-webhook-secret string   > Optional secret used to validate GitLab webhooks. SECURITY WARNING: If
                                       not specified, Atlantis won't be able to validate that the incoming webhook
                                       call came from GitLab. This means that an attacker could spoof calls to
                                       Atlantis and cause it to perform malicious actions. Should be specified via
                                       the ATLANTIS_GITLAB_WEBHOOK_SECRET environment variable.
  -h, --help                           help for server
      --log-level string               > Log level. Either debug, info, warn, or error. (default "info")
      --port int                       > Port to bind to. (default 4141)
      --repo-whitelist string          > Comma separated list of repositories that Atlantis will operate on, ex.
                                       'github.com/runatlantis/atlantis,github.mycompany.com/*'. The format is
                                       {hostname}/{owner}/{repo}. '*' denotes any string until the next comma and
                                       can be used to whitelist all repos (not recommended) or an entire hostname
                                       or organization.
      --require-approval               > Require pull requests to be "Approved" before allowing the apply command
                                       to be run.
      --ssl-cert-file string           > File containing x509 Certificate used for serving HTTPS. If the cert is
                                       signed by a CA, the file should be the concatenation of the server's
                                       certificate, any intermediates, and the CA's certificate.
      --ssl-key-file string            > File containing x509 private key matching --ssl-cert-file.

To look like

Start the atlantis server

Flags can also be set in a yaml config file (see --config).
Config file values are overridden by environment variables which in turn are overridden by flags.

Usage:
  atlantis server [flags]

Flags:
      --allow-fork-prs                 Allow Atlantis to run on pull requests from forks. A security issue for
                                       public repos.

      --atlantis-url string            URL that Atlantis can be reached at. Defaults to http://$(hostname):$port
                                       where $port is from --port.

      --config string                  Path to config file.

      --data-dir string                Path to directory to store Atlantis data. (default "~/.atlantis")

      --gh-hostname string             Hostname of your Github Enterprise installation. If using github.com, no
                                       need to set. (default "github.com")

      --gh-token string                GitHub token of API user. Can also be specified via the ATLANTIS_GH_TOKEN
                                       environment variable.

      --gh-user string                 GitHub username of API user.

      --gh-webhook-secret string       Secret used to validate GitHub webhooks (see
                                       https://developer.github.com/webhooks/securing/). SECURITY WARNING: If not
                                       specified, Atlantis won't be able to validate that the incoming webhook
                                       call came from GitHub. This means that an attacker could spoof calls to
                                       Atlantis and cause it to perform malicious actions. Should be specified via
                                       the ATLANTIS_GH_WEBHOOK_SECRET environment variable.

      --gitlab-hostname string         Hostname of your GitLab Enterprise installation. If using gitlab.com, no
                                       need to set. (default "gitlab.com")

      --gitlab-token string            GitLab token of API user. Can also be specified via the
                                       ATLANTIS_GITLAB_TOKEN environment variable.

      --gitlab-user string             GitLab username of API user.

      --gitlab-webhook-secret string   Optional secret used to validate GitLab webhooks. SECURITY WARNING: If not
                                       specified, Atlantis won't be able to validate that the incoming webhook
                                       call came from GitLab. This means that an attacker could spoof calls to
                                       Atlantis and cause it to perform malicious actions. Should be specified via
                                       the ATLANTIS_GITLAB_WEBHOOK_SECRET environment variable.

  -h, --help                           help for server

      --log-level string               Log level. Either debug, info, warn, or error. (default "info")

      --port int                       Port to bind to. (default 4141)

      --repo-whitelist string          Comma separated list of repositories that Atlantis will operate on, ex.
                                       'github.com/runatlantis/atlantis,github.mycompany.com/*'. The format is
                                       {hostname}/{owner}/{repo}. '*' denotes any string until the next comma and
                                       can be used to whitelist all repos (not recommended) or an entire hostname
                                       or organization.

      --require-approval               Require pull requests to be "Approved" before allowing the apply command to
                                       be run.

      --ssl-cert-file string           File containing x509 Certificate used for serving HTTPS. If the cert is
                                       signed by a CA, the file should be the concatenation of the server's
                                       certificate, any intermediates, and the CA's certificate.

      --ssl-key-file string            File containing x509 private key matching --ssl-cert-file.

Forked repo

Hello,

I don't suppose you could remove this from being a fork of the original?

Minor issue but on a fork it seems you can't so searches on github, so when I was looking for how the Slack integration worked, you are unable to do a code search, which means I need to clone the repo and do local searches.

Not a massive problem but worth considering.

Repo whitelist is case sensitive

Version: 0.3.5

Repo whitelist should be case insensitive.

When I configure the repo whitelist as github.com/247sports/* I get the following log output:

2018/03/29 23:05:31 events_controller.go:136: [DEBUG] server: Ignoring pull request event from non-whitelisted repo

Once I change it to github.com/247Sports/* it works fine.

long plan output does not get sent to github comments

I have a repository where there are several changes, when I run the atlantis plan step the plan passes but no plan output is commented on the pull request. I believe this may be is due to the length of the output.

If I run atlantis plan with -target=some.resource the plan passes and output is commented as expected. The full plan output is never posted. tfplan files are created as expected.

I am running Atlantis in a container FROM runatlantis/atlantis. I started atlantis server with --log-level debug but it does not show any comment posts from atlantis to github.

atlantis apply -w *

Enable users to apply all outstanding plans with a command something like atlantis apply -w * -d *. Exact command to be determined.

When auto plan is implemented, the need for an atlantis plan -w * would be less likely. This is also harder to implement since we need to figure out what your workspaces and dirs are.

Non-filesystem backends

I'm looking at deploying Atlantis to Heroku, since the free tier seems like a good fit for running this service on a project that only sees occasional updates. However, Heroku only provides an ephemeral filesystem, which disappears any time the service shuts down - at least once every 24 hours, and on the free tier, also after 15 minutes without a request. Since Atlantis writes its locks to disk, this means that locks are volatile and disappear quickly.

I'd love to store locks in an external service, such as PostgreSQL or etcd, if possible, so that filesystem ephemerality isn't a blocker.

I had a look at the locking package, and the lock API looks general enough to implement on other systems - treat this issue as a combined question ("is this important to anyone else?") and suggestion ("wouldn't it be nice if").

Make locking behaviour configurable

Currently Atlantis locks a specific repo/directory/workspace combination. So if you run plan in subdir/ for workspace default in runatlantis/atlantis, another pull request will be able to run plan in subdir/ for workspace new in runatlantis/atlantis. This needs to be configurable.

From @lorenzoaiello they would like to be able to lock all directories and workspaces when there's a plan. This is because they have small repos and so if there's a change going on in any of the directories they would like the whole repo locked.

From @grobie, his team would like to completely disable locking because they use a mono-repo and so it would be too disruptive to lock things. He'd like plan to always be available.

Support top level `modules` folder with directory-based config

Given the following directory structure:

modules/
  module1/
    main.tf
vpc/
  main.tf

If a change is made to both module1/main.tf and vpc/main.tf, Atlantis will attempt to run terraform plan in vpc/ (which is correct) and in the root of the project (which is incorrect).

This occurs due to an attempt to support this type of structure:

main.tf
modules/
  module1/
    main.tf

The way Atlantis works right now is we assume if there's a change in any directory called modules we should run plan/apply in the root.

Possible solutions:

  • requiring an atlantis.yaml file in all project roots. We don't run anywhere that doesn't have one
  • detecting if there are no .tf files in the parent of modules/ and not running there if that's the case

Comment back with progress on long apply's

If performing a large change, terraform can sometimes take a long time until it returns. During this time, the only indication that Atlantis is working is the build status icon.

It would be nice if an apply was taking over 30 seconds for Atlantis to comment back with some sort of indication that the operation is under progress.

Speed up by caching plugins and providers

Issue by @lkysow
Monday Sep 04, 2017 at 01:47 GMT
Migrated from hootsuite/atlantis#141
Why was it migrated?


With the latest terraform >= 0.9 providers come as plugins that need to be downloaded. Right now, we're using terraform init which downloads the plugins into the current directory under .terraform/plugins/{os}_{runtime} for every project.

If we move the downloaded plugin to ~/.terraform.d/plugins/ (or ~/.terraform.d/plugins/{os}_{runtime} then terraform won't download the plugin again for any project.

So my proposed solution is:

  • on every terraform init, check if there are any files in {pwd}/.terraform/plugins/os_runtime other than lock.json. If so, move all of them to ~/.terraform.d/plugins/.

Now tf will check ~/.terraform.d/plugins/ before downloading any plugins so it will be back up to speed. Since tf will download any plugins that aren't already existing, we can rely on it to do the downloading.

0.3.3 - Slower Parallelism

Upgraded from 0.2.4 to 0.3.3 last night and it appears as though there is a significant downtick in the number of parallel processing it can handle. The release notes don't seem to show anything specific, has someone noticed any performance impacts post-upgrade?

Update module command

Issue by @johntdyer
Thursday Nov 09, 2017 at 21:57 GMT
Migrated from hootsuite/atlantis#187
Why was it migrated?


Hey guys,

So atlantis plan appears to be doing a plan w/ an old version of one of our modules. I am wondering if there is any way to force atlantis to run terraform get -update=true before the plan ? Perhaps i am missing something fundamental ?

New repo whitelist functionality

I have a question regarding the new repo whitelist functionality.

README.md states:

--repo-whitelist

If someone stole your webhook secret or you don't have any set, they could make Atlantis perform actions on their repository. To mitigate this, you can run Atlantis with --repo-whitelist and whitelist which repositories Atlantis acts on. See atlantis server --help for more details.

But the release notes states:

Must set --repo-whitelist in order to start atlantis server. See atlantis server --help for how that flag works.

Combined, I am unclear as to whether this new flag is REQUIRED or OPTIONAL. I would assume that it is optional (our use case we would never want to explicitly whitelist), but want to confirm.

Add new "mergeable" apply requirement

Issue by @lkysow
Thursday Nov 30, 2017 at 06:54 GMT
Migrated from hootsuite/atlantis#210
Why was it migrated?


image

GitHub has lots of branch protections that we could support in Atlantis by requiring them to "pass" before we allow apply's. Right now you can specify --require-approval but this only looks for an approval, not necessarily the type of approval specified in the branch protections.

  • The API for branch protections is here: https://developer.github.com/v3/repos/branches/
  • We'd have to decide which ones to support (might be all of them)
  • The benefit here would be that there's more configuration available around who can apply

No changes still blocks merge

Atlantis Version: 0.3.3

We have an interesting edge case we've run into that I think should be considered for a functional change.

Currently, when you create a pull request and perform an atlantis plan which results in No changes. Infrastructure is up-to-date. which is expected.

BUT, in our case, because we have Atlantis required for merges, which results in the following stalemate:

screen shot 2018-03-20 at 8 10 48 am

My expected behavior is that if Terraform returns with No Changes, atlantis should report green and "ready to merge", because... well, there are no net changes. We discovered the edge case when refactoring some code that had plenty of code changes, but no output changes.

proxy support

Issue by @automaticgiant
Tuesday Dec 05, 2017 at 01:49 GMT
Migrated from hootsuite/atlantis#218
Why was it migrated?


i need terraform to go through the proxy to aws, but the git stuff to not go through a proxy.
presently, looks like the env won't get passed through to tf anyway, which is a problem.

however i do see also the problem that i can't set no_proxy to exclude enterprise github from atlantis trying to proxy that.

[feature] Redact / Mask atlantis output

Issue by @so0k
Monday Jan 29, 2018 at 06:59 GMT
Migrated from hootsuite/atlantis#235
Why was it migrated?


Is there any way to redact Atlantis output into the PR comments?

In our company there is a concern with secrets in PR comments and we would need finer control over the comments created by Atlantis.

for Terraform, opencredo/terrahelp provides masking functionality by piping Terraform output through it and masking any variables that are in the tfvars file.

At this stage I see 2 options:

  1. Provide a way to pipe terraform output through other binaries before sending it to the github comment renderer (this would be the most flexible way)
  2. Make the comment templates configurable and including a functionmap that can help masking (most likely too complicated as the masking functions would need to be compiled within Atlantis)

or maybe there is a workaround I'm not seeing?

Comment back if user types incorrect command

If a user types terraform plan or atlantis badcommand it would be nice to comment back on the pull request with an error. Otherwise the user might wait until they realize it was wrong.

I think we should comment back on:

  • terraform\s.*
  • atlantis\s.*

Force Plan/Apply

It would be great if we had the ability to "force" a plan and/or apply to bypass the following error:

"Plan Failed: No Terraform files were modified."

The use case in our case, is we store userdata files, and policy statements as flat files using the terraform file function, so despite there being changes in those files, Atlantis won't re-run without adding a whitespace or some other irrelevant change to the TF stack.

Run apply only for a specific project in a multiproject repo

Hi!

I have been playing with a multi-project repository (dev, prod), and I cannot find a way to execute plan or apply only for a specific project.

When I run atlantis plan I get the following message:

To perform exactly these actions, run the following command to apply:
    terraform apply "/tmp/atlantis/data/repos/psalaberria002/atlantis-example/2/default/prod/default.tfplan"

I would like to do atlantis apply "/tmp/atlantis/data/repos/psalaberria002/atlantis-example/2/default/prod/default.tfplan".

Is there a way to do this currently in Atlantis? Maybe there is an existing flag I have missed.

Thank you.

server-side atlantis.yaml

Issue by @so0k
Monday Jan 29, 2018 at 08:24 GMT
Migrated from hootsuite/atlantis#236
Why was it migrated?


Atm, any developer can add pre_get, pre_init, pre/post_plan commands to expose secrets via atlantis.yaml - as Atlantis requires quite significant permissions, this might be a concern.

Drone used to have a signing step required for the drone.yaml file, ensuring unauthorized modifications to atlantis.yaml can be prevented

Atlantis UI protected by GitHub OAuth

Issue by @kiranpraneeth
Tuesday Feb 06, 2018 at 22:48 GMT
Migrated from hootsuite/atlantis#243
Why was it migrated?


It would be great if we can have Oauth callback ability in Atlantis. That would avoid us from being relying on our own HTTP auth methods and rather just use GitHub OAuth to authenticate users to Atlantis. I looked at closed and open issues but didn't find any issues in regards to OAuth. Can you throw some light on this?

Thanks

Documentation clarification around multiple projects.

Greetings,

I have started diving into Terraform and Atlantis. One issue that isn't very clear is how Atlantis supports the multiple "projects" repository layout and separate state files for each.

I want this layout with each "project" having its own state file.

.
├── staging
│   ├── main.tf
|   └── ...
└── production
│   ├── main.tf
|   └── ...
└── modules
    ├── module_1
    └── ...

I can't tell from the documentation how Atlantis will handle it if at all. If it does, how does this work with a backend for each to separate the state files?

Error: getting tunnel url: didn't find tunnels that were expected to be created

$ atlantis version
atlantis 0.3.5

$ atlantis bootstrap
Welcome to Atlantis bootstrap!

This mode walks you through setting up and using Atlantis. We will

  • fork an example terraform project to your username
  • install terraform (if not already in your PATH)
  • install ngrok so we can expose Atlantis to GitHub
  • start Atlantis

Press Ctrl-c at any time to exit

GitHub username: NeckBeardPrince

To continue, we need you to create a GitHub personal access token
with "repo" scope so we can fork an example terraform project.

Follow these instructions to create a token (we don't store any tokens):
https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/#creating-a-token

  • use "atlantis" for the token description
  • add "repo" scope
  • copy the access token

GitHub access token (will be hidden):
=> forking repo
=> fork completed!
=> terraform found in $PATH!
=> downloading ngrok
=> downloaded ngrok successfully!
=> creating secure tunnel
=> started tunnel!
Error: getting tunnel url: didn't find tunnels that were expected to be created

Usage:
atlantis bootstrap [flags]

Flags:
-h, --help help for bootstrap

Detect Terraform version from `terraform.required_version` block

You can set a required terraform version: https://www.terraform.io/docs/configuration/terraform.html

terraform {
  required_version = "> 0.7.0"
}

Atlantis should support detecting this and using this version. This would mean users don't have to also update the terraform_version in their atlantis.yaml files.

The logic should be:

  • if terraform_version is set in atlantis.yaml, use that
  • if it's not set, use the same mechanism as Terraform to check for a required_version
    • if there is a required version, use the latest version that a) satisfies the required_version and b) is installed
    • if there is not a required version use whatever version is in the terraform binary

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.