geerlingguy / ansible-role-ecr_container_build

Ansible Role - ECR Container Build

Ansible Role: ECR Container Build

An Ansible Role that installs builds Docker container images and (optionally) pushes them to AWS ECR Repositories.

Requirements

• Docker
• Pip packages: boto3, docker

Role Variables

Available variables are listed below, along with default values (see defaults/main.yml):

ecr_image_src_dir: ../my-project
ecr_image_name: namespace/my-project

A source directory containing a Dockerfile and any required resources, and the image name (typically in the form namespace/project) for the docker image that is built.

ecr_image_buildargs: {}

Build args to pass to the docker_image module when building the Docker image. Args should be passed as an object with key-value pairs, e.g. { name: value, name2: value2 }

ecr_image_tags: ['latest']

The tags to apply to the final image which is pushed to ECR.

ecr_login_required: false

Set this to true if you are using ECR as the source for your container build (e.g. FROM in Dockerfile).

ecr_push: true

Whether to push the built image to ECR. Set to false if you're just testing the image build portion or you cannot connect to ECR.

ecr_region: us-east-1
ecr_account_id: '123456789012'
ecr_url: "{{ ecr_account_id }}.dkr.ecr.{{ ecr_region }}.amazonaws.com"

AWS account details for ECR.

Dependencies

None.

Example Playbook

Building locally (assuming you already have Docker CE and the docker pip package installed):

---
- hosts: localhost
  connection: local
  gather_facts: false

  vars:
    ecr_image_src_dir: ../my-project
    ecr_image_name: namespace/my-project
    ecr_image_tags: ['latest','1.2.3']
    ecr_account_id: '123456789012'
    pip_install_packages: ['docker']

  roles:
    - role: geerlingguy.ecr_container_build

Building on a remote server:

---
- hosts: localhost
  connection: local
  gather_facts: false

  vars:
    ecr_image_src_dir: ../my-project
    ecr_image_name: namespace/my-project
    ecr_image_tags: ['latest','1.2.3']
    ecr_account_id: '123456789012'
    pip_install_packages: ['docker']

  roles:
    - role: geerlingguy.docker
    - role: geerlingguy.pip
    - role: geerlingguy.ecr_container_build

License

MIT / BSD

Author Information

This role was created in 2018 by Jeff Geerling, author of Ansible for DevOps.

ansible-role-ecr_container_build's Issues

Allow ECR login prior to image build

I have some container builds which pull FROM an ECR repo in the Dockerfile. So I need to make sure the docker login has occurred before that step.

I'd recommend splitting up the two tasks in the ecr.yml include file and adding a new var ecr_login_required to force a login even if ecr_push is false.

Pushing images to ECR sometimes times out

Strangely, one of my playbooks seems to run into this almost every other run:

TASK [geerlingguy.ecr_container_build : Push image to ECR.] ********************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Error pushing image account-id.dkr.ecr.us-east-1.amazonaws.com/repo/app: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out."}

I think we should add a retry to this task since it seems like it can be flaky.

Push to ECR fails

Pushing an image to ECR fails with the following:

TASK [geerlingguy.ecr_container_build : Push image to ECR.] ********************
FAILED - RETRYING: Push image to ECR. (10 retries left).
FAILED - RETRYING: Push image to ECR. (9 retries left).
FAILED - RETRYING: Push image to ECR. (8 retries left).
FAILED - RETRYING: Push image to ECR. (7 retries left).
FAILED - RETRYING: Push image to ECR. (6 retries left).
FAILED - RETRYING: Push image to ECR. (5 retries left).
FAILED - RETRYING: Push image to ECR. (4 retries left).
FAILED - RETRYING: Push image to ECR. (3 retries left).
FAILED - RETRYING: Push image to ECR. (2 retries left).
FAILED - RETRYING: Push image to ECR. (1 retries left).
fatal: [127.0.0.1]: FAILED! => {"attempts": 10, "changed": false, "msg": "Error pulling image - 404 Client Error: Not Found ("pull access denied for , repository does not exist or may require 'docker login'")"}

Images tagged with existing tag don't always get pushed

For example, I'm building a new container version with the tags master and latest, and here's the output:

TASK [geerlingguy.ecr_container_build : Build image.] **************************
changed: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Ensure ECR repo exists.] ***************
ok: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Tag and push the image.] ***************
included: /var/jenkins_home/workspace/microservice_deploy/src/container-build/roles/geerlingguy.ecr_container_build/tasks/tag-and-push.yml for 127.0.0.1
included: /var/jenkins_home/workspace/microservice_deploy/src/container-build/roles/geerlingguy.ecr_container_build/tasks/tag-and-push.yml for 127.0.0.1

TASK [geerlingguy.ecr_container_build : Set the current image tag.] ************
ok: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Print the current tag being pushed.] ***
ok: [127.0.0.1] => {
    "image_tag": "latest"
}

TASK [geerlingguy.ecr_container_build : Apply additional tag if there's more than one.] ***
skipping: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Push image to ECR.] ********************
ok: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Set the current image tag.] ************
ok: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Print the current tag being pushed.] ***
ok: [127.0.0.1] => {
    "image_tag": "master"
}

TASK [geerlingguy.ecr_container_build : Apply additional tag if there's more than one.] ***
ok: [127.0.0.1]

TASK [geerlingguy.ecr_container_build : Push image to ECR.] ********************
ok: [127.0.0.1]

And in ECR, I don't see the new image tags either.

Docker image isn't always pushed to ECR

It seems sometimes (like when the container tag was already present and tagged) the image is not pushed to ECR... See: ansible/ansible#44077

Allow configuration of the AWS profile used when logging into ECR

Currently the 'log into ECR if required' task doesn't allow for someone to use a non-default credentials profile to be specified:

https://github.com/geerlingguy/ansible-role-ecr_container_build/blob/master/tasks/main.yml#L2-L5

I usually have my default profile as the one I'm using, but if you have more than one AWS account you need to be able to configure more than one profile, and specify which one to use. So it would be good to have a variable like ecr_profile, default value default, and change the command for the login to:

shell: "$(aws ecr get-login --no-include-email --region {{ ecr_region }} --profile {{profile}})"

That way ~/.aws/credentials could have:

[something-else]
aws_access_key_id=todo
aws_secret_access_key=todo

[default]
aws_access_key_id=todo
aws_secret_access_key=todo
aws_session_token=todo

And then you could use this role with ecr_profile: something-else and it would use those credentials instead of default.

Output Docker Build

It would be nice to actually see what's happening just like when you run docker build. That way if something fails we are aware where it failed.
Is it possible?

Resolve Ansible 2.12 Deprecation Warnings

There are a few deprecation warnings in both the build and push tasks for this role. This issue aims to resolve both, because warnings are a scourge on my Ansible output.

During Image Build

TASK [geerlingguy.ecr_container_build : Build image.] *******************************************************************************************************************************************************************************************************
[WARNING]: Please specify build.args instead of buildargs. The buildargs option has been renamed and will be removed in Ansible 2.12.

[WARNING]: Please specify build.path instead of path. The path option has been renamed and will be removed in Ansible 2.12.

[WARNING]: The value of the "source" option was determined to be "build". Please set the "source" option explicitly. Autodetection will be removed in Ansible 2.12.

[WARNING]: The "force" option will be removed in Ansible 2.12. Please use the "force_source", "force_absent" or "force_tag" option instead, depending on what you want to force.

[DEPRECATION WARNING]: Param 'force' is deprecated. See the module docs for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: Param 'buildargs' is deprecated. See the module docs for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: Param 'path' is deprecated. See the module docs for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

During Image Push

TASK [geerlingguy.ecr_container_build : Ensure there's not already an image locally tagged with the ecr_url.] ***********************************************************************************************************************************************
[WARNING]: The "force" option will be removed in Ansible 2.12. Please use the "force_source", "force_absent" or "force_tag" option instead, depending on what you want to force.

[DEPRECATION WARNING]: Param 'force' is deprecated. See the module docs for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
changed: [localhost]

Allow passing build args when building docker image

For some of my builds, I need to use --build-arg to pass a build-time argument to the Docker image build. Ansible's docker_image supports passing build args natively, but I need to see how it works, precisely, with defaults (e.g. can I just pass {}?), and ensure that they actually work correctly when passed.

See: https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg

Tag does not exist

Seeing the following error:

fatal: [127.0.0.1]: FAILED! => {"attempts": 10, "changed": true, "cmd": ["docker", "push", ""], "delta": "0:00:00.352007", "end": "2019-01-07 17:54:33.204128", "msg": "non-zero return code", "rc": 1, "start": "2019-01-07 17:54:32.852121", "stderr": "tag does not exist: ", "stderr_lines": ["tag does not exist: "], "stdout": "The push refers to repository []", "stdout_lines": ["The push refers to repository []"]}

ECR login with AWS CLI version 2

It seems that the command to log into ECR has changed for AWS CLI version 2. Instead of:

$(aws ecr get-login --no-include-email --region {{ ecr_region }})

it is now:

aws ecr get-login-password --region {{ ecr_region }} | docker login --username AWS --password-stdin {{ ecr_url }}

With AWS CLI version 2 installed, the task "Log into ECR if required" fails, since the CLI doesn't recognize the "get-login" argument. I wonder if it's possible to support both versions of the CLI either automatically or manually by specifying which CLI version the Ansible role should use. The version that I have describes itself as:

$ aws --version
aws-cli/2.0.46 Python/3.7.3 Linux/5.4.0-45-generic exe/x86_64.ubuntu.20

Many thanks.