unexpected error when getting available versions of collection... community.general about ansible-builder

[kurokobo]

@ryeleo
Hi, this is due to Ansible being out of date and not being able to support recent changes on the Galaxy side.
As your execution-environment.yml if no base image is specified, quay.io/ansible/ansible-runner:latest is used by default, which is no longer maintained and can't be work with recent Galaxy.
You should explicitly specify base image like this for v3;

images:
  base_image:
    name: ghcr.io/ansible-community/community-ee-minimal:latest

or for v1:

build_arg_defaults:
  EE_BASE_IMAGE: ghcr.io/ansible-community/community-ee-minimal:latest

Here are another helpful examples:

• https://github.com/ansible-community/images/tree/main/execution-environments
• https://github.com/ansible/awx-ee
• https://github.com/kurokobo/awx-on-k3s/tree/main/builder

[ryeleo]

I was seeing this on my RHEL7 Jenkins build agent... I'm going to try and see if I can replicate this issue on my local workstation as well.

[ryeleo]

Reproduced on my local development box:

$ ansible-builder build -v3
Ansible Builder is building your execution environment image. Tags: ansible-execution-env:latest
File context/_build/requirements.yml is already up-to-date.
File context/_build/requirements.txt is already up-to-date.
Rewriting Containerfile to capture collection requirements
Running command:
  docker build -f context/Dockerfile -t ansible-execution-env:latest context
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.23kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for quay.io/ansible/ansible-runner:latest
#3 DONE 0.5s

#4 [internal] load metadata for quay.io/ansible/ansible-builder:latest
#4 DONE 0.5s

#5 [galaxy 1/5] FROM quay.io/ansible/ansible-runner:latest@sha256:001a4bde411be863d54c1d293f3d2e7b0ff0e67ef5d7b2f9f7fb56b61694f4e8
#5 DONE 0.0s

#6 [builder 1/5] FROM quay.io/ansible/ansible-builder:latest@sha256:75c4e4b5be6aab3386381b1e202ad382002f6830fd4b524bcd5ddc718e4f6a89
#6 CACHED

#7 [stage-2 2/5] RUN pip3 install --upgrade pip setuptools
#7 CACHED

#8 [internal] load build context
#8 transferring context: 113B done
#8 DONE 0.0s

#9 [galaxy 2/5] ADD _build /build
#9 CACHED

#10 [galaxy 3/5] WORKDIR /build
#10 CACHED

#11 [galaxy 4/5] RUN ansible-galaxy role install  -r requirements.yml --roles-path "/usr/share/ansible/roles"
#11 CACHED

#12 [galaxy 5/5] RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install  -r requirements.yml --collections-path "/usr/share/ansible/collections"
#12 0.835 Starting galaxy collection install process
#12 0.835 Process install dependency map
#12 4.230 [WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an
#12 4.230 unexpected error when getting available versions of collection
#12 4.230 community.general: '/api/v3/plugin/ansible/content/published/collections/index/
#12 4.230 community/general/versions/'
#12 4.231 ERROR! Unexpected Exception, this is probably a bug: '/api/v3/plugin/ansible/content/published/collections/index/community/general/versions/'
#12 4.231 to see the full traceback, use -vvv
#12 ERROR: process "/bin/sh -c ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path \"/usr/share/ansible/collections\"" did not complete successfully: exit code: 250
------
 > [galaxy 5/5] RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install  -r requirements.yml --collections-path "/usr/share/ansible/collections":
#12 0.835 Starting galaxy collection install process
#12 0.835 Process install dependency map
#12 4.230 [WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an
#12 4.230 unexpected error when getting available versions of collection
#12 4.230 community.general: '/api/v3/plugin/ansible/content/published/collections/index/
#12 4.230 community/general/versions/'
#12 4.231 ERROR! Unexpected Exception, this is probably a bug: '/api/v3/plugin/ansible/content/published/collections/index/community/general/versions/'
#12 4.231 to see the full traceback, use -vvv
------
Dockerfile:13
--------------------
  11 |     
  12 |     RUN ansible-galaxy role install $ANSIBLE_GALAXY_CLI_ROLE_OPTS -r requirements.yml --roles-path "/usr/share/ansible/roles"
  13 | >>> RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"
  14 |     
  15 |     FROM $EE_BUILDER_IMAGE as builder
--------------------
ERROR: failed to solve: process "/bin/sh -c ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path \"/usr/share/ansible/collections\"" did not complete successfully: exit code: 250

[ryeleo]

Tried making a venv with pip install "ansible-builder<=3", still seeing the same behavior...

ℹ🤔 Given all these failures, I'm really hoping this is a simple Docker networking issue on my end (even though it happened on two of my workstations). Hopefully this is going to make sense to me in the morning (otherwise I might try to spin up a new VM somewhere to try this build again with a totally untainted environment). 🤞😅🤞

$ ansible-builder --version
1.2.0

I also updated my "execution-environment.yml" to use version 1:

---
version: 1

dependencies:
  galaxy: requirements.yml

additional_build_steps:
  prepend: |
    RUN pip3 install --upgrade pip setuptools

The output using ansible-builder 1.2.0 seems the same as for v3.0.0.

$ ansible-builder build -v3
Ansible Builder is building your execution environment image. Tags: ansible-execution-env:latest
File context/_build/requirements.yml is already up-to-date.
File context/_build/requirements.txt is already up-to-date.
Rewriting Containerfile to capture collection requirements
Running command:
  docker build -f context/Dockerfile -t ansible-execution-env:latest context
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.23kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for quay.io/ansible/ansible-runner:latest
#3 DONE 0.1s

#4 [internal] load metadata for quay.io/ansible/ansible-builder:latest
#4 DONE 0.2s

#5 [galaxy 1/5] FROM quay.io/ansible/ansible-runner:latest@sha256:001a4bde411be863d54c1d293f3d2e7b0ff0e67ef5d7b2f9f7fb56b61694f4e8
#5 DONE 0.0s

#6 [builder 1/5] FROM quay.io/ansible/ansible-builder:latest@sha256:75c4e4b5be6aab3386381b1e202ad382002f6830fd4b524bcd5ddc718e4f6a89
#6 CACHED

#7 [stage-2 2/5] RUN pip3 install --upgrade pip setuptools
#7 CACHED

#8 [internal] load build context
#8 transferring context: 113B done
#8 DONE 0.0s

#9 [galaxy 2/5] ADD _build /build
#9 CACHED

#10 [galaxy 3/5] WORKDIR /build
#10 CACHED

#11 [galaxy 4/5] RUN ansible-galaxy role install  -r requirements.yml --roles-path "/usr/share/ansible/roles"
#11 CACHED

#12 [galaxy 5/5] RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install  -r requirements.yml --collections-path "/usr/share/ansible/collections"
#12 0.785 Starting galaxy collection install process
#12 0.785 Process install dependency map
#12 3.717 [WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an
#12 3.717 unexpected error when getting available versions of collection
#12 3.717 community.general: '/api/v3/plugin/ansible/content/published/collections/index/
#12 3.717 community/general/versions/'
#12 3.724 ERROR! Unexpected Exception, this is probably a bug: '/api/v3/plugin/ansible/content/published/collections/index/community/general/versions/'
#12 3.725 to see the full traceback, use -vvv
#12 ERROR: process "/bin/sh -c ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path \"/usr/share/ansible/collections\"" did not complete successfully: exit code: 250
------
 > [galaxy 5/5] RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install  -r requirements.yml --collections-path "/usr/share/ansible/collections":
#12 0.785 Starting galaxy collection install process
#12 0.785 Process install dependency map
#12 3.717 [WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an
#12 3.717 unexpected error when getting available versions of collection
#12 3.717 community.general: '/api/v3/plugin/ansible/content/published/collections/index/
#12 3.717 community/general/versions/'
#12 3.724 ERROR! Unexpected Exception, this is probably a bug: '/api/v3/plugin/ansible/content/published/collections/index/community/general/versions/'
#12 3.725 to see the full traceback, use -vvv
------
Dockerfile:13
--------------------
  11 |     
  12 |     RUN ansible-galaxy role install $ANSIBLE_GALAXY_CLI_ROLE_OPTS -r requirements.yml --roles-path "/usr/share/ansible/roles"
  13 | >>> RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"
  14 |     
  15 |     FROM $EE_BUILDER_IMAGE as builder
--------------------
ERROR: failed to solve: process "/bin/sh -c ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path \"/usr/share/ansible/collections\"" did not complete successfully: exit code: 250

An error occured (rc=1), see output line(s) above for details.

[ryeleo]

WHOOPS, I am just now realizing that I am using python3.8, and the latest ansible-builder (3.0.0) doesn't support python 3.8. 🤦‍♂️

I will go switch to a newer version of python for my venv and see what happens.

[ryeleo]

@kurokobo, I am very grateful for your advice! You are correct, this did fix the issue!

I do feel a fool for not having thought to look more into the ansible-builder documentation, which chats about the 'base image', and has a nice 'sample' showing how it can be done.

I just glanced over the "Breaking Changes" in the release notes and assumed that 'it is the same project, I doubt anything I'm using broke.' 👍😅👍

Thanks again!