ansible / ansible-zuul-jobs Goto Github PK

View Code? Open in Web Editor NEW

20.0 5.0 41.0 1.98 MB

Zuul job definitions for the Ansible tenant.

License: Apache License 2.0

Python 75.98% Jinja 24.02%

ansible-zuul-jobs's Introduction

ansible-zuul-jobs

Zuul job definitions for Ansible tenant.

ansible-zuul-jobs's People

Contributors

Stargazers

Watchers

ansible-zuul-jobs's Issues

Enable fips mode on stable branch libssh jobs for arista.eos

FIPS mode is disabled for stable-2.11 and stable-2.9 jobs.
It has to be enabled after ansible/ansible#75706 is merged for 2.11.
For 2.9 - ansible/ansible#73992 has to be backported.

Need Ansible Core 2.13 sanity test job for ansible.posix

ansible.posix is using ansible-zuul-jobs for sanity testing.
However, there is no testing with 2.13 now, so we need to add 2.13 to zuul.d/project-templates.yaml in ansible.posix section.

For example ansible-collections/ansible.posix#386 (comment)

migrate away from fedora-34-4vcpu nodeset

@jillr @tremble @goneri Can we migrate your jobs away from fedora-34-4vcpu? This image only runs in aws, and we only have capacity to run 5 jobs at a time.

I would like to see it migrated to centos-8 nodesets, given we run them in many more providers.

Also, do your controller jobs really need 4vcpu? or can it be scaled down to 1vcpu / 2vcpu.

Lint extra docsite docs and links

I was asked to add an additional antsibull-docs lint-collection-docs test to the community.vmware collection.

But befor I do as proposed using GitHub Actions, I wanted to ask if it wouldn't be possible to add a Zuul job for this. This way, all the other collections tested with Zuul could get the extra lint test, too.

cc @felixfontein @goneri @komaldesai13

network-ee-build-container-image* jobs stuck during tag push

Jobs stay stuck at and timeout

LOOP [push-to-intermediate-registry : Push tag to intermediate registry]

e.g: https://8e0e82302d2fae3b5390-711bba4e58c405e4ee9bb7d6b67e10dd.ssl.cf1.rackcdn.com/456/b206a692460d0c462acc95ffbd26a2b92eadd920/check/network-ee-build-container-image/db64cbf/job-output.txt

Need Ansible Core 2.14 sanity test job for ansible.posix

We need to cover Ansible Core 2.14 in the sanity tests for ansible.posix collection.

Reference to #1623

ansible-fedora-35-1vcpu node is obsolete and must be removed

Hello,

Many jobs still require the ansible-fedora-35-1vcpu node type, but since this node is obsolete, these jobs end up with NODE_FAILURE, see for example ansible-collections/ansible.netcommon#618 (comment)

This fires up our alerting and could be fixed by removing the obsolete node types from the jobs definitions.

build-container-image: Build a container image fails often

The network-ee-build-container-image jobs depends on remote resources (quay.io or internal registry) and will fail if there is a network problem. A retry mechanism would be useful to reduce the number of failure.

e.g: https://dashboard.zuul.ansible.com/t/ansible/build/c032b4d52cc445618c2c3acec4ccf7bd/console

KVM / virt nodes ?

molecule-vagrant is needing support for virtualization and from time to time, the tests are failing due to missing support for virtualization. The tests are failing like this:

centos-8 | Error while creating domain: Error saving the server: Call to virDomainDefineXML failed: invalid argument: could not find capabilities for domaintype=kvm

(ref: https://dashboard.zuul.ansible.com/t/ansible/build/426dd647e97141b9b87f49b6a4c4b364/log/job-output.txt)

Not only it makes the testsuite fail randomly, but it's also a waste of resources.
Is there a way to run tests only on hosts providing virtualization ?

Make network-ee-unit-tests and integration tests voting

Revert changes of #1251 when ansible/ansible#76442 this gets resolved.

AWS jobs not testing aliased changes

Visible in this test PR.

Testing with ansible-collections/community.aws#700 with aws_config_rule as the only change no tests were run.
https://f7d8ed960619cc8d3749-b7d7790083b1c99ed8702216bc5cc6fc.ssl.cf1.rackcdn.com/700/14c86de5a5302cc9fa000d0d1940c0feda399ac1/check/ansible-test-splitter/4ffcef6/job-output.txt

https://github.com/ansible/ansible-zuul-jobs/blob/master/zuul.d/project-templates.yaml#L108 should have the effect of testing only modules which have changed modules or tests.

It happens this is an alias in aws_config. I added a change to a non-aliased file (aws_region_info) and only aws_config_rule is identified by the test splitter.
https://f7d8ed960619cc8d3749-b7d7790083b1c99ed8702216bc5cc6fc.ssl.cf1.rackcdn.com/700/14c86de5a5302cc9fa000d0d1940c0feda399ac1/check/ansible-test-splitter/4ffcef6/job-output.txt

I'd venture a guess it's something in roles/ansible-test-splitter/files/test_changed.py?

AWS sanity tests are broken

https://dashboard.zuul.ansible.com/t/ansible/build/2b628c2b181348c98246cfe02005bf6e/console

Jobs are failing due to this task's until statement and no PRs can land.
https://github.com/ansible/ansible-zuul-jobs/blob/master/roles/download-artifact-fork/tasks/main.yaml#L35

Maybe related to this recent AWS change? The timing is awfully suspicious, I don't see the connection though.
#915

urllib3 / six install broken in ansible-ee-tests-stable-2.12 / ansible-ee-tests-stable-devel environments

Tests using boto3/botocore are failing due to a broken urllib/six install. This is likely caused by ansible.netcommon installing python38-six, which then gets reinstalled via pip

need podman and docker roles

Molecule tox jobs may need docker-ce and/or podman installed, we need some roles we can add to enable them. I will try to use the roles from upstream zuul-jobs.

PS. Assign it to me

Consider renaming the default branch to main

As per $title

fedora-31 image got broken due to mirrors

Apparently last week the fedora-31 image started to fail in pre phase as it failed to download RPMs from one of fedora mirrors. This clearly counts as zuul infra issue.

repo: downloading from remote: fedora-modular
error: Status code: 403 for http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml (IP: 129.7.128.189) (http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml).
error: Status code: 403 for http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml (IP: 129.7.128.189) (http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml).
error: Status code: 403 for http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml (IP: 129.7.128.189) (http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml).
error: Status code: 403 for http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml (IP: 129.7.128.189) (http://pubmirror1.math.uh.edu/fedora-buffet/fedora/linux/releases/31/Modular/x86_64/os/repodata/repomd.xml).
Fedora Modular 31 - x86_64                      539  B/s | 199  B     00:00

https://dashboard.zuul.ansible.com/t/ansible/build/16d1970891534aae8d3969cbf83c15d3

In correlation with unresolved #445 this means that currently we have no way to test py37 and py38 on any zuul node. This is a critical issue as both are current stable versions of python used in production.

Since the last week we where not able to merge anything in molecule* projects.

fedora-32 nodeset missing, needed for py38

Apparently we are missing fedora-32 nodepool image, which is a real deal because it woud have sorted the missing py38 testing environment.

For example any jobs in tox-py38 category would fail because there is no python3.8 installed on any of the nodes.

If we enable fedora-32, we can change the tox jobs to point to it and cover this python version too.

Zuul jobs failing

Zuul jobs are failing, but I don't understand why. See ansible-collections/community.vmware#2029 or ansible-collections/community.vmware#2039 for examples.

Any idea what's wrong there? It looks like a Zuul issue for me, but if you think the problem is in community.vmware it would be great if you could give me a hint how to fix it.

ansible-test-sanity-docker needs to move to Py38 or later

Summary

ansible-test(core 2.12.0.dev0) is no longer supports Python3.6 and ansible-test-sanity-docker in CI tests has failed due to using unsupported version Python3,6.8.
It probably needs to move to Python3.8 or later.

Expected result

ansible-test-sanity-docker test is passed without errors.

Actual result

The following example is for test PR(ansible-collections/ansible.posix/pull/268) collection. It failed with the following result:

...snip...
2021-09-21 01:59:59.170029 | TASK [ansible-test : debug]
2021-09-21 02:00:09.511613 | About to run: ansible-test sanity --requirements --python 3.6 --docker -v
2021-09-21 02:00:09.550683 | 
2021-09-21 02:00:09.550809 | TASK [ansible-test : Run the test suite]
2021-09-21 02:00:09.996844 | centos-8-stream | This version of ansible-test cannot be executed with Python version 3.6.8. Supported Python versions are: 3.8, 3.9, 3.10
2021-09-21 02:00:10.124150 | centos-8-stream | ERROR
2021-09-21 02:00:10.124483 | centos-8-stream | {
2021-09-21 02:00:10.124567 | centos-8-stream |   "delta": "0:00:00.036070",
2021-09-21 02:00:10.124626 | centos-8-stream |   "end": "2021-09-21 02:00:10.000016",
2021-09-21 02:00:10.124681 | centos-8-stream |   "msg": "non-zero return code",
2021-09-21 02:00:10.124734 | centos-8-stream |   "rc": 1,
2021-09-21 02:00:10.124787 | centos-8-stream |   "start": "2021-09-21 02:00:09.963946"
2021-09-21 02:00:10.124881 | centos-8-stream | }
...snip...
```

TASK [Install ansible into virtualenv] is failed with 'ansible-core' python requirement in the unit test of ansible.posix

Description

ansible-test-units-posix in one of PRs has been failed with the ERROR: Package 'ansible-core' requires a different Python: 3.8.13 not in '>=3.9'.

Collections

ansible.posix

Expected results

ansible-test-units-posix is finished without errors.

Actual result

ansible-test-units-posix has been failed with the following error:

2022-08-19 05:11:41.154472 | TASK [Install ansible into virtualenv]
2022-08-19 05:11:41.657218 | controller | Processing ./src/github.com/ansible/ansible
2022-08-19 05:11:41.657496 | controller |   DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
2022-08-19 05:11:41.657509 | controller |    pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
2022-08-19 05:11:43.048523 | controller |   Installing build dependencies: started
2022-08-19 05:11:45.377006 | controller |   Installing build dependencies: finished with status 'done'
2022-08-19 05:11:45.380549 | controller |   Getting requirements to build wheel: started
2022-08-19 05:11:45.872052 | controller |   Getting requirements to build wheel: finished with status 'done'
2022-08-19 05:11:45.876328 | controller |     Preparing wheel metadata: started
2022-08-19 05:11:46.366512 | controller |     Preparing wheel metadata: finished with status 'done'
2022-08-19 05:11:46.486742 | controller | Collecting packaging
2022-08-19 05:11:46.490650 | controller |   Using cached packaging-21.3-py3-none-any.whl (40 kB)
2022-08-19 05:11:46.545887 | controller | Collecting jinja2>=3.0.0
2022-08-19 05:11:46.564821 | controller |   Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
2022-08-19 05:11:46.699069 | controller | Collecting PyYAML>=5.1
2022-08-19 05:11:46.706269 | controller |   Downloading PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
2022-08-19 05:11:47.235800 | controller | Collecting cryptography
2022-08-19 05:11:47.244534 | controller |   Downloading cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
2022-08-19 05:11:47.349749 | controller | Collecting resolvelib<0.9.0,>=0.5.3
2022-08-19 05:11:47.353905 | controller |   Downloading resolvelib-0.8.1-py2.py3-none-any.whl (16 kB)
2022-08-19 05:11:47.358604 | controller | ERROR: Package 'ansible-core' requires a different Python: 3.8.13 not in '>=3.9'
2022-08-19 05:11:47.361281 | controller | WARNING: You are using pip version 21.2.3; however, version 22.2.2 is available.
2022-08-19 05:11:47.361295 | controller | You should consider upgrading via the '/home/zuul/venv/bin/python -m pip install --upgrade pip' command.
2022-08-19 05:11:48.187772 | controller | ERROR
2022-08-19 05:11:48.188167 | controller | {
2022-08-19 05:11:48.188222 | controller |   "delta": "0:00:06.336295",
2022-08-19 05:11:48.188259 | controller |   "end": "2022-08-19 05:11:47.713807",
2022-08-19 05:11:48.188294 | controller |   "msg": "non-zero return code",
2022-08-19 05:11:48.188327 | controller |   "rc": 1,
2022-08-19 05:11:48.188360 | controller |   "start": "2022-08-19 05:11:41.377512"
2022-08-19 05:11:48.188392 | controller | }
2022-08-19 05:11:48.202703 | 
2022-08-19 05:11:48.202788 | PLAY RECAP
2022-08-19 05:11:48.202833 | controller | ok: 10 changed: 6 unreachable: 0 failed: 1 skipped: 4 rescued: 0 ignored: 0

Wrong collections path in galaxy-importer job

After ansible-network/releases#17 stopped bringing in the entire Ansible package it became apparent that the incorrect collections path is being used in this job. Collections which use doc fragments from other collections have been failing CI for all jobs since (ie; community.aws used amazon.aws fragments, and community.okd uses fragments from kubernetes.core).

https://bfcb0036379da16f975f-4282179313616d046bba66d871c7a9ef.ssl.cf5.rackcdn.com/579/16b009a77e4f66d679b0308972f29c92577bf4d3/check/ansible-galaxy-importer/bf50899/job-output.html#l779

In irc we determined this has to do with the collections path
https://gist.github.com/jillr/f2a01a25cc902a1a89194a2dccecf749

ansible-tox-py35 and ansible-tox-py38 are missing

Currently, we do not have these jobs defined:

ansible-tox-py35
ansible-tox-py36

AWS Pipelines are broken

For some days now (I'm having difficulty narrowing it down as there seem to be multiple related causes) CI has been failing for both collections. At least for the last 3 days no jobs pass.

Observed issues:

Timeouts. We have a job splitter but it doesn't seem to be doing a good job of dividing tests. To my knowledge there's no way to manual sort targets into groups the way we did with shippable to try to achieve some balance here.
All PRs seem to be running the equivalent of ci_complete on check. Which leads to timeouts, since jobs are unbalanced.
My understanding is that we can mark tests that take a while to run as "slow" in the aliases file and they'll run in their own job, rather than ansible-test-cloud-integration-aws-py36_N. That does not appear to be working, contributing to timeouts.

We don't want ci_complete to run on ansible/check, only when requested or on gate. As far as I can tell nothing ever sets the necessary var for this task to execute:
https://github.com/ansible/ansible-zuul-jobs/blob/master/roles/ansible-test-splitter/tasks/main.yaml#L4

And indeed the conditional is False.

2021-07-29 22:54:17.896606 | PLAY [controller]
2021-07-29 22:54:17.942431 | 
2021-07-29 22:54:17.942612 | TASK [ansible-test-splitter : copy]
2021-07-29 22:54:18.035965 | fedora-34 | skipping: Conditional result was False
2021-07-29 22:54:18.086002 | 
2021-07-29 22:54:18.086124 | TASK [ansible-test-splitter : Identify the changed targets]
2021-07-29 22:54:18.202013 | fedora-34 | skipping: Conditional result was False
2021-07-29 22:54:18.236949 | 
2021-07-29 22:54:18.237066 | TASK [ansible-test-splitter : set_fact]
2021-07-29 22:54:18.321551 | fedora-34 | skipping: Conditional result was False

I'm reading my way through the AWS Zuul config in its entirety trying to figure out what we ought to be doing but I haven't come up with anything yet.

Fixed-By: #1013

Unit tests are running against an EoL version of Ansible-Core

the base unit test jobs are explicitly using stable-2.11 - for example:

- job:
    name: ansible-test-units-base-python37
    parent: ansible-test-units-base
    nodeset: controller-node
    abstract: true
    required-projects:
      - name: github.com/ansible/ansible
        override-checkout: stable-2.11
    vars:
      ansible_test_python: 3.7

stable-2.11's no longer supported, with the last release being 6 months ago.

fedora-30 nodeset used for tox py27 jobs does not have python2-libselinux

Currently tox 27 jobs are using a fedora-30 node but they miss the python2-libselinux package which is critical for some Ansible modules, mainly blocking testing molecule.

I cannot put python2-libselinux inside bindep because is a conditional dependency, it is needed only for the python version that you are using.

Thus I think we need to be sure that when we setup tox nodesets we need to double-check that selinux bindings are also installed. As you know that is one library that cannot be compiled by pip.

Build failed with /tmp/wait_for_vcenter: line 4: unexpected EOF while looking for matching ``'

Hi,

I created a PR at ansible-collections/community.vmware#1623. The build failed with

2023-02-06 21:57:39.020052 \| TASK [Run wait_for_vcenter]
--
935 | 2023-02-06 21:57:39.369813 \| controller \| + server=https://vcenter.test
936 | 2023-02-06 21:57:39.369883 \| controller \| /tmp/wait_for_vcenter: line 4: unexpected EOF while looking for matching ``'
937 | 2023-02-06 22:49:32.052073 \| controller \| ERROR
938 | 2023-02-06 22:49:32.052322 \| controller \| {
939 | 2023-02-06 22:49:32.052375 \| controller \|   "attempts": 600,
940 | 2023-02-06 22:49:32.052440 \| controller \|   "delta": "0:00:00.001959",
941 | 2023-02-06 22:49:32.052476 \| controller \|   "end": "2023-02-06 22:49:32.018234",
942 | 2023-02-06 22:49:32.052508 \| controller \|   "msg": "non-zero return code",
943 | 2023-02-06 22:49:32.052538 \| controller \|   "rc": 2,
944 | 2023-02-06 22:49:32.052568 \| controller \|   "start": "2023-02-06 22:49:32.016275"
945 | 2023-02-06 22:49:32.052597 \| controller \| }

ansible-zuul-jobs/playbooks/ansible-cloud/vcenter-integration-controller/pre.yaml

Lines 16 to 35 in 33e5b0f

	- name: Prepare the wait for vcenter script
	copy:
	content: \|
	#!/bin/bash
	set -eux
	server='https://vcenter.test'
	password="{{ lookup('file', '{{ zuul.executor.work_root }}/vcenter/tmp/vcenter_password.txt') }}"
	session_id=$(curl -q --http1.1 -X POST -k -u "[email protected]:${password}" ${server}/rest/com/vmware/cis/session\|jq -r .value)
	ret=$(curl -k -H vmware-api-session-id:$session_id -X GET ${server}/api/vcenter/vm)
	if [ "${ret}" = "[]" ]; then
	echo "vCenter is ready"
	exit 0
	fi
	exit 1
	dest: "/tmp/wait_for_vcenter"
	- name: Run wait_for_vcenter
	command: "bash /tmp/wait_for_vcenter"
	register: result
	until: result is not failed
	retries: 600

It would be great if you could catch this error and output a more useful error message.

Thank you,
Carsten

ansible-test-sanity-docker jobs TIME_OUT

sanity tests often fail during /usr/bin/python3.8 -m rstcheck --report warning --ignore-substitutions _,br,release,today,version CHANGELOG.rst

e.g: https://36a2412fb528f2268b0e-ca3223f8735de702b39597abafa0b0a3.ssl.cf2.rackcdn.com/93/a0740c2b2784cbd0dee4e6166500b899381767b2/check/ansible-test-sanity-docker-stable-2.10/d8c9751/job-output.txt

ansible_test_collections variable should go away

The ansible_test_collections was introduced to be able to identify the modules coming from core and the ones coming from the collections. We only support collections now and we don't need this logic in our test playbooks. Worst, it's recently be the cause of a regression.

The idea is to first turn the value True by default and the second step is to remove the variable from our playbooks.

Some ansible-ee-integration-arista-eos-libssh jobs missing nslookup binary

❌ ansible-ee-integration-arista-eos-libssh-latest RETRY_LIMIT in 12m 16s
❌ ansible-ee-integration-arista-eos-libssh-stable-2.11 RETRY_LIMIT in 13m 00s
❌ ansible-ee-integration-arista-eos-libssh-stable-2.12 RETRY_LIMIT in 12m 22s

ansible / ansible-zuul-jobs Goto Github PK

ansible-zuul-jobs's Introduction

ansible-zuul-jobs

ansible-zuul-jobs's People

Contributors

Stargazers

Watchers

Forkers

ansible-zuul-jobs's Issues

Summary

Expected result

Actual result

Description

Collections

Expected results

Actual result

Recommend Projects

Recommend Topics

Recommend Org

Jobs