linux-system-roles / template

example template for creating new subsystem roles

Role Name

A template for an ansible role that configures some GNU/Linux subsystem or service. A brief description of the role goes here.

Requirements

Any prerequisites that may not be covered by Ansible itself or the role should be mentioned here. This includes platform dependencies not managed by the role, hardware requirements, external collections, etc. There should be a distinction between control node requirements (like collections) and managed node requirements (like special hardware, platform provisioning).

Collection requirements

For instance, if the role depends on some collections and has a meta/collection-requirements.yml file for installing those dependencies, it should be mentioned here that the user should run

ansible-galaxy collection install -vv -r meta/collection-requirements.yml

on the control node before using the role.

Role Variables

A description of all input variables (i.e. variables that are defined in defaults/main.yml) for the role should go here as these form an API of the role. Each variable should have its own section e.g.

template_foo

This variable is required. It is a string that lists the foo of the role. There is no default value.

template_bar

This variable is optional. It is a boolean that tells the role to disable bar. The default value is true.

Variables that are not intended as input, like variables defined in vars/main.yml, variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) can be also mentioned here but keep in mind that as these are probably not part of the role API they may change during the lifetime.

Example of setting the variables:

template_foo: "oof"
template_bar: false

Variables Exported by the Role

This section is optional. Some roles may export variables for playbooks to use later. These are analogous to "return values" in Ansible modules. For example, if a role performs some action that will require a system reboot, but the user wants to defer the reboot, the role might set a variable like template_reboot_needed: true that the playbook can use to reboot at a more convenient time.

Example:

template_reboot_needed

Default false - if true, this means a reboot is needed to apply the changes made by the role

Example Playbook

Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too:

- name: Manage the template subsystem
  hosts: all
  vars:
    template_foo: "foo foo!"
    template_bar: false
  roles:
    - linux-system-roles.template

More examples can be provided in the examples/ directory. These can be useful, especially for documentation.

License

Whenever possible, please prefer MIT.

Author Information

An optional section for the role authors to include contact information, or a website (HTML is not allowed).

template's Issues

RFE: Update the minimum Ansible version and sync this to other roles

The minimum Ansible version in

template/meta/main.yml

Line 24 in d2239c7

min_ansible_version: 2.6

looks like something that should be updated and synced to roles. Other values like platform versions, probably, too.

[RFE] Common Linux System Roles Travis CI Setup

Common Linux System Roles Travis CI Setup

The following is just a draft and may be changed based on the comments below. The used code snippets may contain flaws and some constructs were not yet tested.

This RFE will propose how to achieve to have the same Travis CI setup across all Linux system roles.

Current Status

In Linux system roles project, there are typically two approaches of how Travis CI is set up (i.e. what is invoked by Travis depending on the content of .travis.yml):

• Travis runs the Molecule. This is common for timesync, kdump, and selinux roles. As there is no Python code in these roles, there is no need to run Python linters and then .travis.yml just contains Molecule commands together with necessary environment settings to run them.

• Travis runs the tox, from which are run Python linters and Molecule. This approach is used by network role. Also, .travis.yml is a bit more complex than in the previous roles. It uses build matrix and before_install section. When a new pull request is made, Travis creates for every element of the build matrix a job with environment based on the element of the build matrix definition. Then it prepares environment by running the commands from before_install and install sections, respectivelly, and finally it fires up commands specified in script. In script there is just one command: tox. tox is a tool that looks for tox.ini and based on its content it runs specified commands, mostly Python linters, but as can be seen in network's tox.ini, it can also run Molecule. Using tox-travis plugin, we can bind concrette test environments with concrette Travis jobs together, depending on the version of Python interpreter used.

The molecule setup is common for all Linux system roles.

Proposal

The proposed Travis CI setup that should be common for all Linux system roles is derived from network role.

The proposed general format of .travis.yml will be:

# Ubuntu distro name (here xenial):
dist: xenial
language: python
matrix:
  include:
    - python: 2.6
      # For Python 2.6, use older distro (here trusty):
      dist: trusty
      # We use LSRENV (the name of the variable can be changed if desired) to
      # save the unique value (here the desired Python interpreter version) to
      # distinguish between Travis jobs. This can be usefull if some role wish
      # to perform a specific action depending on the job environment:
      env: LSRENV=2.6
      # For now, only network role use Python linters, so save Travis CI
      # resources if we just need to run Molecule. For Travis conditions
      # syntax, see https://docs.travis-ci.com/user/conditions-v1
      if: repo = "linux-system-roles/network"
    - python: 2.7
      env: LSRENV=2.7
      if: repo = "linux-system-roles/network"
    - python: 3.5
      env: LSRENV=3.5
      # Under 3.5, we run Molecule
    - python: 3.6
      env: LSRENV=3.6
      if: repo = "linux-system-roles/network"
    - python: 3.7
      env: LSRENV=3.7
      if: repo = "linux-system-roles/network"
    - python: 3.7-dev
      env: LSRENV=3.7-dev
      if: repo = "linux-system-roles/network"
    - python: 3.8-dev
      env: LSRENV=3.8-dev
      if: repo = "linux-system-roles/network"
    #- python: nightly
    #  env: LSRENV=nightly
    #  if: repo = "linux-system-roles/network"

services:
  - docker

before_install:
  # In .travis/before_install.sh can be commands that install packages specific
  # for the concrette role, or set specific environment variables. To decide
  # what to set or install in which environment, the value of LSRENV should be
  # helpfull:
  - if [ -f .travis/before_install.sh ]; then .travis/before_install.sh; fi

install:
  - pip install molecule docker tox tox-travis

script:
  - tox

The proposed general format of tox.ini will be:

[tox]
# In `network` role's `tox.ini`, there is the `envlist` variable holding the
# list of test environments whose tests are run when `tox` is launched. But
# when using tox-travis plugin, the test environments are listed in `[travis]`
# section, so I am not sure if `envlist` is necessary:
#envlist = ...
skipsdist = true
skip_missing_interpreters = true

[testenv]
basepython = python3
# List common dependencies for Python interpreters here:
deps =
    py{26,27,36,37,38}: pytest-cov
    py{27,36,37,38}: pytest>=3.5.1
    py{26,27}: mock
    py26: pytest

[base]
passenv = *
setenv =
    PYTHONPATH = {toxinidir}/library:{toxinidir}/module_utils
    LC_ALL = C
changedir = {toxinidir}/tests
covtarget = {toxinidir}/library --cov {toxinidir}/module_utils
pytesttarget = .

[testenv:black]
deps =
    black
commands =
    black --check --diff --include "^[^.].*\.py$" --exclude "/(\.[^.].*|tests/roles)/" .

[testenv:py26]
install_command =
    pip install {opts} {packages}
list_dependencies_command =
    pip freeze
basepython = python2.6
passenv = {[base]passenv}
setenv =
    {[base]setenv}
changedir = {[base]changedir}
commands =
    pytest --durations=5 \
           --cov={[base]covtarget} \
           --cov-report=html:htmlcov-py26 \
           --cov-report=term \
           {posargs} \
           {[base]pytesttarget}

[testenv:py27]
basepython = python2.7
passenv = {[base]passenv}
setenv =
    {[base]setenv}
changedir = {[base]changedir}
commands =
    pytest --durations=5 \
           --cov={[base]covtarget} \
           --cov-report=html:htmlcov-py27 \
           --cov-report=term \
           {posargs} \
           {[base]pytesttarget}

[testenv:py36]
basepython = python3.6
passenv = {[base]passenv}
setenv =
    {[base]setenv}
changedir = {[base]changedir}
commands =
    pytest --durations=5 \
           --cov={[base]covtarget} \
           --cov-report=html:htmlcov-py36 \
           --cov-report=term \
           {posargs} \
           {[base]pytesttarget}

[testenv:py37]
basepython = python3.7
passenv = {[base]passenv}
setenv =
    {[base]setenv}
changedir = {[base]changedir}
commands =
    pytest --durations=5 \
           --cov={[base]covtarget} \
           --cov-report=html:htmlcov-py37
           --cov-report=term \
           {posargs} \
           {[base]pytesttarget}

[testenv:py38]
basepython = python3.8
passenv = {[base]passenv}
setenv =
    {[base]setenv}
changedir = {[base]changedir}
commands =
    pytest --durations=5 \
           --cov={[base]covtarget} \
           --cov-report=html:htmlcov-py38 \
           --cov-report=term \
           {posargs} \
           {[base]pytesttarget}

# Custom role settings may set LSR_PYLINT_DIRS environment variable if there is
# a reason to have other than default directories (library/ module_utils/ and
# tests/). For example, network role should have this set like
#
#   LSR_PYLINT_DIRS="library/network_connections.py module_utils/network_lsr tests/unit/test_network_connections.py"
#
[testenv:pylint]
basepython = python2.7
setenv =
    {[base]setenv}
deps =
    pylint>=1.8.4
    ansible
commands =
    pylint --errors-only \
           {posargs} \
           {env:LSR_PYLINT_DIRS:library module_utils tests}

[testenv:flake8]
basepython = python2.7
deps =
    flake8>=3.5
whitelist_externals = flake8
commands =
    flake8 --statistics {posargs} .

[testenv:coveralls]
basepython = python2.7
passenv = TRAVIS TRAVIS_*
deps =
    coveralls
changedir = {[base]changedir}
commands =
    coveralls

# Here we provide a way how a role can add its custom command to be run. Such
# extra command is run at the end of each testenv run and is driven by
# environment variables. Involved environment variables are:
#
#   LSR_TEXTRA_DEPS
#     - contains dependency needed by commands to run smoothly; if more than
#       one dependency is needed, use external file together with '-r' option
#       (see PEP 508)
#
#   LSR_TEXTRA_DIR
#     - directory to which to cd
#
#   LSR_TEXTRA_CMD
#     - custom command to be run
#
# Example: `network` system role need to run `./tests/ensure_non_running_provider.py`
#          to check for the existence of `*_provider.yml` playbooks. The script
#          is run in Python 3.6.
#
#          To make this possible, we add to `.travis/before_install.sh` a snippet:
#
#              if [ x${LSRENV} = x3.6 ]; then
#                LSR_TEXTRA_DEPS='PyYAML'
#                LSR_TEXTRA_DIR='tests'
#                LSR_TEXTRA_CMD='./ensure_non_running_provider.py'
#              fi
#
[testenv:extra]
passenv = *
deps =
    {env:LSR_TEXTRA_DEPS}
changedir = {toxinidir}/{env:LSR_TEXTRA_DIR:.}
commands =
    {env:LSR_TEXTRA_CMD:python --version}

# LSR_MOLECULE_DEPS may contain aditional Molecule dependencies. For example,
# in `network` system role, LSR_MOLECULE_DEPS can be set as
#
#   LSR_MOLECULE_DEPS='-rmolecule_requirements.txt'
#
# where `molecule_requirements.txt` contains two lines:
#
#   jmespath
#   git+https://github.com/tyll/selinux-pypi-shim@fulllocation
#
[molecule]
deps =
    docker
    molecule
    {env:LSR_MOLECULE_DEPS}

[testenv:molecule_lint]
deps =
    {[molecule]deps}
commands_pre =
    molecule --version
    ansible --version
commands =
    molecule {posargs} lint

[testenv:molecule_syntax]
deps =
    {[molecule]deps}
commands =
    molecule {posargs} syntax

[testenv:molecule_test]
deps =
    {[molecule]deps}
commands =
    molecule {posargs} test

[pytest]
addopts = -rxs

[flake8]
show_source = true
max-line-length = 88
ignore = E402,W503

[pylint]
max-line-length = 88
disable = wrong-import-position

[pycodestyle]
max-line-length = 88

[travis]
python =
  2.6: py26,extra
  2.7: py27,coveralls,flake8,pylint,extra
  3.5: molecule_lint,molecule_syntax,molecule_test,extra
  3.6: py36,black,extra
  3.7: py37,extra
  3.8: py38,extra

Other configuration files used by Python linters can be reused from network role.

Fix filemode warnings in molecule tests

https://travis-ci.com/github/linux-system-roles/network/jobs/373329596#L843

In the molecule tests there are warnings that should be fixed:

[WARNING]: File '/home/travis/.cache/molecule/network/default/Dockerfile_docker
844_io_linuxsystemroles_centos_6' created with default permissions '600'. The
845previous default was '666'. Specify 'mode' to avoid this warning.
846
847[WARNING]: File '/home/travis/.cache/molecule/network/default/Dockerfile_docker
848_io_linuxsystemroles_centos_7' created with default permissions '600'. The
849previous default was '666'. Specify 'mode' to avoid this warning.
850
851

virtualenv requires Python '!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7' but the running Python is 2.6.9

using the template recently I got the error above: https://travis-ci.org/github/linux-system-roles/timesync/jobs/667191881
We should not drop Python 2.6 support though, we need to test it for RHEL 6. Can we lock virtualenv to a version that supported Python 2.6? IIRC this used to work.

Best practices/linting rule for single blank line before each task

As discussed in one crypto policies PR, this is considered a best practice, but not explicitly mentioned anywhere and there is no enforcement in linting tools, making it manual work to enforce this and hard for new contributors.

We should try to create some linting rule for this and mention this in the best practices to ensure better readability of the roles code, tests and playbooks.

linux-system-roles/crypto_policies#3 (comment)

RFE: Support pylint and flake8 for Python 3, too

It would be nice to have pylint and flake8 running with Python 3, too, to check helper scripts that are only written for Python 3 for example for testing.

Add examples for config variables and format them so that they can be uncommented

There is no explanation in

template/.travis/config.sh

Line 26 in d2239c7

# - RUN_PYLINT_INCLUDE

how to use variables like RUN_PYLINT_EXCLUDE. It would be great to add an example how to exclude a file since I am struggling with it. Ideally the example would not be indented to allow to uncomment and amend it.

pylint tox environment hangs when installing ansible on Fedora 31

This seems to hang forever here on Fedora 31:

tox -vve pylint
pylint create: /home/till/scm/system-roles/template/.tox/env-2.7
pylint installdeps: ansible, colorama, pylint>=1.8.4, -rpylint_extra_requirements.txt
[...]
Python 2.7.
Collecting ansible
Collecting colorama
  Using cached https://files.pythonhosted.org/packages/c9/dc/45cdef1b4d119eb96316b3117e6d5708a08029992b2fee2c143c7a0a5cc5/colorama-0.4.3-py2.py3-none-any.whl
Collecting pylint>=1.8.4
  Using cached https://files.pythonhosted.org/packages/36/3b/fa4025a424adafd85c6195001b1c130ecb8d8b30784a1c4cb68e7b5e5ae7/pylint-1.9.5-py2.py3-none-any.whl
Collecting jinja2 (from ansible)
  Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
Collecting PyYAML (from ansible)
Collecting cryptography (from ansible)
  Using cached https://files.pythonhosted.org/packages/83/2e/2f39291aeb57931a97200d103dbab8530f27553410ae4b3357f080505d26/cryptography-2.9-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting configparser; python_version == "2.7" (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl
Collecting mccabe (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
Collecting six (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting isort>=4.2.5 (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl
Collecting backports.functools-lru-cache; python_version == "2.7" (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/da/d1/080d2bb13773803648281a49e3918f65b31b7beebf009887a529357fd44a/backports.functools_lru_cache-1.6.1-py2.py3-none-any.whl
Collecting astroid<2.0,>=1.6 (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/8b/29/0f7ec6fbf28a158886b7de49aee3a77a8a47a7e24c60e9fd6ec98ee2ec02/astroid-1.6.6-py2.py3-none-any.whl
Collecting singledispatch; python_version < "3.4" (from pylint>=1.8.4)
  Using cached https://files.pythonhosted.org/packages/c5/10/369f50bcd4621b263927b0a1519987a04383d4a98fb10438042ad410cf88/singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2->ansible)

Not sure why it sitll works on Travis.

add CODEOWNERS.template file

Add a file CODEOWNERS.template. This will be synced to the other repos as CODEOWNERS and will be the default github CODEOWNERS for that repo if it does not already have such a file. This will require a change to the https://github.com/linux-system-roles/auto-maintenance/blob/master/sync-template.sh to have the ability to copy a file to a repo with a different name.

RFE: Support podman for molecule tests

Originally reported at linux-system-roles/network#165 with some background info but since molecule testing is inherited from the template it seems to belong here.

To quote stringts or not to quote

Looking at files like defaults/main.yml and vars/main.yml, I see that strings are being quoted there. When working on another role @richm suggested to me that strings don't need to be quoted. Now I'm a little bit confused.

Looking at the template I would assume that's fine to quote strings. If that's not the case, the quotes should be removed from the var files in the template (I could volunteer to do that). But which way is correct? Please tell me if we have to quote or not to quote strings.