thegeeklab / ansible-doctor Goto Github PK

ansible-doctor should not silently fail if the given template not exist

I want to create a table:

# @var test: >
#
# | Attribute | Description |
# | --- | --- |
# | att | test |
#
# @end

expected result:

### test

| Attribute | Description |
| --- | --- |
| att | test |

result:

### test

| Attribute | Description | | --- | --- | | att | test |

An extra linefeed is not allowed in markdown:

### test

| Attribute | Description |

| --- | --- |

| att | test |

The last example could easily created but is not working due to wrong syntax.

Is there a way to create tables with annotations?

Kind Regards
Thorsten

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(docker): update python:3.12-alpine docker digest to b7662fc

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): lock file maintenance

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• fix(deps): update dependency ansible-core to v2.17.1

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Hi @xoxys,
Hope you are doing well.
This pertains to the error produced by the ansible-doctor command when generating README.md files for roles with an empty defaults folder (no values in the main.yml file under defaults folder) in the current version 3.1.2. I am trying to generate the README.md file but it is failing with below error.

aradhana.saxena@DEL1-LHP-N82532:~/bacula_client_rework_17Nov/nuxeo-devops/playbooks/roles$ ansible-doctor bacula_client/ -f 
Traceback (most recent call last):
  File "/usr/local/bin/ansible-doctor", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/ansibledoctor/cli.py", line 127, in main
    AnsibleDoctor()
  File "/usr/local/lib/python3.10/dist-packages/ansibledoctor/cli.py", line 23, in __init__
    self._execute()
  File "/usr/local/lib/python3.10/dist-packages/ansibledoctor/cli.py", line 121, in _execute
    doc_parser = Parser()
  File "/usr/local/lib/python3.10/dist-packages/ansibledoctor/doc_parser.py", line 29, in __init__
    self._parse_var_files()
  File "/usr/local/lib/python3.10/dist-packages/ansibledoctor/doc_parser.py", line 38, in _parse_var_files
    raw = parse_yaml(yamlfile)
  File "/usr/local/lib/python3.10/dist-packages/ansibledoctor/utils/yamlhelper.py", line 55, in parse_yaml
    data = defaultdict(dict, data)
TypeError: 'NoneType' object is not iterable

However this error didn't occur in previous version 3.0.0 version even when there is an empty main.yml file under defaults folder and README.md file is successfully generated

aradhana.saxena@DEL1-LHP-N82532:~/bacula_client_rework_17Nov/nuxeo-devops/playbooks/roles$ ansible-doctor --version ansible-doctor 3.0.0 aradhana.saxena@DEL1-LHP-N82532:~/bacula_client_rework_17Nov/nuxeo-devops/playbooks/roles$ ansible-doctor bacula_client/ [WARNING] This files will be overwritten: ... /home/aradhana.saxena/bacula_client_rework_17Nov/nuxeo-devops/playbooks/roles/bacula_client/README.md Do you want to continue? [Y/n] Y
Please guide if this check is intentionally added in this current version

Hello,
The default readme template lists dependencies as literal ordereddict objects.
https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#using-role-dependencies

Steps to reproduce

1. Add the following dependency to example/demo-role/meta/main.yml

dependencies:
  - role: example-role

3. Generate docs with no options

Expected Result

## Dependencies

- example-role

Actual Result

## Dependencies

- ordereddict([('role', 'example-role')])

Description

When using ansible-doctor 2.0 for a ansible role with the following in the meta/main.yml this is written in to the readme. As you can see ansible doctor uses a new line for each letter which is kind of strange.

meta/main.yml:

  description: Installs Sentry.io on premise self hosted version.

`README.md´:

# common_roles_pre

I
n
c
l
u
d
e
s

c
o
m
m
o
n

p
r
e

A
n
s
i
b
l
e

r
o
l
e
s

t
h
a
t

a
r
e

r
u
n

b
e
f
o
r
e

t
h
e

m
a
i
n

r
o
l
e
.

## Table of content

Say I have a role that loads some values from all.yaml...how would I be able to export those values to markdown here?
The {{ group_vars }} ansible variable isnt available to this tool by default (which makes sense since its not targeting any group).

Would I be able to make a role called all or something and load the variables there?

Add better parsing for log level passed from config (and catch exception)

Traceback (most recent call last):
  File "/usr/local/bin/ansible-doctor", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/__main__.py", line 8, in main
    AnsibleDoctor()
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/Cli.py", line 23, in __init__
    self.config = self._get_config()
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/Cli.py", line 60, in _get_config
    self.log.set_level(config.config["logging"]["level"])
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/Utils.py", line 150, in set_level
    self.logger.setLevel(s)
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1353, in setLevel
    self.level = _checkLevel(level)
  File "/usr/local/lib/python3.7/logging/__init__.py", line 192, in _checkLevel
    raise ValueError("Unknown level: %r" % level)
ValueError: Unknown level: 'info'

Hi,

I started using ansible-doctor to document my roles, and, following the documentation, I see that I should be able to override an existing variable value, like so:

---
# defaults file for test

# @var testvar:value: "foo"
testvar: "secret"

However, when I run ansible-doctor, I have an error:

[CRITICAL] Unable to merge annotation values:
... too many values to unpack (expected 2) other=['"foo"']

Is this a normal behavior?

Thanks a lot for your answer!

Some multiline yaml values looks malformed in generated doc:

freshrss_open_ports:
  - name: allow_freshrss_web
    rules: "-A INPUT -m state --state NEW -p tcp --dport {{ freshrss_exposed_port\
      \ }} -j ACCEPT\n"
    state: present

When using custom yaml tags like Ansibles !unsafe notation, yaml load will fail with:

Traceback (most recent call last):
  File "/usr/local/bin/ansible-doctor", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/__main__.py", line 8, in main
    AnsibleDoctor()
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/Cli.py", line 25, in __init__
    doc_parser = Parser()
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/DocumentationParser.py", line 29, in __init__
    self._parse_var_files()
  File "/usr/local/lib/python3.7/site-packages/ansibledoctor/DocumentationParser.py", line 38, in _parse_var_files
    data = defaultdict(dict, (ruamel.yaml.safe_load(yaml_file) or {}))
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/main.py", line 996, in safe_load
    return load(stream, SafeLoader, version)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/main.py", line 951, in load
    return loader._constructor.get_single_data()
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 113, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 123, in construct_document
    for _dummy in generator:
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 723, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 440, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 255, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 146, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 181, in construct_non_recursive_object
    data = constructor(self, node)
  File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 743, in construct_undefined
    node.start_mark,
ruamel.yaml.constructor.ConstructorError: could not determine a constructor for the tag '!unsafe'
  in "/drone/src/defaults/main.yml", line 149, column 30

Currently, the newest release of ansible-doctor has a hard dependency on ansible-core 2.13, which in turn forces us to use Ansible 6.

This version of Ansible had been deprecated (https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html) .

Could we loosen the restriction here to be ansible-core >=2.13 , so that users can use more modern versions?

When generating the README.md file using ansible-doctor, its auto discovering feature is detecting ansible module tags which is then appearing under "Discovered tags," despite the documentation stating that the task tags should be displayed under "Discovered tags" heading.
Please confirm if this is the expected behavior. Also please confirm if there is any way to skip auto-detection of ansible module tags.

For this file system:

.
├── default
│   └── main.yml
└── subset
    └── main.yml

default/main.yml:

# @var default_variable:value: >
# 2
# @end

subset/main.yml:

# @var subset_variable:value: >
# 1
# @end

I ran this command: ansible-doctor -n (same issue if I run mkdir tasks && ansible-doctor) and the generated README.md looks like (triple backticks replaced by triple single quotes):

# 

## Table of content

* [Default Variables](#default-variables)
  * [default_variable](#default_variable)
  * [subset_variable](#subset_variable)
* [Dependencies](#dependencies)
* [License](#license)
* [Author](#author)

---

## Default Variables

### default_variable

#### Default value

'''YAML
2
'''

### subset_variable

#### Default value

'''YAML
1
'''

## Dependencies

None.

The documentation mentions this:

After the successful check, ansible-doctor will try to read some static files into a dictionary:
* defaults/main.yml
* meta/main.yml

But with the above file system, ansible-doctor is clearly searching annotations into subset/ directory to generate the README.md. Did I miss something ? Is it an expected behavior or is it the last update which lead to this issue ?

It would be pretty great if we were able to have a way to pass in a URL where the application could pull down a remote tar.gz, zip or whatever that provided a shared template. Currently, in order to use this among the many roles we maintain, we'd have to add our custom template into each role to be checked in with each project.

Yes, there are multiple ways around this, but it would be nice to have a single URL each of the projects could point to to have the pull down a template archive to use.

Is there any way to skip some playbook tags? For example, I don't want the tags "never" and "always" to get into the documentation. everyone knows their meaning

If you put the markdown around the author name/email it doesn't render anything into the README.md

According to the title, I am opening this issue to know if ansible-doctor provides a feature to use "environment variables" in jinja2 templates (page path, page URL, ...). I am not an advanced user of jinja2, but after several researches, it seems there is no way in a jinja2 template to know the page path without using flask, sphinx or other python modules. To avoid confusion for readers here is what I am calling a page path:

www.mywebsite.domain/my/page/path/
                    ^^^^^^^^^^^^^^

Here is a list of several solutions I considered without real satisfaction:

• Putting the page path in a jinja2 variable, but I'm using ansible-doctor to generate documentation for ~150 of my ansible roles (this number is always bigger) and the template goal is to be reusable,
• Using meta.name tag to build the page path. It works but this solution leads to:
  1. hardly human-readable templates,
  2. not easily maintainable templates (pages path could change),
• Declaring a new meta.path tag I will use in my ansible roles. Again this solution works but the path page could change so this solution is not easily maintainable.

After searching in ansible-doctor/ansibledoctor/doc_generator.py, I did not found something to satisfy my needs.

If I do not miss something, I would like to open a pull request to provide some environment variables in jinja2 templates (page path and page URL in a first time). Before this I would like to know:

1. if the possibility to add a new dependency to provide this feature is too ambitious,
2. if it is, how could we provide this feature with current ansible-doctor dependencies ?,
3. if I'm misunderstanding ansible-doctor purpose. This is the second time I am requesting a new feature in one mounth and maybe ansible-doctor is not designed for huge projects like mine. My goal is not to change ansible-doctor to fit my needs. My goal is to add generic documentation for my ansible roles with less effort.

Thanks for reading.

I couldn't find a way to add line breaks to multi line comments.

# @var demo_role_other_tags:description: >
# If a variable need some more explanation, \n
# this is a good place to do so.
# Maybe this way <br>
# @end

Is it possible or is any newline character removed by default?

Hi,
I am trying to use ansible-doctor pre-commit hook but getting error as below:

aradhana.saxena@DEL1-LHP-N82532:~/pre-hook-testing/devops$ git commit -m "testung"
ansible-doctor...........................................................Failed
- hook id: ansible-doctor
- exit code: 1

[CRITICAL] No Ansible role detected

Steps followed:
I have added .pre-commit-config.yaml file to the root of my repository but when I am running the commit, I am getting above error.
Content of .pre-commit-config.yaml

aradhana.saxena@DEL1-LHP-N82532:~/pre-hook-testing/devops$ cat .pre-commit-config.yaml 
---
repos:
  - repo: https://github.com/thegeeklab/ansible-doctor
  # change ref to the latest release from https://github.com/thegeeklab/ansible-doctor/releases
    rev: v3.1.2
    hooks:
      - id: ansible-doctor

It seems ansible-doctor is unable to find the role. My directory structure is as follows:

.
├── inventory
│   ├── group_vars
│   │   ├── abx.yaml
│   │   └── abz.yaml
│   ├── host_vars
│   │   ├── abc.yaml
│   │   ├── xyz.yaml
│   ├── inventory.aws_ec2.yml
│   └── static_inventory.yml
├── playbooks
│   ├── deploy-configuration.yaml
│   ├── deploy-infra.yaml
│   └── roles
│       ├── basic_configuration
│       │   ├── meta
│       │   │   └── main.yml
│       │   ├── README.md
│       │   └── tasks
│       │       └── main.yml
│       ├── ec2
│       │   ├── meta
│       │   │   └── main.yml
│       │   ├── README.md
│       │   └── tasks
│       │       └── main.yml
│       ├── rds
│       │   ├── defaults
│       │   │   └── main.yaml
│       │   ├── meta
│       │   │   ├── argument_specs.yml
│       │   │   └── main.yml
│       │   ├── README.md
│       │   └── tasks
│       │       └── main.yml
│       ├── s3
│       │   ├── defaults
│       │   │   └── main.yaml
│       │   ├── meta
│       │   │   └── main.yml
│       │   ├── README.md
│       │   └── tasks
│       │       └── main.yml
│       ├── system_configuration
│       │   ├── defaults
│       │   │   └── main.yaml
│       │   ├── meta
│       │   │   └── main.yml
│       │   ├── README.md
│       │   └── tasks
│       │       └── main.yml
│       └── vpc_subnet
│           ├── defaults
│           │   └── main.yml
│           ├── meta
│           │   └── main.yml
│           ├── README.md
│           └── tasks
│               ├── alb.yml
│               └── main.yml
├── .pre-commit-config.yaml
├── README.md
└── requirements.txt

Please guide how to resolve this error.

I did some tests with this project and I will use it for #oravirt/ansible-oracle.
It looks really great for me.

There is 1 big wish:

• recursive search for .ansibledoctor.yml
  What about an option to search in roles for '.ansibledoctor.yml' and execute ansible-doctor in each directory with this file?
  That will allow 1 execution for all roles.
  Finally this makes a CI Action very simpel at the end. I like to test for missing executions of ansible-doctor for new commits in a PR.
• support for pre-commit
  If the previous functionality is existing, it is very simpel to add pre-commit for git.

Regards
Thorsten

I am opening this issue with the idea to fork and making a pull request on this repository to add a CLI feature. Ansible-doctor is really helpful to add a documentation for my roles and I know that ansible-doctor is dedicated to and designed for Ansible roles. But I would like to use it also on my playbooks or on a(/several) file(s) without creating a "tasks" directory, a "default" directory and putting each annotated file inside. It would be very useful if ansible-doctor with a -f or --files flag with any number of arguments after, could:

1. parse annotations in the files given in the command line (and not in "default" and "meta" directories),
2. disable the role detection by checking if "tasks" directory is in the role directory.

Do you think that I could implement and add this feature ? If yes, do you have any suggestion or advice to allow me to start in the right way ?

In addition to #624

I'm sorry to comment this issue, but it appear to be not helping.

I'm trying this following ansibledoctor.yml configuration:

---
logging:
  level: warning
force_overwrite: true
exclude_tags:
  - always
  - never

And never tag still appear in autodiscovery tags section.

And i think I'm not the only one.
I even found this roles README with a ansibledoctor.yml configuration with exclude_tags where the mentionned tag appear in README (always)
https://github.com/oravirt/ansible-oracle/tree/master/roles/oraswgi_install

I also tried with an environment variable without succes :

EXCLUDE_TAGS=["never"] ansible-doctor

Originally posted by @marcpaulchand in #286 (comment)

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

I get the following error, during installation of ansible-lint and ansible-doctor in same python environment:

The conflict is caused by:
    ansible-doctor 2.0.3 depends on ruamel.yaml==0.17.21
    ansible-lint 6.16.0 depends on ruamel.yaml<0.18 and >=0.17.24

Does ansible-doctor really need this hard dependency?

Hello,
First of all, I would like to thank you for this tool, it greatly helps me tame Ansible. I would like to have such a table of contents in README.md, but generate the output into another file rather than README.md. However, I haven't seen any option to configure this. Would it be possible to make this configurable?

I'm trying to use ansible-doctor with an Ansible collection repository and I would like to define the configuration once in the root directory (i.e., as a sibling to the roles directory) and have it recurse over all roles inside. For example, the structure looks like so

.:
roles/
.ansibledoctor.yml

./roles:
roleA/
roleB/

./roles/roleA:
tasks/

./roles/roleA/tasks:
main.yml

./roles/roleB:
tasks/

./roles/roleB/tasks:
main.yml

In the MWE, the .ansibledoctor.yml contains

---
base_dir: roles

However I can't get ansible-doctor to use it...

1. It finds the config but not the roles when called without arguments (the automatic role detection doesn't traverse into the roles directory).

   $ ansible-doctor -v
   [INFO] Using config file: C:\Users\hauserv\git\test\.ansibledoctor.yml
   [CRITICAL] No Ansible role detected
   

2. It finds neither when called with --recursive (it either ignores the folder-based config altogether or perhaps traverses into the roles directory, fails to find a .ansibledoctor.yml and uses a global config)

   $ ansible-doctor -vr
   [INFO] Using config file: C:\Users\hauserv\AppData\Local\ansible-doctor\ansible-doctor\config.yml
   [CRITICAL] No Ansible role detected
   

3. It finds the roles but not the config when both base_dir is passed in as a positional argument in addition to the --config (it claims to use config files such as roles/roleA/.ansibledoctor.yml which don't exist).

   $ ansible-doctor -vr -c .ansibledoctor.yml roles
   [INFO] Using config file: C:\Users\hauserv\git\test\roles\roleA\.ansibledoctor.yml
   [INFO] Ansible role detected: roleA
   [INFO] Finding annotations for: @meta
   [INFO] Finding annotations for: @todo
   [INFO] Finding annotations for: @var
   [INFO] Finding annotations for: @example
   [INFO] Finding annotations for: @tag
   [INFO] Using template dir: C:\Python39\Lib\site-packages\ansibledoctor\templates\readme
   [INFO] Using output dir: C:\Users\hauserv\git\test\roles\roleA
   [INFO] Writing to: C:\Users\hauserv\git\test\roles\roleA\README.md
   [INFO] Using config file: C:\Users\hauserv\git\test\roles\roleB\.ansibledoctor.yml
   [INFO] Ansible role detected: roleB
   [INFO] Finding annotations for: @meta
   [INFO] Finding annotations for: @todo
   [INFO] Finding annotations for: @var
   [INFO] Finding annotations for: @example
   [INFO] Finding annotations for: @tag
   [INFO] Using template dir: C:\Python39\Lib\site-packages\ansibledoctor\templates\readme
   [INFO] Using output dir: C:\Users\hauserv\git\test\roles\roleB
   [INFO] Writing to: C:\Users\hauserv\git\test\roles\roleB\README.md
   

Hi,

wanted to use this tool, but it's not compatible with my dev environment.
The ruamel yaml module deprecated some functions in v0.18.0, see https://pypi.org/project/ruamel.yaml/0.18.0.

Hi,

I think there is a bug when using a custom template file. We have the error
[CRITICAL] Can not open template dir ${HOME}/ansible-doctor/README.md.j2

Config is passed through env

export ANSIBLE_DOCTOR_TEMPLATE_DIR=${HOME}/ansible-doctor
export ANSIBLE_DOCTOR_TEMPLATE=README.md.j2

In the Generator._scan_template function there is a test for the custom template directory. But this test is using the full path of the template file. So it fails everytime.

The config.get_template function used to set the template_dir var in the doc_generator.py file is returning the full path.

You may consider using os.path.dirname(template_dir) on L40 to L43 instead of template_dir.

I'm using version 2.1.1.

Thank you for your work

edit: add ansible-doctor version

The use case :
one folder with multi playbooks and roles/requirements.

.
├── roles
│   └── requirements.yml
└── playbook_1.yml
└── playbook_2.yml

The expected result :

one document README.md or one document per playbook README_playbookname.md and the requirements listed in section "Dependencies"

If this use case is outside the scope of this project, then I will probably fork it .

There is no template for tags in the template dir.
Because of that, tags wont be added to the generated markdown file.
Same for Todos.
Moreover, teh README.j2 template does not include a tags nor todos sections.

I have experienced contributing on my own and faced some difficulties to onboard to non-learned sections.

Suggestions to help generation and templating:

• Add commands to run to generate and test readme template
• Add commands to run to generate and test hugo template

Suggestions to help to test:

• Add unit test (pytest + lint) to ensure generation is well formatted

# Traefik basic auth credentials. Default `mailhog:mailhog`. If you need to create secure credentials
# see https://docs.traefik.io/middlewares/basicauth/#users for more details.

Will be result in https://docs.traefik.io/middlewares/basicauth/users

i would try to create a custom template to get requirement roles documented in the readme.md aswell,
ansible galaxy works using a requirements.yml in the meta folder of a role with the following syntax:

- src: [email protected]:package-roles/package.git
  version: main
- src: [email protected]:package-roles/firewall.git
  version: main
- src: [email protected]:package-roles/php.git
  version: main

any idea how one could archive that?

another question revolves around dictionary variables, and how to properly get doctor to detect them and document them.
currently its just this:

# @var apache2_listen_ports:type: dictionary
# @var apache2_listen_ports:description: >
# if the https port is omitted, the default vhost will only listen on http
# otherwise a redirect is included in the vhost that redirects http requests to https
# @end
apache2_listen_ports:
  http: 80
  https: 443

and doctor itself would also only detect the initial dict var, is there a way to have it split them out, and detect and document the keys in the dict as vars aswell?

In ansible-doctor version 2.0.3 the feature "extend var options with type and deprecated (#462)" has been introduced. Unfortunately the default "readme" template hasn't been extended as well.
In order to use the new options it seems to be necessary to create an own template which is a kind of unfortunate.

If meta descriptions contain code blocks the contained code gets modified: Indentation (done with spaces) is removed and quotes (single and double quotes) are converted to HTML code (see below example). This makes the generated content hard to read and understand.

Comment

# @meta description: >
# This is a test description with an usage example. There's an interesting result.
# 
# ```yaml
# - name: My to be included role
#   ansible.builtin.include_role:
#     name: namespace.collection.role_name
#   vars:
#     first_var: "test-value"
#     second_var: '1234567890'
# ```
# @end

Result

This is a test description with an usage example. There's an interesting result.

```yaml
- name: My to be included role
ansible.builtin.include_role:
name: namespace.collection.role_name
vars:
first_var: "test-value"
second_var: '1234567890'
```