sivel / ansible-testing Goto Github PK

Module name should not conflict with other task arguments (become for example)

Detect new arguments for modules, and validate that they have the correct version_added.

New modules should import what they need, no longer importing * from module_utils.

In the documentation section of my module, I have this part:

  type:
    description:
      - the type of certificate to generate: RSA or ECDSA. If type is ecdsa, you must specify a curve to use.
    required: false
    default: "rsa"
    choices: [ "rsa", "ecdsa" ]

This throws an error:

 ERROR: DOCUMENTATION.options.type.description.0: invalid list value

Removing the colon got rid of the error.

New Network modules, such as ansible/ansible-modules-core#4065 don't require ansible.module_utils.basic

Instead they get access to this module via a OS specific Network module, such as:

from ansible.module_utils.eos import *
or
from ansible.module_utils.ios import *```

To unblock the development work by @privateip, we have made this a warning see #31

A full fix is still required which is to only skip the test for ansible.module_utils.basic for networking modules. This could be done by looking to see if the file is under ./network/

I'm not sure there is anything we can do here, but the line number you get from a YAML parsing error is the line number within the YAML document, not the module.

Maybe there is a way we can offset that line number somehow? This may require not displaying the traceback, but using the data in the traceback to construct our own error.

With ZipModule, we should not require that json come from basic.py, instead we should just ensure that json is imported within a try/except with a fallback to simplejson.

We detect version added and compare to current devel. However it is too dumb. It will assume that all modules must have a version_added for the next release, not just new modules. We'll have to find out how to make that determination better.

When documentation is invalid, still properly report on EXAMPLES and RETURN.

Currently if documentation is invalid, EXAMPLES and RETURN are listed as missing.

It could be useful to assign pep8 like codes to errors/warnings/etc.

This could also give us the ability to filter out certain codes as well.

Ensure that the call to main looks like:

if __name__ == '__main__':
    main()

With the addition of the ZipModule code for ansible 2.1, we no longer require module_utils to be located at the bottom.

Drop this check.

Look for subprocess or os.call, etc... and error that module.run_command should be used

It was discussed that any new module referencing boto2 should be treated as a fail

Agreed in the New Modules meeting on 6th April 2016

Need to decide what to do if any existing modules are still using boto2

import json is not required nor recommended. ansible.module_utils.basic already provides json, and does so by also attempting to import simplejson.

Currently if new args are added, they need to have a version_added indicating the upcoming version, however of the module also targets that new version we shouldn't error.

Getting the following stack track:

Traceback (most recent call last):
  File "/usr/local/bin/ansible-validate-modules", line 9, in <module>
    load_entry_point('ansible-testing==0.0.1b', 'console_scripts', 'ansible-validate-modules')()
  File "/Library/Python/2.7/site-packages/ansible_testing/modules.py", line 513, in main
    mv.validate()
  File "/Library/Python/2.7/site-packages/ansible_testing/modules.py", line 398, in validate
    self._check_version_added(doc)
  File "/Library/Python/2.7/site-packages/ansible_testing/modules.py", line 341, in _check_version_added
    'number: %s' % version_added)
UnboundLocalError: local variable 'version_added' referenced before assignment

Detect GPL license header in module files

As of now we are adding a warning if things like json or re are imported in the module and in basic.py. Due to changes coming to the modules, the duplicate import is probably ok. We should ignore imports when scanning basic.py.

Look for sys.exit in modules

Right now we cannot effectively analyze the argument_spec as we need to run the code to do so.

See if we can somehow mock AnsibleModule to allow us to run the module and inspect argument_spec

This can also help us better detect when new arguments are added. As of now, if we are updating documentation to add missing arguments, it will complain that we need to set version_added to the next release, which isn't correct.

We should recommend ansible.module_utils.urls instead of using requests at this time

Check the shebang to ensure that it is #!/usr/bin/python

Modules without an extension get stopped early, and don't get additional feedback. We should stills can them.

Ensure invalid YAML in RETURN is properly indicated.

There is some bad logic in there somewhere. I haven't looked at it in depth. However I am currently running with uncommited code that looks like the following:

diff --git a/ansible_testing/modules.py b/ansible_testing/modules.py
index f0acaa3..10d135d 100644
--- a/ansible_testing/modules.py
+++ b/ansible_testing/modules.py
@@ -100,6 +100,7 @@ class ModuleValidator(Validator):
         'ansible.module_utils.facts',
         'ansible.module_utils.splitter',
         'ansible.module_utils.known_hosts',
+        'ansible.module_utils.rax',
     ))
     BOTTOM_IMPORTS_BLACKLIST = frozenset((
         'command.py',
@@ -203,11 +204,11 @@ class ModuleValidator(Validator):
                     if child.module in self.BOTTOM_IMPORTS:
                         if (child.lineno < main - 10 and
                                 not self._is_bottom_import_blacklisted()):
-                            self.errors.append('%s import not near main()' %
-                                               child.module)
-                    else:
-                        self.warnings.append('%s import not near main()' %
-                                             child.module)
+                            self.errors.append('%s import not near call to '
+                                               'main()' % child.module)
+                    #else:
+                    #    self.warnings.append('%s import not near main()' %
+                    #                         child.module)

                     linenos.append(child.lineno)

I haven't really taken the time to validate those changes from above.