saltstack / pytest-salt-factories Goto Github PK

View Code? Open in Web Editor NEW

19.0 19.0 22.0 2.07 MB

PyTest Salt Factories Plugin

Home Page: https://pytest-salt-factories.readthedocs.io/

License: Apache License 2.0

Python 100.00%

pytest pytest-plugin salt saltstack testing

pytest-salt-factories's People

Contributors

Stargazers

Watchers

pytest-salt-factories's Issues

The Pytest log handler for Salt throws ``KeyError``

Traceback (most recent call last):                                                                                                                                                                                  
  File "/testing/salt/loader/lazy.py", line 1106, in _process_virtual                                                                                                                                               
    virtual = self.run(virtual_attr)                                                                                                                                                                                
  File "/testing/salt/loader/lazy.py", line 1201, in run                                                  
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)                                                                                                                                   
  File "/testing/.nox/pytest-parametrized-3-crypto-none-transport-zeromq-coverage-false/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run                                                       
    return callable(*args, **kwargs)                                                                                                                                                                                
  File "/testing/salt/loader/lazy.py", line 1216, in _run_as                                                                                                                                                        
    return _func_or_method(*args, **kwargs)                                                                                                                                                                         
  File "/testing/.nox/pytest-parametrized-3-crypto-none-transport-zeromq-coverage-false/lib/python3.6/site-packages/saltfactories/utils/saltext/log_handlers/pytest_log_handler.py", line 50, in __virtual__        
    pytest_config = __opts__[pytest_key]                                                                                                                                                                            
KeyError: 'pytest-master'                                                                                                                                                                                           
[ERROR   ] Exception raised when processing __virtual__ function for salt.loaded.ext.log_handlers.pytest_log_handler. Module will not be loaded: 'pytest-master'

On system install's the minion's end up with the ID of the master instead of their own ID

The problem is here

Redirect the old factory exceptions.

For backwards compatibility, salt-factories should redirect to the old factory related exceptions, now in pytest-shell-utilities.

Also, show a deprecation warning.

Replace the notion of ``system_install`` by ``system_service``

This is to make it more explicit on what the variable does.

Also, the variable should be not be set on the salt factories manager, it should be set on the daemons.

Container tests not passing on macOS

Fix handling passing ``--timeout`` to Sat's CLI's

        # Handle the timeout CLI flag, if supported                                                       
        if self.__cli_timeout_supported__:                                                                                                                                                                          
            salt_cli_timeout_next = False                                                                                                                                                                           
            for arg in args:                                                                                                                                                                                        
                if arg.startswith("--timeout="):                                                                                                                                                                    
                    # Let's actually change the _terminal_timeout value which is used to                                                                                                                            
                    # calculate when the run() method should actually timeout                                                                                                                                       
                    try:                                                                                                                                                                                            
                        salt_cli_timeout = int(arg.split("--timeout=")[-1])                                                                                                                                         
                    except ValueError:                                                                                                                                                                              
                        # Not a number? Let salt do it's error handling                                                                                                                                             
                        break                                                                                                                                                                                       
>                   if salt_cli_timeout >= self.impl._terminal_timeout:                                                                                                                                             
E                   TypeError: '>=' not supported between instances of 'int' and 'NoneType'

Add codeowners file

repo needs a codeowners file added
https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-on-github/about-code-owners#example-of-a-codeowners-file

The container daemon should do a pre-start pull

Prior to starting the container daemon, a pull of the image should be attempted.

add security MD file

repository needs a security.md file added for compliance and project health

The containers factory does not accept the ``stats_processes`` keyword

Allow defining which ``python_executable`` to use, where needed

add issue/PR templates

repository can reuse the templates from Salt or others

The functional `Loaders` class should clean up the cachedir on each reset.

Handle docker client initialization error on macOS

 /Users/runner/work/pytest-salt-factories/pytest-salt-factories/src/saltfactories/manager.py:622: in get_container
    **factory_class_kwargs
<attrs generated init saltfactories.daemons.container.Container>:30: in __init__
    self.docker_client = __attr_factory_docker_client(self)
/Users/runner/work/pytest-salt-factories/pytest-salt-factories/src/saltfactories/daemons/container.py:130: in _default_docker_client
    docker_client = docker.from_env()
/Users/runner/work/pytest-salt-factories/pytest-salt-factories/.nox/tests-3-7/lib/python3.7/site-packages/docker/client.py:101: in from_env
    **kwargs_from_env(**kwargs)
/Users/runner/work/pytest-salt-factories/pytest-salt-factories/.nox/tests-3-7/lib/python3.7/site-packages/docker/client.py:45: in __init__
    self.api = APIClient(*args, **kwargs)
/Users/runner/work/pytest-salt-factories/pytest-salt-factories/.nox/tests-3-7/lib/python3.7/site-packages/docker/api/client.py:197: in __init__
    self._version = self._retrieve_server_version()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x10c1b1910>

    def _retrieve_server_version(self):
        try:
            return self.version(api_version=False)["ApiVersion"]
        except KeyError:
            raise DockerException(
                'Invalid response from docker daemon: key "ApiVersion"'
                ' is missing.'
            )
        except Exception as e:
            raise DockerException(
>               f'Error while fetching server API version: {e}'
            )
E           docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

Remove the need to pass a `docker_client` instance when using containers

The `salt_factories_config` fixture cannot return a dictionary with `root_dir`

To configure the salt-factories manager, a salt_factories_config fixture is used to customize some of the settings, however, since root_dir is passed explicitly, we cannot override that setting.

Set lower required python version to 3.5.2

This still allows installing on older distros

src/ based layout breaks pytest

During Debian package build, the pytest test cases can easily be executed by running this command in the root directory:

python3 -m pytest -ra tests/functional tests/unit

Commit 1daf13f switches the project to an src/ based layout and breaks this way of running pytest. Specifying --rootdir does not help.

Missing salt-cloud CLI support

Missing salt-api daemon support

release cadence published

when do we expect 1.0.0 version to be released and what will be the release cadence? this can be simple, but we should define it and publish docs as such

Allow switching to system service behavior through a CLI flag

Salt is not a hard dependency

Salt isn't a hard dependency of pytest-salt-factories, which means if you want to use some of it's markers in a non-salt project everything fails here:

  File "/usr/local/lib/python3.6/site-packages/saltfactories/plugins/factories.py", line 16, in <module>
    import salt.config
ModuleNotFoundError: No module named 'salt'

in version 0.13.0 of pytest-salt-factories there was a try/except around these salt imports, maybe we can bring those back?

Source distribution repack removes most of the original sdist contents

Allow configuring the ``python_executable`` that should be used

Remove deprecated code

saltfactories.utils.ports
saltfactories.utils.processes
Deprecation warnings and import redirects on saltfactories.exceptions

[Security] Workflow testing.yml is using vulnerable action pre-commit/action

The workflow testing.yml is referencing action pre-commit/action using references v2.0.0. However this reference is missing the commit 80db042ff08cdddbbbfb6f89c06a6bfc4dddf0b7 which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.

Newer version of black broke pre-commit

Deprecate instead of remove the ``ports`` and ``processes`` utils modules

Instead of just removing saltfactories.utils.ports and saltfactories.utils.processes, redirect the imports to the right library and show a deprecation warning.

Default timeout not used

When there's a timeout key defined in the configuration, or when timeout is passed to the CLI factory contructor, namely for CLI scripts, the --timeout flag is not added to the generated cmdline.

The spm CLI is not properly configured by default

Allow configuring the ``scripts_path`` setting

Inject into the ``engines_dirs`` and ``log_handlers_dirs`` in the config when not running against a locally cloned salt

Basically, revert 52050f6 and add some logic of when it should be injected.

Log messages from SaltDaemon in Docker container are missing

When running an integration test that uses saltfactories.daemons.container.SaltDaemon, the Python log messages emitted by the daemon do not show up in the test log. IIUC, this is because Python does not see the entry_points.txt for pytest-salt-factories so it doesn't know that it should load saltfactories.utils.saltext.log_handlers.pytest_log_handler.

One solution (that I tested successfully) is to pip install pytest-salt-factories when building the Docker image or after starting the Docker container. This assumes that pip is installed in the image, pip runs with sufficient privileges to install the package, and the version of pytest-salt-factories installed in the container is compatible with the version loaded by nox for the test.

Another solution (untested) might be to bind mount CODE_ROOT_DIR.parent instead of CODE_ROOT_DIR and add it to Python's sys.path when launching the daemon. This assumes that all of the packages in that directory are compatible with the container, which might be running a different version of Python (or even a different machine architecture).

Maybe another solution (untested) would be to bind mount CODE_ROOT_DIR.parent / "pytest_salt_factories-1.0.0rc20.dist-info" (where the specific .dist-info dir is discovered programmatically) in addition to CODE_ROOT_DIR, and add CODE_ROOT_DIR.parent to sys.path. This assumes that the pytest-salt-factories distribution is binary compatible with the container, which seems likely.

Perhaps the best solution would be to create a virtualenv inside the container and pip install all daemon dependencies plus pytest-salt-factories before starting the daemon.

Additional context: saltstack/salt#62791 (comment)

session-markers-minion overriding opts for func-tests-minion

If you run the test_refresh_db test individually on the head of master of Salt you will get a failure:

nox -e 'pytest-zeromq-3(coverage=True)' -- tests/pytests/functional/modules/test_pkg.py::test_refresh_db --run-slow --run-destructive

    @pytest.mark.destructive_test                                                                                                                                                                                                                             
    @pytest.mark.requires_salt_modules("pkg.refresh_db")                                                                                                                                                                                                      
    @pytest.mark.slow_test                                                                                                                                                                                                                                    
    @pytest.mark.requires_network                                                                                                                                                                                                                             
    def test_refresh_db(grains, modules, tmp_path, minion_opts):                                                                                                                                                                                              
        """                                                                                                                                                                                                                                                   
        test refreshing the package database                                                                                                                                                                                                                  
        """                                                                                                                                                                                                                                                   
        rtag = salt.utils.pkg.rtag(minion_opts)                                                                                                                                                                                                               
        salt.utils.pkg.write_rtag(minion_opts)                                                                                                                                                                                                                
        assert os.path.isfile(rtag) is True                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                              
        ret = modules.pkg.refresh_db()                                                                                                                                                                                                                        
        if not isinstance(ret, dict):                                                                                                                                                                                                                         
            pytest.skip("Upstream repo did not return coherent results: {}".format(ret))                                                                                                                                                                      
                                                                                                                                                                                                                                                              
        if grains["os_family"] == "RedHat":                                                                                                                                                                                                                   
            assert ret in (True, None)                                                                                                                                                                                                                        
        elif grains["os_family"] == "Suse":                                                                                                                                                                                                                   
            if not isinstance(ret, dict):                                                                                                                                                                                                                     
                pytest.skip("Upstream repo did not return coherent results. Skipping test.")                                                                                                                                                                  
            assert ret != {}                                                                                                                                                                                                                                  
            for source, state in ret.items():
                assert state in (True, False, None)
     
>       assert os.path.isfile(rtag) is False
E       AssertionError: assert True is False
E        +  where True = <function isfile at 0x7fabf6c88280>('/tmp/stsuite/func-tests-minion/cache/pkg_refresh')
E        +    where <function isfile at 0x7fabf6c88280> = <module 'posixpath' from '/home/ch3ll/.pyenv/versions/3.8.6/lib/python3.8/posixpath.py'>.isfile
E        +      where <module 'posixpath' from '/home/ch3ll/.pyenv/versions/3.8.6/lib/python3.8/posixpath.py'> = os.path

The reason for this failure is __opts__ changes here: https://github.com/saltstack/salt/blob/master/salt/modules/aptpkg.py#L524

from /tmp/stsuite/func-tests-minion/cache/pkg_refresh to /tmp/stsuite/session-markers-minion/cache/pkg_refresh.

If found out the session-markers-minion is getting set here: https://github.com/saltstack/pytest-salt-factories/blob/master/src/saltfactories/plugins/markers.py#L48

If I remove the refresh_db call here: https://github.com/saltstack/salt/blob/master/tests/pytests/functional/modules/test_pkg.py#L42 it begins to pass again.

@s0undt3ch any ideas why this is occurring? I'm not certain its an issue with pytest-salt-factories or the test suite in Salt.

I'll keep diving into it, but would appreciate any early insight you might have.

The ``SaltDaemon`` Container daemon base classes order is wrong

It's run() method runs the daemon's run() method instead of the container's run() method.

Drop Support for Py 3.5 and Py 3.6

The main code base will drop support for Py3.5 and Py3.6, but salt-factories will still provide a "downgraded" source for platforms that still use those python versions

Bump deprecations targetted to 2.0.0 to 3.0.0

Adjust the stacklevel of warn_until

When pytest is rewriting, any warn_until calls will report the calling code as pytest's rewrite module.
We need to adjust the stack level in this case to report the offending module properly

`code_dir` must always be passed for coverege support

Allow tracking coverage without having to pass code_dir

Support user provided callables to confirm that the daemon is up and running.

Make use of the changes made in saltstack/pytest-shell-utilities#20

Use salt's entry-points instead of relying on loader ``*_dirs`` configs

Fix ``SaltKeyOptionParser`` new logging detection

Now that the new logging changes are merged into Salt's master branch, adjust detection to those changes.

Adjust to the loaders changes for the Silicon release of Salt

Catch exceptions when removing containers

failed on teardown with "docker.errors.APIError: 409 Client Error: Conflict ("removal of container 2b3989f0b4191fa143f35943f8f9c6976e40cf9a0cf887a23b89334eb9b34904 is already in progress")"

Remove salt dependency

Installing salt-factories currently requires salt as a dependency.

Beside this being a chicken and egg issue with salt, though, not a problem to run salt's test suite, when trying to run other test suite's, salt-pkg for example, dependency management becomes a nightmare, mostly on windows, because salt defines the dependencies to use at install time, which is really not good when trying to ping dependencies down.

Try to not depend, at all, on salt.

saltstack / pytest-salt-factories Goto Github PK

pytest-salt-factories's People

Contributors

Stargazers

Watchers

Forkers

pytest-salt-factories's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs