internet/ansible-freeipa
4
0
Fork 0
mirror of https://github.com/freeipa/ansible-freeipa.git synced 2026-03-26 21:33:05 +00:00
Code Issues Projects Releases Wiki Activity

Add support to define which playbook tests to execute with pytest.

Browse Source 
pytest provide the means to skip tests based on patterns, but writing
these patterns for ansible-freeipa might not be feasible.

This PR allows the selection of playbook tests and modules that will
be executed with pytest using the environmentt variables IPA_ENABLED_TESTS
IPA_ENABLED_MODULES, IPA_DISABLED_TESTS or IPA_DISABLED_MODULES.

When using IPA_ENABLED_MODULES, all modules will be disabled, and only
the modules in the enabled list will be tested. If using the test
filter, IPA_ENABLED_TESTS, all tests are disabled, unless they are in
the enabled test lists.

If the IPA_DISABLED_* version is used, tests and modules are enabled by
default, and the list is used to disable the module or specific test.

To disable a test or module in Azure CI, edit the file
`tests/azure/variables` and add the desired tests or modules to the
parameter variables `enabled_modules`, 'enabled_tests`, `disabled_tests`
or `disable_modules`.

Note that, if added to the `master` branch, this will affect the tests
for every pipeline that it is include (including 'nightly'), so it should
be used with care.

It can be used with TEMP commits to enable only the desired tests,
speeding up upstream tests.
This commit is contained in:
Rafael Guterres Jeffman
2020-08-17 20:58:54 -03:00
parent 4ff5aaa172
commit 13cff6354b
6 changed files with 123 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
tests/README.md
View File

@@ -63,6 +63,24 @@ IPA_SERVER_HOST=<ipaserver_host_or_ip> pytest -rs
For a complete list of options check `pytest --help`.
### Disabling and enabling playbook tests
Sometimes it is useful to enable or disable specific playbook tests. To only run a subset of modules or tests, use the variables IPA_ENABLED_MODULES and IPA ENABLED_TESTS, to define a comma-separated list of modules or tests to be enabled. Any test or module not in the list will not be executed. For example, to run only `sudorule` and `sudocmd` tests:
```
IPA_ENABLE_MODULES="sudorule,sudocmd" IPA_SERVER_HOST=<ipaserver_host_or_ip> pytest
```
If all but a few selected tests are to be executed, use the IPA_DISABLED_MODULES or IPA_DISABLED_TESTS. For example, to run all, but "test_service_certificate" test:
```
IPA_DISABLED_TESTS=test_service_certificate IPA_SERVER_HOST=<ipaserver_host_or_ip> pytest
```
If none of this variables are defined, all tests will be executed.
To configure the tests that will run for your pull request, add a TEMP commit, with the configuration defined in the file `tests/azure/templates/variables.yml`. Set the variables `ipa_enable_modules`, `ipa_enable_tests`, `ipa_disable_modules`, and `ipa_disable_tests`, in the same way as the equivalent environment variables.
### Types of tests
#### Playbook tests
@@ -119,6 +137,7 @@ molecule destroy -s c8s
See [Running the tests](#running-the-tests) section for more information on available options.
## Upcoming/desired improvements:
* A script to pre-config the complete test environment using virsh.