To modify Azure tests and depend on shell scripts and pytest instead
of molecule, the run-tests.sh script has been rewritten to depend on
bash functions and on a bash script that prepare and start a testing
container.
This patch adds a new script, 'utils/setup_test_container.sh' that
can be used to start a new container, using either podman or docker,
based on the available ansible-freeipa images. The new container can
then be used to run ansible-freeipa tests against it.
Also the following files with bash functions were added, and are
used by both scripts:
utils/shansible: Functions to run playbooks in the container
utils/shcontainer: Functions to setup/run a container
utils/shfun: Generic shell helper functions (e.g.: log)
When running ansible-freeipa's Azure pipelines for nightly and weekly
tests, due to the amount of tests to execute, tests are grouped and
executed in parallel jobs.
Due to a still unkonwn issue, depending on the order the tests are
executed, some random failures may occur and debugging them is hard due
to current implementation of the tests.
This patch adds support for replicating the tests of a specific Azure
test group once the seed used to create groups and the group number are
provided, allowing the test failures to be replicated on the developer's
workstation where it can be more easily debugged.
A new option is added to 'utils/run-tests.sh', '-A SEED.G' that is used
to define the seed and group to replicate the tests. The seed is a date,
with the format "YYYYMMDD", so, for example '-A 20230611.2' would
execute the same tests, in the same order as the second group of tests
for date 2023-06-11. To aid in usability 'YYYY-MM-DD' may also be used.
When using '-A' neither '-s' (test suites) or specific tests (positional
arguments) can be used.
Also, to help fixing tests, an option to stop the tests on the first
test failure ('-x') was added to the script.
When runing tests using 'utils/run-tests.sh' from inside an existing
Python virtual environment the Ansible collections are not installed due
to the order of execution of the script. On a machine that does not have
the 'containers.*' collection the test fails as there is no container
connector available.
This patch moves the section that installs Ansible collections to run
after the virtual environment is configured, and then install the
collections (usually, only 'containers.podman'), allowing the tests to
be executed.
ansible-core 2.15 has been released on May 15th, 2023, and version 2.12
has reached EOL on May 22nd, 2023.
This patch updates the ansible-core versions used on upstream CI tests
to reflect Ansible's new releases.
