The script utils/build-galaxy-release.sh has been renamed to
utils/build-collection.sh, the script provides the same options, but
requires an extra argument now:
build-collection.sh [options] rpm|aah|galaxy
The namespace and name are defined according to the argument:
rpm freeipa.ansible_freeipa - General use and RPMs
galaxy freeipa.ansible_freeipa - Ansible Galaxy
aah redhat.rhel_idm - Ansible AutomationHub
The generated file README-COLLECTION.md is set in galaxy.yml as the
documentation entry point for the collections generated with aah and galaxy
as Ansible AutomationHub and also Ansible Galaxy are not able to render the
documentation README files in the collection properly.
The commit also changes the calls of utils/build-galaxy-release.sh to
utils/build-collection.sh.
This change reduces the size of the Ansible collection and removes unused
files.
While creating the collection, the role plugins have been linked into the
global plugin folders, but a subsequent sed call replaced the links with
changed files. The original files have also been changed with another sed
call in the same way. This resulted in the duplication of the changed
files.
The plugins of the roles are now moved into the global plugin folders and
only changed there. The now empty plugin folders in the roles are
removed.
Modify the plugin templates so that the code generated does not hide
errors when querying IPA wih *_show command by handlig only the exeption
where an object is not found.
Pydocstyle has been deprecated is no longer in development. It is also
not a requirement for Ansible tests.
This patch removes pydocstyle from current checks performed.
All scripts related to the Azure CI now reside on inrfa/azure, but the
scripts that evaluate the changes made against ansible-freeipa's main
development branch.
This patch move these scripts to the proper locations.
For some time now, we had some issues with molecule when building test
images for ansible-freeipa, and replaced the image creation with custom
build scripts that use commom container tools (like Dockerfiles and the
build command).
As there's no more tasks that require the use of molecule, this patch
removes the last bits used by it, and fixes documentation and lint
scripts and configuration.
To force setting the IPA_ENABLE_* variables to run all tests, source the
script using '-I' or set the environment variable SKIP_GIT_TESTS to
'True'.
This will allow the correct selection of Azure pipelines tests to be
based on a single environment variable, what will reduce the number of
test running templates to a singe file.
To modify Azure tests and depend on shell scripts and pytest instead
of molecule, the run-tests.sh script has been rewritten to depend on
bash functions and on a bash script that prepare and start a testing
container.
This patch adds a new script, 'utils/setup_test_container.sh' that
can be used to start a new container, using either podman or docker,
based on the available ansible-freeipa images. The new container can
then be used to run ansible-freeipa tests against it.
Also the following files with bash functions were added, and are
used by both scripts:
utils/shansible: Functions to run playbooks in the container
utils/shcontainer: Functions to setup/run a container
utils/shfun: Generic shell helper functions (e.g.: log)
Add two shell libaries with utilities to write shell scripts.
The 'utils/shlog' file provides macro names for ANSI escape sequences to
control color output on terminals, a 'log' functions with pre-defined
behavior for ERROR, WARN, DEBUG, INFO and SUCCESS level messages, and
the 'quiet' function which executes a command and hides its output.
The 'utils/shfun' file provides an interruptinon handler for SIGINT, and
the following functions:
- run_if_exists: run a command if it is available
- cleanup: cleanup environment, possibly stopping a container and a
Python virtual environment.
- start_virtual_environmnt: initiates a Python virtual environment
- in_python_virtualenv: test if the script is running inside a Python
virtual environment
- die: abort the script with an error message end exit code 1
New files:
- utils/shlog
- utils/shfun
ansible-freeipa roles do not work with Ansible 2.8 anymore, so the
minimum supported version is changed to 2.15, the oldest supported
Ansible version as of today.
This patch updates the minimum supported Ansible version and the list
of available modules.
ansible-freeipa roles do not work with Ansible 2.8 anymore, so the
minimum supported version is changed to 2.15, the oldest supported
Ansible version as of today.
Two new options have been added to enable the offline build within rpm:
-o <A.B.C> Build offline without using git, using version A.B.C
Also enables -a
-p <path> Installation the generated collection in the path, the
ansible_collections sub directory will be created and will
contain the collection: ansible_collections/<namespace>/<name>
Also enables -i
The usage text has been fixed also for specifying namespace and name.
The collection variable has been renamed to name.
Example usage:
utils/build-galaxy-release.sh -o 1.12.1 \
-p %{buildroot}%{_datadir}/ansible/collections \
freeipa ansible_freeipa
When running ansible-freeipa's Azure pipelines for nightly and weekly
tests, due to the amount of tests to execute, tests are grouped and
executed in parallel jobs.
Due to a still unkonwn issue, depending on the order the tests are
executed, some random failures may occur and debugging them is hard due
to current implementation of the tests.
This patch adds support for replicating the tests of a specific Azure
test group once the seed used to create groups and the group number are
provided, allowing the test failures to be replicated on the developer's
workstation where it can be more easily debugged.
A new option is added to 'utils/run-tests.sh', '-A SEED.G' that is used
to define the seed and group to replicate the tests. The seed is a date,
with the format "YYYYMMDD", so, for example '-A 20230611.2' would
execute the same tests, in the same order as the second group of tests
for date 2023-06-11. To aid in usability 'YYYY-MM-DD' may also be used.
When using '-A' neither '-s' (test suites) or specific tests (positional
arguments) can be used.
Also, to help fixing tests, an option to stop the tests on the first
test failure ('-x') was added to the script.
Due to an error on processing Ansible key 'import_tasks' the script that
creates a list of modules to test is broken making some modules to be
not tested.
By fixing the handling of 'import_tasks' and module import, the list is
correct again and the list of modules to be tested now include the ones
which depend on the modified module.
There is a new idp management module placed in the plugins folder:
plugins/modules/ipaidp.py
The idp module allows to ensure presence or absence of external Identity
Providers.
Here is the documentation for the module:
README-idp.md
New idp example playbooks:
playbooks/idp/idp-present.yml
playbooks/idp/idp-absent.yml
New tests for the module:
tests/idp/test_idp.yml
tests/idp/test_idp_client_context.yml
There is a new idoverrideuser management module placed in the plugins
folder:
plugins/modules/ipaidoverrideuser.py
The idoverrideuser module allows to ensure presence and absence of
idoverrides for users and certificate members.
Here is the documentation for the module:
README-idoverrideuser.md
New example playbooks have been added:
playbooks/idoverrideuser/idoverrideuser-absent.yml
playbooks/idoverrideuser/idoverrideuser-certificate-absent.yml
playbooks/idoverrideuser/idoverrideuser-certificate-present.yml
playbooks/idoverrideuser/idoverrideuser-present.yml
New tests for the module can be found at:
tests/idoverrideuser/test_idoverrideuser.yml
tests/idoverrideuser/test_idoverrideuser_client_context.yml
Recent pylint versions warn against the use of an 'else' in a
'try-except' block if using a 'return' on the 'except' part is is the
idom used by ansible-freeipa when retrieving IPA data objects.
This change removes the usage of the 'else:' in such cases, and modify
the templates so that new modules do not have the same issue in the
future.
As ansible-freeipa roles do not support version 2.8 anymore, change the
minimum supported version to 2.13, which is the currently minimum
available and supported Ansible version.
This patch fixes documentation on all plugin READMEs, spec file and
module templates.
Currently, there is a hard coded timeout in galaxy-importer that
prevents larger collections to execute the ansible-lint step [1].
This patch modifies the calls to ansible-lint on development tools and
upstream CI to use the same arguments as galaxy-importer and disables
the execution of the ansible-lint step for the Ansible's sanity test.
Requested ansible-lint version for tools is also updated, as a more
recent one is required.
This change will not allow development using an environment using Python
2.7, due to newer ansible-lint requirements. Roles and modules tests
against target nodes using Python 2.7 is still possible.
[1]: https://github.com/ansible/galaxy-importer/pull/231
When runing tests using 'utils/run-tests.sh' from inside an existing
Python virtual environment the Ansible collections are not installed due
to the order of execution of the script. On a machine that does not have
the 'containers.*' collection the test fails as there is no container
connector available.
This patch moves the section that installs Ansible collections to run
after the virtual environment is configured, and then install the
collections (usually, only 'containers.podman'), allowing the tests to
be executed.
ansible-core 2.15 has been released on May 15th, 2023, and version 2.12
has reached EOL on May 22nd, 2023.
This patch updates the ansible-core versions used on upstream CI tests
to reflect Ansible's new releases.
The usage of module_defaults allows to reduce the size of the tests and
to have the needed information in the tasks only. The default values for the
parameters are automatically passed to the module by Ansible.
It is not possible to use a module group for module_defaults as this could
only be done with Ansible Collections. The tests are also used upstream and
downstream without a collection.
Without groups of a collection it is needed to add the defaults for all
modules separately.
Simple example:
module_defaults:
ipahost:
ipaadmin_password: SomeADMINpassword
ipaapi_context: "{{ ipa_context | default(omit) }}"
Several module example using YAML anchors and aliases:
module_defaults:
ipahost: &ipa_module_defaults
ipaadmin_password: SomeADMINpassword
ipaapi_context: "{{ ipa_context | default(omit) }}"
ipauser: *ipa_module_defaults
ipagroup: *ipa_module_defaults
The documentation for "become" and "gather_facts" has been updated to
make sure that these parameters are enabled only in new tests if it is
really needed.
The module action group <collection-prefix>.modules is created
automatically while building the galaxy release.
The action group can be used for module_defaults in this way:
module_defauls:
group/<collection-prefix>.modules:
ipaadmin_password: SomeADMINpassword
Example:
module_defaults:
group/freeipa.ansible_freeipa.modules:
ipaadmin_password: SomeADMINpassword
ipaapi_context: "{{ ipa_context | default(omit) }}"
collections:
- freeipa.ansible_freeipa
The section module_defaults was not handled by utils/galaxyfy.py, also
there was no verification that only roles and modules provided by
ansible-freeipa are matched for prepending the collection prefix.
The list of modules and roles is needed in several scripts now,
therefore it makes sense to have one place for this.
Here are the current variables:
BASE_DIR: Base directory of the repo
ROLES: List of roles in the roles folder
MANAGEMENT_MODULES: List of management modules in the plugins/modules
folder
ROLES_MODULES: List of modules in the roles/*/library folders
ALL_MODULES: List of all modules, the management and the roles
modules
All lists are sorted.
