Currently, there is a hard coded timeout in galaxy-importer that
prevents larger collections to execute the ansible-lint step [1].
This patch modifies the calls to ansible-lint on development tools and
upstream CI to use the same arguments as galaxy-importer and disables
the execution of the ansible-lint step for the Ansible's sanity test.
Requested ansible-lint version for tools is also updated, as a more
recent one is required.
This change will not allow development using an environment using Python
2.7, due to newer ansible-lint requirements. Roles and modules tests
against target nodes using Python 2.7 is still possible.
[1]: https://github.com/ansible/galaxy-importer/pull/231
When runing tests using 'utils/run-tests.sh' from inside an existing
Python virtual environment the Ansible collections are not installed due
to the order of execution of the script. On a machine that does not have
the 'containers.*' collection the test fails as there is no container
connector available.
This patch moves the section that installs Ansible collections to run
after the virtual environment is configured, and then install the
collections (usually, only 'containers.podman'), allowing the tests to
be executed.
ansible-core 2.15 has been released on May 15th, 2023, and version 2.12
has reached EOL on May 22nd, 2023.
This patch updates the ansible-core versions used on upstream CI tests
to reflect Ansible's new releases.
The usage of module_defaults allows to reduce the size of the tests and
to have the needed information in the tasks only. The default values for the
parameters are automatically passed to the module by Ansible.
It is not possible to use a module group for module_defaults as this could
only be done with Ansible Collections. The tests are also used upstream and
downstream without a collection.
Without groups of a collection it is needed to add the defaults for all
modules separately.
Simple example:
module_defaults:
ipahost:
ipaadmin_password: SomeADMINpassword
ipaapi_context: "{{ ipa_context | default(omit) }}"
Several module example using YAML anchors and aliases:
module_defaults:
ipahost: &ipa_module_defaults
ipaadmin_password: SomeADMINpassword
ipaapi_context: "{{ ipa_context | default(omit) }}"
ipauser: *ipa_module_defaults
ipagroup: *ipa_module_defaults
The documentation for "become" and "gather_facts" has been updated to
make sure that these parameters are enabled only in new tests if it is
really needed.
The module action group <collection-prefix>.modules is created
automatically while building the galaxy release.
The action group can be used for module_defaults in this way:
module_defauls:
group/<collection-prefix>.modules:
ipaadmin_password: SomeADMINpassword
Example:
module_defaults:
group/freeipa.ansible_freeipa.modules:
ipaadmin_password: SomeADMINpassword
ipaapi_context: "{{ ipa_context | default(omit) }}"
collections:
- freeipa.ansible_freeipa
The section module_defaults was not handled by utils/galaxyfy.py, also
there was no verification that only roles and modules provided by
ansible-freeipa are matched for prepending the collection prefix.
The list of modules and roles is needed in several scripts now,
therefore it makes sense to have one place for this.
Here are the current variables:
BASE_DIR: Base directory of the repo
ROLES: List of roles in the roles folder
MANAGEMENT_MODULES: List of management modules in the plugins/modules
folder
ROLES_MODULES: List of modules in the roles/*/library folders
ALL_MODULES: List of all modules, the management and the roles
modules
All lists are sorted.
The ansible builtins are using the ansible.builtin. prefix now, but
galaxyfy was not supporting the prefix. Therefore vars in set_fact tasks
got the collection prefix and include_role tasks have not been handled
correctly.
The deprecated shell scripts used to deplay IPA are outdated and are
not needed to deploy IPA. There is no documentation about them, and
they would need to be updated and maintained in the future.
The generation of the OTP for client deployment is now completely
happening on the first of the given or detected servers with delegate_to.
The module ipaclient_get_otp has been replaced by a new module using code
from ipahost module and module_utils ansible_freeipa_module.
The action plugin ipaclient_get_otp has been removed and with this also
ipaclient_get_facts.
If an admin keytab is used instead of an admin password, it is copied to
the server as a temporary file to enable the OTP generation. The temporary
file is removed again after using the ipaclient_get_otp module.
The utils script build-galaxy-release.sh has been updated to not copy the
ipaclient action plugin to the global plugins folder of the collection.
This change is import for the use of the ipaclient role with AAP as only
the base environment is sufficient now.
The ipaclient README and also the global README have been updated as
kinit is not needed anymore on the controller for OTP.
Fixes#903 (Allow the use of principals other than admin when using
ipaadmin_keytab)
ansible-test with ansible-2.14 is adding a lot of new tests to ensure
that the documentation section and the agument spec is complete.
To aid the creation of new_modules that follow these rules, the
'new_module' script and its templates were modified to use the new
ansible-test rules. The 'new_module' script now requires a new
argument, github_user, and the code templates provide the required
fields (like 'type' and 'elements') to the provide examples.
This patch forces the addition of a remote repository pointing to the
main ansible-freeipa repo, and fetch its contents before confaring the
modified files. The remote repository is removed after the modified
file list is generated.
If the repository is setup in a way that master branch is not available
for comparing the current HEAD against it, the comparison will fail and
not module/role will be scheduled for testing.
This patch forces fetching 'master' from ansible-freeipa repository,
allowing the comparison to be performed.
The current workflow for bug fixing or new enhancements in
ansible-freeipa includes running Ansible playbooks tests for all the
available plugins for every pull request, even for contained
modifications.
This patch creates a new workflow for pull requests where only the
affected plugins are tested in the PR. Changes that might affect other
parts of the code will trigger tests for the parts affected.
A utility script, utils/filter_tests, is used to set the variables
IPA_ENABLED_MODULES and IPA_ENABLED_TESTS before executing the tests,
effectively limiting which tests are executed. The script uses the
python plugin 'utils/filter_plugins.py' which lists all test modules
that should be executed for a list of modified source files.
Tests are selected for execution based on the plugin name. For example,
a change to 'plugins/modules/ipalocation.py' would trigger all playbook
tests under 'tests/location'. If a test playbook is modified, it is
scheduled to be executed. Changes to any file under
'plugins/module_utils' will force the execution of all tests, since any
module might be affected by that change.
The nature of the change is not evaluated, so a simple typo fix of a
comment in a file under 'plugins/module_utils' would still schedule all
test playbooks to be executed.
For roles, any file changed under the role directory would set the role
to be included in the tests. Playbook tests for roles must be created
under 'tests/<rolename>_role', where role name in the name of the role
without 'ipa', for example, the 'ipabackup' role test playbooks would
be stored under 'tests/backup_role'.
Since there is the possibility that the list of tests to be executed
might be less than the number of tests groups used (3), a new pytest
dependency was added, pytest-custom_exit_code, so that having no tests
to run isn't a test failure.
A new pipeline on Azure needs to be created to use the new test script.
Before "short description" was used in most plugins, modules and also
in the new module templates.
ansible-doc was therefore not showing the short description. To fix the
issue the flag was renamed to short_description instead.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2121362
'ansible-doc' -l lists most idm modules as 'UNDOCUMENTED'
Since test configuration can vary in different scenarios (test images)
this patch adds a script to list the scenarios configuration, and a
step to the playbook test jobs to display the scenario configuration.
Recent versions of ansible-test require the existence of a CHANGELOG
file in the root of the collection. This changes extracts the changes
of the latest available release tag using `utils/changelog` and create
the CHANGELOG file with the result of the command.
The generated changelog will include the changes for the latest release
and, if present, the available changes that were not part of a release.
The --tag TAG option is now printing the changes for the given TAG and
not since the given tag. The new option --galaxy is printing the changelog
since the latest tag and also for the latest tag.
These changes are simplifying the generation of the changelog file that
is needed to pass the tests for galaxy and AutomationHub collections.
The "-i" option can be used to install the generated collection using
the ansible-galaxy collection install command. It is using the force
flag to install the collection if there is already a collection with the
same name and namespace. The ansible-galaxy collection build command is
already using the force flag to create the collection.
This patch add the lines necessary to allow the use of the attribute
`delete_continue`, as it is a commom attribute, and if newer commom
attributes are added to IPAAnsibleModule in the future, the usage will
be similar.
Some modules should be compared in a case insensitive manner, and this
patch adds an example of a call to IPAAnsibleModule.params_get_lowercase
and a note on its usage.
This patch refactors the module template for modules with member
management, in a way that the addition of member management command
logic is not duplicated in different states or actions.
This idiom has been applied recently along with other fixes to modules
with idempotence issues reducing the modules code size and centering
code logic in specific blocks.
The script now fixes the references for all doc_fragments in the
plugins/doc_fragments folder. So far it was only fixing the references
for ipamodule_base_docs.
PR #762 (automount location: add support for delete_continue) added an
other doc_fragment and the references have not been fixes as needed.
The script is now using a build dir for the creation of the Ansible
Collection. Additionally only files known to the fit repo are pulled in
by default. The new "-a" option is pulling in all files from local repo.
The new -k" option can be used to keep the build dir for verification of
the changes to the files.
The colleciton is placed into the main repo dir and no git reset --hard
is used in the repo to preserve local changes.
The Variables and also the Return Variables sections contained an extra
header with the module name. This is only needed if there are more than
one module in the README.
These are indent issues, one item per line for argument_specs items
containing options dicts and missing or overflow spaces for comments
and dict delimiters.
