156 Commits

Author SHA1 Message Date
Thomas Woerner
b760863847 utils/get_test_modules.py: Support ansible.builtin. prefix
The ansible.builtin. prefix was not supported. Therefore tasks have not
been identified properly.
2023-01-09 18:04:07 +01:00
Thomas Woerner
e3bf82d873 utils/galaxyfy.py: Support builtins with ansible.builtin. prefix
The ansible builtins are using the ansible.builtin. prefix now, but
galaxyfy was not supporting the prefix. Therefore vars in set_fact tasks
got the collection prefix and include_role tasks have not been handled
correctly.
2023-01-09 17:36:56 +01:00
Thomas Woerner
27cbd40182 ansible-freeipa.spec.in: Fix for loop with wildcard
The issue within the for loops to remove python shebangs and to remove the
execution flag from python files has been solved.
2022-12-06 10:18:08 +01:00
Rafael Guterres Jeffman
8459e1c454 utils: Remove deprecated shell scripts used to deploy IPA.
The deprecated shell scripts used to deplay IPA are outdated and are
not needed to deploy IPA. There is no documentation about them, and
they would need to be updated and maintained in the future.
2022-11-28 12:34:09 -03:00
Thomas Woerner
624e0d3435 ipaclient: No kinit on controller for deployment using OTP
The generation of the OTP for client deployment is now completely
happening on the first of the given or detected servers with delegate_to.
The module ipaclient_get_otp has been replaced by a new module using code
from ipahost module and module_utils ansible_freeipa_module.

The action plugin ipaclient_get_otp has been removed and with this also
ipaclient_get_facts.

If an admin keytab is used instead of an admin password, it is copied to
the server as a temporary file to enable the OTP generation. The temporary
file is removed again after using the ipaclient_get_otp module.

The utils script build-galaxy-release.sh has been updated to not copy the
ipaclient action plugin to the global plugins folder of the collection.

This change is import for the use of the ipaclient role with AAP as only
the base environment is sufficient now.

The ipaclient README and also the global README have been updated as
kinit is not needed anymore on the controller for OTP.

Fixes #903 (Allow the use of principals other than admin when using
            ipaadmin_keytab)
2022-11-23 15:20:47 +01:00
Rafael Guterres Jeffman
29dccf3d8a pylint: Fix pylint issues on utils/galaxyfy-module-EXAMPLES.py 2022-11-11 15:59:42 -03:00
Rafael Guterres Jeffman
9a8d756ad6 new_module: Modify new_module and templates for Ansible 2.14
ansible-test with ansible-2.14 is adding a lot of new tests to ensure
that the documentation section and the agument spec is complete.

To aid the creation of new_modules that follow these rules, the
'new_module' script and its templates were modified to use the new
ansible-test rules. The 'new_module' script now requires a new
argument, github_user, and the code templates provide the required
fields (like 'type' and 'elements') to the provide examples.
2022-10-05 08:46:53 -03:00
Rafael Guterres Jeffman
433d1096f8 Merge pull request #911 from t-woerner/update_spec_file
utils/ansible-freeipa.spec.in: Sync with Fedora rawhide spec file
2022-09-09 14:46:30 -03:00
Rafael Guterres Jeffman
777f25d91c upstream CI: Force retrieval of ansible-freeipa master.
This patch forces the addition of a remote repository pointing to the
main ansible-freeipa repo, and fetch its contents before confaring the
modified files. The remote repository is removed after the modified
file list is generated.
2022-09-08 08:51:56 -03:00
Thomas Woerner
68a99ba5f9 utils/ansible-freeipa.spec.in: Sync with Fedora rawhide spec file
The spec file was updated and was lacking several updates.
2022-09-08 13:42:30 +02:00
Rafael Guterres Jeffman
1eb83548fa upstream CI: Ensure 'master' branch is available for set_test_modules
If the repository is setup in a way that master branch is not available
for comparing the current HEAD against it, the comparison will fail and
not module/role will be scheduled for testing.

This patch forces fetching 'master' from ansible-freeipa repository,
allowing the comparison to be performed.
2022-09-06 15:47:01 -03:00
Rafael Guterres Jeffman
2514158498 upstream CI: run PR tests only for affected plugins
The current workflow for bug fixing or new enhancements in
ansible-freeipa includes running Ansible playbooks tests for all the
available plugins for every pull request, even for contained
modifications.

This patch creates a new workflow for pull requests where only the
affected plugins are tested in the PR. Changes that might affect other
parts of the code will trigger tests for the parts affected.

A utility script, utils/filter_tests, is used to set the variables
IPA_ENABLED_MODULES and IPA_ENABLED_TESTS before executing the tests,
effectively limiting which tests are executed. The script uses the
python plugin 'utils/filter_plugins.py' which lists all test modules
that should be executed for a list of modified source files.

Tests are selected for execution based on the plugin name. For example,
a change to 'plugins/modules/ipalocation.py' would trigger all playbook
tests under 'tests/location'. If a test playbook is modified, it is
scheduled to be executed. Changes to any file under
'plugins/module_utils' will force the execution of all tests, since any
module might be affected by that change.

The nature of the change is not evaluated, so a simple typo fix of a
comment in a file under 'plugins/module_utils' would still schedule all
test playbooks to be executed.

For roles, any file changed under the role directory would set the role
to be included in the tests. Playbook tests for roles must be created
under 'tests/<rolename>_role', where role name in the name of the role
without 'ipa', for example, the 'ipabackup' role test playbooks would
be stored under 'tests/backup_role'.

Since there is the possibility that the list of tests to be executed
might be less than the number of tests groups used (3), a new pytest
dependency was added, pytest-custom_exit_code, so that having no tests
to run isn't a test failure.

A new pipeline on Azure needs to be created to use the new test script.
2022-09-02 19:06:46 -03:00
Rafael Guterres Jeffman
c6cc4df77b check_test_configuration: Add support for IPA_* environment variables
This patch adds support for IPA_ENABLED_* and IPA_DISABLED_* environment
variables as existing global configuration for the tests.
2022-08-31 15:06:43 -03:00
Rafael Guterres Jeffman
7e6e6c2dc2 run-tests: Run tests locally with upstream CI images
This patch allows local execution of playbook tests using ustream CI
testing images. Either 'podman' or 'docker' can be used to execute the
tests.
2022-08-29 18:13:30 -03:00
Thomas Woerner
81143be96a Fix short_description flag in plugins, role modules and templates
Before "short description" was used in most plugins, modules and also
in the new module templates.

ansible-doc was therefore not showing the short description. To fix the
issue the flag was renamed to short_description instead.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2121362
       'ansible-doc' -l lists most idm modules as 'UNDOCUMENTED'
2022-08-26 15:27:00 +02:00
Rafael Guterres Jeffman
abef329b8a upstream ci: Add step to display scenario configuration
Since test configuration can vary in different scenarios (test images)
this patch adds a script to list the scenarios configuration, and a
step to the playbook test jobs to display the scenario configuration.
2022-08-24 16:52:38 -03:00
Rafael Guterres Jeffman
d82abdbef9 build-galaxy-release: Automatically create CHANGELOG.
Recent versions of ansible-test require the existence of a CHANGELOG
file in the root of the collection. This changes extracts the changes
of the latest available release tag using `utils/changelog` and create
the CHANGELOG file with the result of the command.

The generated changelog will include the changes for the latest release
and, if present, the available changes that were not part of a release.
2022-06-14 10:33:08 -03:00
Thomas Woerner
8b8cbdd8c2 utils/changelog: Fixed --tag option, new --galaxy option
The --tag TAG option is now printing the changes for the given TAG and
not since the given tag. The new option --galaxy is printing the changelog
since the latest tag and also for the latest tag.

These changes are simplifying the generation of the changelog file that
is needed to pass the tests for galaxy and AutomationHub collections.
2022-06-14 15:07:11 +02:00
Rafael Guterres Jeffman
0297cbe973 Merge pull request #829 from t-woerner/build-galaxy-release_with_install
utils/build-galaxy-release.sh: Add "-i" to install generated collection
2022-05-12 12:12:15 -03:00
Thomas Woerner
1ec0d1e640 utils/build-galaxy-release.sh: Add "-i" to install generated collection
The "-i" option can be used to install the generated collection using
the ansible-galaxy collection install command. It is using the force
flag to install the collection if there is already a collection with the
same name and namespace. The ansible-galaxy collection build command is
already using the force flag to create the collection.
2022-05-12 15:06:39 +02:00
Thomas Woerner
ce143bad52 Merge pull request #805 from rjeffman/templates_add_password_example_playbooks
utils/new_module templates: Add missing password to example playbooks.
2022-04-26 12:39:18 +02:00
Rafael Guterres Jeffman
f770b5d581 utils/new_module templates: Add missing password to example playbooks.
Add missing ipaadmin_password to example playbooks so new modules have
all necessary fields set on basic files.
2022-04-11 18:06:57 -03:00
Rafael Guterres Jeffman
4df2cab42a module templates: Add delete_commit code template.
This patch add the lines necessary to allow the use of the attribute
`delete_continue`, as it is a commom attribute, and if newer commom
attributes are added to IPAAnsibleModule in the future, the usage will
be similar.
2022-02-17 15:22:18 -03:00
Rafael Guterres Jeffman
5d6324e2da module templates: Add example and note for case insensitive members.
Some modules should be compared in a case insensitive manner, and this
patch adds an example of a call to IPAAnsibleModule.params_get_lowercase
and a note on its usage.
2022-02-17 14:31:48 -03:00
Rafael Guterres Jeffman
8772379dcc module templates: Refactor member management.
This patch refactors the module template for modules with member
management, in a way that the addition of member management command
logic is not duplicated in different states or actions.

This idiom has been applied recently along with other fixes to modules
with idempotence issues reducing the modules code size and centering
code logic in specific blocks.
2022-02-17 14:31:48 -03:00
Thomas Woerner
2af7602a8c build-galaxy-release: Fix refs for all doc_fragments in plugins/doc_fragments
The script now fixes the references for all doc_fragments in the
plugins/doc_fragments folder. So far it was only fixing the references
for ipamodule_base_docs.

PR #762 (automount location: add support for delete_continue) added an
other doc_fragment and the references have not been fixes as needed.
2022-02-14 10:51:20 +01:00
Rafael Guterres Jeffman
7632f90edb Merge pull request #724 from t-woerner/enhance_utils_build-galaxy-release_sh
build-galaxy-release.sh: Use build dir, new options, checks, no reset
2022-01-13 10:26:25 -03:00
Rafael Guterres Jeffman
ce8487e394 pylint: Enable pylint for ansible-freeipa roles.
This patch enables pylint evaluation for ansible-freeipa roles in
both the local script 'utils/lint-check.sh' and in upstream CI.
2022-01-12 12:09:46 -03:00
Thomas Woerner
de8d724663 build-galaxy-release.sh: Use build dir, new options, checks, no reset
The script is now using a build dir for the creation of the Ansible
Collection. Additionally only files known to the fit repo are pulled in
by default. The new "-a" option is pulling in all files from local repo.
The new -k" option can be used to keep the build dir for verification of
the changes to the files.

The colleciton is placed into the main repo dir and no git reset --hard
is used in the repo to preserve local changes.
2022-01-12 13:46:33 +01:00
Rafael Guterres Jeffman
e0bdfdfe32 Merge pull request #712 from t-woerner/readme_fixes
Readme fixes
2021-12-23 11:33:32 -03:00
Thomas Woerner
65937ed8c3 module README files: Drop extra module header in Variables section
The Variables and also the Return Variables sections contained an extra
header with the module name. This is only needed if there are more than
one module in the README.
2021-12-23 15:25:46 +01:00
Thomas Woerner
85006d611f Fix ansible-test reported pep8 errors
These are indent issues, one item per line for argument_specs items
containing options dicts and missing or overflow spaces for comments
and dict delimiters.
2021-12-14 18:37:37 +01:00
Thomas Woerner
f47d134335 utils/gen_module_docs.py: Drop duplicate setup_adtrust key 2021-12-14 17:53:25 +01:00
Thomas Woerner
85257b9d03 build-galaxy-release: Real cleanup of ipabackup_get_backup_dir.py link
plugins/modules/ipabackup_* needs to be cleaned up not
plugins/action/ipabackup_*
2021-12-14 17:26:19 +01:00
Rafael Guterres Jeffman
8ba665213f linters: Fix shellcheck warnings in 'utils' scripts. 2021-11-24 21:01:29 -03:00
Thomas Woerner
f113e7071e build-galaxy-release: Cleanup of ipabackup_get_backup_dir.py link
The link for plugins/modules/ipabackup_get_backup_dir.py from
roles/ipabackup/library/ipabackup_get_backup_dir.py was left over
after the script finished.
2021-11-24 13:02:24 +01:00
Thomas Woerner
2e8c2f881f Use ansible.module_utils.six instead of six
This patch is needed to pass Automation Hub tests.
2021-11-24 10:07:02 +01:00
Thomas Woerner
b88cab07ff Add missing whitespace around arithmetic operator
This patch is needed to pass Automation Hub tests.
2021-11-24 10:07:02 +01:00
Thomas Woerner
10b16a3bbf Remove non-module shebang
This patch is needed to pass Automation Hub tests.
2021-11-24 10:07:02 +01:00
Thomas Woerner
5c871242a7 Add __future__ imports and __metaclass__ for automationhub
This patch is needed to pass Automation Hub tests.
2021-11-24 10:07:02 +01:00
Thomas Woerner
0d57d69a99 galaxyfy: Fix newline issue in module examples
The newlines in module examples have been removed due to wrong strip for
the input lines.
2021-10-12 16:18:57 +02:00
Thomas Woerner
c92e9a5ca1 galaxyfy: Fix roles after vars
If roles have been used after vars, the name of the role was not changed
as the "vars:" section was turning off changeable. A "roles:" section is
now turning on changeable.
2021-10-12 10:56:58 +02:00
Thomas Woerner
7ee6fc3238 build-galaxy-release.sh: Use proper action plugins path plugins/action
The action plugins path was wrong. It was "plugins/action_plugins" and
should have been "plugins/action".
2021-10-11 17:06:17 +02:00
Thomas Woerner
48d0b2f5a2 Merge pull request #654 from rjeffman/templates_variables_invtest
Fix module templates README and invalid parameter tests.
2021-10-04 21:41:02 +02:00
Rafael Guterres Jeffman
2a169ceb46 templates: Add IPA API connection variables to README.
All modules should use the same description for IPA API connection
variables. This change add description for ipaapi_context and
ipaapi_ldap_cache variable to the module README templates.
2021-10-04 14:11:04 -03:00
Rafael Guterres Jeffman
aa8648425b templates: Use IPAAnsibleModule params_fail_used_invalid.
Modify new module templates to use IPAAnsibleModule method
`params_fail_used_invalid` to check for invalid parameter use for
a given state/action.
2021-10-04 14:06:43 -03:00
Thomas Woerner
53476687be utils/build-galaxy-release: Fix doc_fragment prefix for galaxy
The items in extends_documentation_fragments of the modules need to
have the proper galaxy prefix.

The documentation fragment

  - ipamodule_base_docs

needs to be changed to

  - ${collection_prefix}.ipamodule_base_docs
2021-10-04 18:35:06 +02:00
Rafael Guterres Jeffman
e069395ba0 Make ansible-lint and yamllint use more strict rules.
This patch modifies configuration of both ansible-lint and yamllint
to check for more rules, resulting in a more strict verification.

For ansible-lint verification of errors 301, 305 and 505 are skipped,
due to false positives. For the same reason, 'experimental' rules
are skipped.

ansible-lint error 306 is skipped since the fix is to set pipefail,
which is not available in all shells (for example dash, which runs
ansible-freeipa CI).

Yamllint disabled rules (comments, and indentation) would introduce a
huge amount of small changes, and are left for future changes, it
deemed necessary.
2021-09-29 15:49:00 -03:00
Rafael Guterres Jeffman
1c679c5a9a templates: Allow execution of plugins in an IPA client host.
Modify templates to create the basic tests for `ipaapi_context: client`.
2021-09-27 13:51:53 -03:00
Thomas Woerner
e897ecb27a utils/templates/ipamodule+member.py.in: Use execute_ipa_commands
execute_ipa_commands replces the check mode exit, the loop over the
generated commands and also in the member failure handling for modules
with member support.
2021-09-08 14:34:07 +02:00