Fix all deprecated module_utils imports before ansible-core 2.24 removal
SUMMARY
Fixes all deprecated ansible.module_utils imports across the entire collection that will be removed in ansible-core 2.24.
This PR comprehensively addresses deprecation warnings reported in #686 by updating import statements in 20 files to use the new recommended import paths, and removes 8 unused test utility files that contained deprecated imports.
Deprecated imports replaced:
Deprecated import
Replacement
ansible.module_utils._text
ansible.module_utils.common.text.converters
ansible.module_utils.common._collections_compat
collections.abc
ansible.module_utils.six.moves.shlex_quote
shlex.quote
ansible.module_utils.six.moves.reduce
functools.reduce
ansible.module_utils.six.moves.urllib.parse.urlparse
urllib.parse.urlparse
ansible.module_utils.six.string_types
basestring/str (Python 2/3 compatible)
ansible.module_utils.six.text_type
str
ansible.module_utils.six.PY3
Removed (simplified Python 2/3 conditionals)
ansible.module_utils.six.with_metaclass
Native metaclass= syntax
ansible.module_utils.six.iteritems
dict.items()
Files fixed (20 files, 1 commit per file for easier review):
plugins/action/patch.py
plugins/action/synchronize.py
plugins/callback/cgroup_perf_recap.py
plugins/callback/json.py
plugins/callback/jsonl.py
plugins/callback/profile_roles.py
plugins/callback/profile_tasks.py
plugins/modules/acl.py
plugins/modules/authorized_key.py
plugins/modules/firewalld_info.py
plugins/modules/mount.py
plugins/modules/patch.py
plugins/modules/rhel_rpm_ostree.py
plugins/modules/rpm_ostree_upgrade.py
plugins/modules/seboolean.py
plugins/modules/synchronize.py
plugins/modules/sysctl.py
plugins/shell/csh.py
plugins/shell/fish.py
tests/unit/modules/system/test_mount.py
Files deleted (8 unused test utility files):
These files are dead code - none of them are imported or used anywhere in the test suite or the collection. Removing them also addresses Python 2.7 compatibility concerns raised in code review, as several contained deprecated imports that would be incorrect to fix for Python 2.
tests/unit/compat/builtins.py
tests/unit/mock/loader.py
tests/unit/mock/path.py
tests/unit/mock/procenv.py
tests/unit/mock/vault_helper.py
tests/unit/mock/yaml_helper.py
tests/unit/modules/conftest.py
tests/unit/modules/utils.py
Completeness verified with:
git grep -n -P '_compat|utils._text|utils.six' -- '*.py' | grep -v yml
This command returns no results, confirming all deprecated imports have been replaced.
Notes on Python 2.7 compatibility:
For modules that may run on Python 2.7 managed hosts (e.g., authorized_key.py, synchronize.py, sysctl.py), Python 2/3 compatible fallbacks were used instead of direct Python 3 replacements:
authorized_key.py: try/except ImportError for urllib.parse.urlparse (falls back to urlparse on Python 2)
synchronize.py: try/except ImportError for shlex.quote (falls back to pipes.quote on Python 2)
sysctl.py: uses sys.version_info to set string_types to str on Python 3 (basestring on Python 2)
Also removes corresponding pylint:ansible-bad-import-from entries from tests/sanity/ignore-2.21.txt and tests/sanity/ignore-2.22.txt where applicable.
Fixes#686
ISSUE TYPE
Bugfix Pull Request
ADDITIONAL INFORMATION
Approach:
Each file is fixed in a separate commit for easier code review. The changelog fragment is added in a final commit. Corresponding pylint:ansible-bad-import-from ignore entries in tests/sanity/ignore-2.21.txt and tests/sanity/ignore-2.22.txt are removed in the same commit as the file fix (or the file removal commit).
CI results:
All 59 checks passing (Azure Pipelines sanity, units, lint, Docker, Remote across ansible-core 2.17 through devel, and Zuul ansible/check).
Reviewed-by: Felix Fontein <felix@fontein.de>
Reviewed-by: Pavel Bar
Reviewed-by: Abhijeet Kasurde
(cherry picked from commit 2022c1bd86)
Co-authored-by: centosinfra-prod-github-app[bot] <161850885+centosinfra-prod-github-app[bot]@users.noreply.github.com>
plugins/callback/profile_tasks.py: Add option to provide a different date/time format
SUMMARY
The new datetime_format key will offer the possibility of providing a different date/time format than the default one ('%A %d %B %Y %H:%M:%S %z').
The iso8601 value can be used as an '%Y-%m-%dT%H:%M:%S.%f' alias (format of the ISO 8601 date/time standard).
The code has changed from using the time API to the datetime one in order to support sub-second precision (needed by the ISO 8601 format, for example).
Fixes: #279
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
plugins/callback/profile_tasks.py
ADDITIONAL INFORMATION
Output with no key which keeps current behavior:
TASK [Import subscription manifest] *******************************************************************************************************************************************************************************************************************************************************************
Thursday 10 April 2025 00:52:11 +0200 (0:00:17.416) 0:00:17.453 ********
changed: [localhost]
Output with datetime_format = 'iso8601':
TASK [Import subscription manifest] *******************************************************************************************************************************************************************************************************************************************************************
2025-04-10T00:55:19.967718 (0:00:15.664) 0:00:15.691 ********************
changed: [localhost]
Output with datetime_format = '%Y-%m-%dT%H:%M:%S.%f%z' (ISO 8601 with UTC offset information):
TASK [Import subscription manifest] *******************************************************************************************************************************************************************************************************************************************************************
2025-04-10T00:57:49.290347+0200 (0:00:16.265) 0:00:16.293 ***************
changed: [localhost]
Reviewed-by: Hideki Saito <saito@fgrep.org>
(cherry picked from commit 1994b2cf1c)
Co-authored-by: softwarefactory-project-zuul[bot] <33884098+softwarefactory-project-zuul[bot]@users.noreply.github.com>
Fixes#462 notice permission denied on authorized_key module
SUMMARY
As of right now the authorized_key module does not notice on an "absent" if a authorized_keys file is simply not readable to the executing user. I am trying to fix that
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
authorized_key
ADDITIONAL INFORMATION
Execute as a user that does not have access to the root users authorized keys file
- name: Delete key from root user
ansible.posix.authorized_key:
state: absent
user: root
key: ssh-rsa xxxxxxxx
- name: Delete key from root user
become: true
ansible.posix.authorized_key:
state: absent
user: root
key: ssh-rsa xxxxxxxx
The one without become will succeed before my change and will fail with a permission denied error after my change. The 2nd task will actually remove a key from root user if become privileges are available for the executing user
Reviewed-by: Brian Coca
Reviewed-by: Klaas Demter
Reviewed-by: Felix Fontein <felix@fontein.de>
Reviewed-by: Hideki Saito <saito@fgrep.org>
(cherry picked from commit 72a6eb9729)
Co-authored-by: softwarefactory-project-zuul[bot] <33884098+softwarefactory-project-zuul[bot]@users.noreply.github.com>
Use module.warn() instead of returning warnings
SUMMARY
Returning warnings as warnings has been deprecated.
Ref: #635.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
firewalld_info
mount
Reviewed-by: Hideki Saito <saito@fgrep.org>
(cherry picked from commit 6da1331018)
Co-authored-by: softwarefactory-project-zuul[bot] <33884098+softwarefactory-project-zuul[bot]@users.noreply.github.com>
profile_* callbacks: avoid deprecated/deleted functions
SUMMARY
The profile_roles and profile_tasks callbacks define methods playbook_on_setup and playbook_on_stats which have been deleted/deprecated:
playbook_on_stats has been deprecated, v2_playbook_on_stats should be used instead (that one has already been there for many years: ansible/ansible@ba0e532 was added in 2015).
playbook_on_setup has been deleted (ansible/ansible@eec57ec), and its v2 variant was already deleted in 2017: ansible/ansible@59d5481
Ref: #635
ISSUE TYPE
Bugfix Pull Request
Feature Pull Request
COMPONENT NAME
profile_roles
profile_tasks
Reviewed-by: Abhijeet Kasurde
Reviewed-by: Hideki Saito <saito@fgrep.org>
Fixes issue related to latest ansible-core devel branch
SUMMARY
Fixes a bug related to updating the ansible-core devel branch.
Fixes incorrect load path for json module in cgroup_perf_recap
Remove unnecessary condition from seboolean integration tests
Optimize conditions for selinux integration tests
Fixes#630
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
ansible.posix.cgroup_perf_recap
ADDITIONAL INFORMATION
N/A
[Breaking Change] [firewalld] Change type of icmp_block_inversion option from str to bool
SUMMARY
Changed the type of icmp_block_inversion option from str to bool
Fixes#586
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
ansible.posix.firewalld
ADDITIONAL INFORMATION
Related #582 and #584
Reviewed-by: Adam Miller <admiller@redhat.com>
Reviewed-by: Andrew Klychkov <aklychko@redhat.com>
Remove comment from fstab entry on updating.
SUMMARY
Fix#595.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
ansible.posix.mount
ADDITIONAL INFORMATION
Reviewed-by: Hideki Saito <saito@fgrep.org>
Reviewed-by: Vladimir Botka <vbotka@gmail.com>
authorized_key: Allow local path to a key
SUMMARY
Add option to specify an absolute path to file with SSH key(s) for authorized_key
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
authorized_key
ADDITIONAL INFORMATION
Before this change you would need to get key using ansible.builtin.slurp or something like ansible.builtin.command: cat <file> with register
I tried to keep it as simple as possible
# Now this is possible
- name: Set authorized keys taken from path
ansible.posix.authorized_key:
user: charlie
state: present
key: /home/charlie/.ssh/id_rsa.pub
Reviewed-by: Hideki Saito <saito@fgrep.org>
Reviewed-by: alexander
(feat) add no_log option for 'opts' parameter
SUMMARY
Allows you to set no_log on just the opts parameter.
This is useful for CIFS/SMB mounts that would otherwise leak secrets.
Adds feature from issue: . #497
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
mount
Reviewed-by: Hideki Saito <saito@fgrep.org>
maintain proper formating of the remote paths when defined as user@ho…
…st:/... or host:/...
SUMMARY
update _format_rsync_rsh_target for proper handling of remote rsh/ssh paths. fixes#360
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
ansible.posix.synchronize
Reviewed-by: Adam Miller <admiller@redhat.com>
Reviewed-by: Hideki Saito <saito@fgrep.org>
Firewalld: Add functionality to set forwarding
SUMMARY
Adds firewalld functionality to do the equivalent of firewall-cmd --add-forwarding --zone={zone}.
Functionality is exactly analogous to the firewall-cmd --add-masquerade --zone={zone} already present.
Fixes#529
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
firewalld
ADDITIONAL INFORMATION
Usage:
- ansible.posix.firewalld:
forward: true
state: enabled
permanent: true
zone: internal
Reviewed-by: Abhijeet Kasurde
Reviewed-by: Hideki Saito <saito@fgrep.org>
Sometimes it's necessary to configure SELinux before it's enabled on the
system. There's `ignore_selinux_state` which should allow it. Before
this change `seboolean` module failed on SELinux disabled system even
with `ignore_selinux_state: true` and SELinux policy installed while
`semanage boolean` worked as expected:
$ ansible -i 192.168.121.153, -m seboolean -a "name=ssh_sysadm_login state=on ignore_selinux_state=true" all
192.168.121.153 | FAILED! => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"msg": "Failed to get list of boolean names"
}
$ ssh root@192.168.121.153 semanage boolean -l | grep ssh_sysadm_login
ssh_sysadm_login (off , off) Allow ssh to sysadm login
It's caused by `selinux.security_get_boolean_names()` and
`selinux.security_get_boolean_active(name)` which required SELinux
enabled system.
This change adds a fallback to semanage API which works in SELinux
disabled system when SELinux targeted policy is installed:
ANSIBLE_LIBRARY=plugins/modules ansible -i 192.168.121.153, -m seboolean -a "name=ssh_sysadm_login state=on persistent=true ignore_selinux_state=true" all
192.168.121.153 | CHANGED => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": true,
"name": "ssh_sysadm_login",
"persistent": true,
"state": true
}
$ ssh root@192.168.121.153 semanage boolean -l | grep ssh_sysadm_login
ssh_sysadm_login (on , on) Allow ssh to sysadm login
Note that without `persistent=true` this module is effectively NO-OP now.
Signed-off-by: Petr Lautrbach <lautrbach@redhat.com>
the CI failures are unrelated and shouldn't even be showing up ... I'm going to sort that out separately but that doesn't need to prevent this merge, all relevant CI tests passed
firewalld: make offline do something
SUMMARY
ansible.posix.firewalld has an offline flag, but it currently does not do anything. What most people expect it to do is allow the task to proceed even when firewalld is offline, so it makes the most sense for it to override the immediate flag and prevent the module from throwing an error in that case.
Fixes#81.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
firewalld
ADDITIONAL INFORMATION
Reviewed-by: Adam Miller <admiller@redhat.com>
The seboolean, selinux, firewalld, and firewalld_info modules depend on
system bindings that are only available for the default system python
interpreter. ansible-core is not packaged for the default system python
interpreter on RHEL 8 and 9. When automatic interpreter discovery does
not occur (e.g. when using implicit localhost [1]), ansible-core will
not use the system interpreter to run ansible modules and the
aforementioned modules will not work even if the bindings are installed.
The RHEL ansible-core maintainers as well as the EPEL ansible and
ansible-collection-* package maintainers (inc. me) have gotten multiple
bug reports about this. We have been telling people to fix their setup
to use the correct Python interpreter. Fortunately, ansible-core 2.11
and above have a module utility that'll respawn modules to use the
correct system interpreter.
[1] https://docs.ansible.com/ansible/latest/inventory/implicit_localhost.html
json[l] callback: add parameter to set JSON prettyprint indent level
SUMMARY
Add ANSIBLE_JSON_INDENT parameter to both the json and jsonl callback plugins. The default values are different between the two modules to maintain their existing behavior:
json: indent==4, causing a prettyprint output
jsonl: indent==0, causing a 1-line output
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
ansible.posix.json
ansible.posix.jsonl
ADDITIONAL INFORMATION
One specific use-case that is enabled by this feature: if a user chooses to use the jsonl plugin so that they still get output at the end of each task (vs. only at the end of the play), they may also want human-readable output so that they can monitor the status of their play. For example, setting the jsonl indent level to 4 gives a) output at the end of each task, and b) making that output be both machine readable and human readable.
Using this trivial playbook shows the change:
- hosts: localhost
gather_facts: false
tasks:
- name: hello, world
debug:
msg: hello, world
When using the jsonl callback, here's what one JSON emit looks like before the change:
{"_event":"v2_playbook_on_play_start","_timestamp":"2023-04-08T12:11:48.001806Z","play":{"duration":{"start":"2023-04-08T12:11:48.001383Z"},"id":"acde4800-1122-f32c-94c3-000000000001","name":"localhost"},"tasks":[]}
After the change, setting ANSIBLE_JSON_INDENT to 4, the same output looks like this:
{
"_event":"v2_playbook_on_play_start",
"_timestamp":"2023-04-08T12:12:47.787516Z",
"play":{
"duration":{
"start":"2023-04-08T12:12:47.787164Z"
},
"id":"acde4800-1122-2946-e3e4-000000000001",
"name":"localhost"
},
"tasks":[]
}
Both outputs are suitable for automated processes to parse the machine readable output. The second output has the benefit of being human readable.
Reviewed-by: Adam Miller <admiller@redhat.com>
Reviewed-by: Jeff Squyres
Add ANSIBLE_JSON_INDENT parameter to both the json and jsonl callback
plugins. The default values are different between the two modules to
maintain their existing behavior:
* json: indent==4, causing a prettyprint output
* jsonl: indent==0, causing a 1-line output
One specific use-case that is enabled by this feature: if a user
chooses to use the jsonl plugin so that they still get output at the
end of each task (vs. only at the end of the play), they may also want
human-readable output so that they can monitor the status of their
play. For example, setting the jsonl indent level to 4 gives a)
output at the end of each task, and b) making that output be both
machine readable and human readable.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
json[l] callback: add play/task path info
Add the play and task path info (i.e., filename and line number) to the JSON that is emitted from the json and jsonl callback plugins, allowing more accurate post-mortem analysis.
SUMMARY
Add the play and task path info (i.e., filename and line number) to the JSON that is emitted from the json and jsonl callback plugins, allowing more accurate post-mortem analysis.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
ansible.posix.json
ansible.posix.jsonl
ADDITIONAL INFORMATION
By also including the file/line number in the JSON data emitted, post-mortem analysis can unambiguously tie play/task log data to the specific play / task that generated it. Without this information, it could be difficult for automated processes to precisely map log output back to the task that generated it (especially with playbooks that either do not name tasks, or do not name tasks uniquely).
Using this trivial playbook shows the change:
- hosts: localhost
gather_facts: false
tasks:
- name: hello, world
debug:
msg: hello, world
When using the json callback, here's what it looks like before the change (for brevity, just the plays section of the output):
{
"play": {
"duration": {
"end": "2023-04-08T11:35:39.694000Z",
"start": "2023-04-08T11:35:39.657056Z"
},
"id": "acde4800-1122-6387-7abd-000000000001",
"name": "localhost",
},
"tasks": [
{
"hosts": {
"localhost": {
"_ansible_no_log": null,
"_ansible_verbose_always": true,
"action": "debug",
"changed": false,
"msg": "hello, world"
}
},
"task": {
"duration": {
"end": "2023-04-08T11:35:39.694000Z",
"start": "2023-04-08T11:35:39.672132Z"
},
"id": "acde4800-1122-6387-7abd-000000000003",
"name": "hello, world",
}
}
]
}
After the change, there is a new path key/value in both the play and the task:
{
"play": {
"duration": {
"end": "2023-04-08T11:35:39.694000Z",
"start": "2023-04-08T11:35:39.657056Z"
},
"id": "acde4800-1122-6387-7abd-000000000001",
"name": "localhost",
"path": "/tmp/plays/hello.yaml:1"
},
"tasks": [
{
"hosts": {
"localhost": {
"_ansible_no_log": null,
"_ansible_verbose_always": true,
"action": "debug",
"changed": false,
"msg": "hello, world"
}
},
"task": {
"duration": {
"end": "2023-04-08T11:35:39.694000Z",
"start": "2023-04-08T11:35:39.672132Z"
},
"id": "acde4800-1122-6387-7abd-000000000003",
"name": "hello, world",
"path": "/tmp/plays/hello.yaml:4"
}
}
]
}
The effect is the same in the jsonl plugin, but the output is squashed into a single line.
Reviewed-by: Adam Miller <admiller@redhat.com>
