Fix all deprecated module_utils imports before ansible-core 2.24 removal
SUMMARY
Fixes all deprecated ansible.module_utils imports across the entire collection that will be removed in ansible-core 2.24.
This PR comprehensively addresses deprecation warnings reported in #686 by updating import statements in 20 files to use the new recommended import paths, and removes 8 unused test utility files that contained deprecated imports.
Deprecated imports replaced:
Deprecated import
Replacement
ansible.module_utils._text
ansible.module_utils.common.text.converters
ansible.module_utils.common._collections_compat
collections.abc
ansible.module_utils.six.moves.shlex_quote
shlex.quote
ansible.module_utils.six.moves.reduce
functools.reduce
ansible.module_utils.six.moves.urllib.parse.urlparse
urllib.parse.urlparse
ansible.module_utils.six.string_types
basestring/str (Python 2/3 compatible)
ansible.module_utils.six.text_type
str
ansible.module_utils.six.PY3
Removed (simplified Python 2/3 conditionals)
ansible.module_utils.six.with_metaclass
Native metaclass= syntax
ansible.module_utils.six.iteritems
dict.items()
Files fixed (20 files, 1 commit per file for easier review):
plugins/action/patch.py
plugins/action/synchronize.py
plugins/callback/cgroup_perf_recap.py
plugins/callback/json.py
plugins/callback/jsonl.py
plugins/callback/profile_roles.py
plugins/callback/profile_tasks.py
plugins/modules/acl.py
plugins/modules/authorized_key.py
plugins/modules/firewalld_info.py
plugins/modules/mount.py
plugins/modules/patch.py
plugins/modules/rhel_rpm_ostree.py
plugins/modules/rpm_ostree_upgrade.py
plugins/modules/seboolean.py
plugins/modules/synchronize.py
plugins/modules/sysctl.py
plugins/shell/csh.py
plugins/shell/fish.py
tests/unit/modules/system/test_mount.py
Files deleted (8 unused test utility files):
These files are dead code - none of them are imported or used anywhere in the test suite or the collection. Removing them also addresses Python 2.7 compatibility concerns raised in code review, as several contained deprecated imports that would be incorrect to fix for Python 2.
tests/unit/compat/builtins.py
tests/unit/mock/loader.py
tests/unit/mock/path.py
tests/unit/mock/procenv.py
tests/unit/mock/vault_helper.py
tests/unit/mock/yaml_helper.py
tests/unit/modules/conftest.py
tests/unit/modules/utils.py
Completeness verified with:
git grep -n -P '_compat|utils._text|utils.six' -- '*.py' | grep -v yml
This command returns no results, confirming all deprecated imports have been replaced.
Notes on Python 2.7 compatibility:
For modules that may run on Python 2.7 managed hosts (e.g., authorized_key.py, synchronize.py, sysctl.py), Python 2/3 compatible fallbacks were used instead of direct Python 3 replacements:
authorized_key.py: try/except ImportError for urllib.parse.urlparse (falls back to urlparse on Python 2)
synchronize.py: try/except ImportError for shlex.quote (falls back to pipes.quote on Python 2)
sysctl.py: uses sys.version_info to set string_types to str on Python 3 (basestring on Python 2)
Also removes corresponding pylint:ansible-bad-import-from entries from tests/sanity/ignore-2.21.txt and tests/sanity/ignore-2.22.txt where applicable.
Fixes#686
ISSUE TYPE
Bugfix Pull Request
ADDITIONAL INFORMATION
Approach:
Each file is fixed in a separate commit for easier code review. The changelog fragment is added in a final commit. Corresponding pylint:ansible-bad-import-from ignore entries in tests/sanity/ignore-2.21.txt and tests/sanity/ignore-2.22.txt are removed in the same commit as the file fix (or the file removal commit).
CI results:
All 59 checks passing (Azure Pipelines sanity, units, lint, Docker, Remote across ansible-core 2.17 through devel, and Zuul ansible/check).
Reviewed-by: Felix Fontein <felix@fontein.de>
Reviewed-by: Pavel Bar
Reviewed-by: Abhijeet Kasurde
(cherry picked from commit 2022c1bd86)
Co-authored-by: centosinfra-prod-github-app[bot] <161850885+centosinfra-prod-github-app[bot]@users.noreply.github.com>
json[l] callback: add parameter to set JSON prettyprint indent level
SUMMARY
Add ANSIBLE_JSON_INDENT parameter to both the json and jsonl callback plugins. The default values are different between the two modules to maintain their existing behavior:
json: indent==4, causing a prettyprint output
jsonl: indent==0, causing a 1-line output
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
ansible.posix.json
ansible.posix.jsonl
ADDITIONAL INFORMATION
One specific use-case that is enabled by this feature: if a user chooses to use the jsonl plugin so that they still get output at the end of each task (vs. only at the end of the play), they may also want human-readable output so that they can monitor the status of their play. For example, setting the jsonl indent level to 4 gives a) output at the end of each task, and b) making that output be both machine readable and human readable.
Using this trivial playbook shows the change:
- hosts: localhost
gather_facts: false
tasks:
- name: hello, world
debug:
msg: hello, world
When using the jsonl callback, here's what one JSON emit looks like before the change:
{"_event":"v2_playbook_on_play_start","_timestamp":"2023-04-08T12:11:48.001806Z","play":{"duration":{"start":"2023-04-08T12:11:48.001383Z"},"id":"acde4800-1122-f32c-94c3-000000000001","name":"localhost"},"tasks":[]}
After the change, setting ANSIBLE_JSON_INDENT to 4, the same output looks like this:
{
"_event":"v2_playbook_on_play_start",
"_timestamp":"2023-04-08T12:12:47.787516Z",
"play":{
"duration":{
"start":"2023-04-08T12:12:47.787164Z"
},
"id":"acde4800-1122-2946-e3e4-000000000001",
"name":"localhost"
},
"tasks":[]
}
Both outputs are suitable for automated processes to parse the machine readable output. The second output has the benefit of being human readable.
Reviewed-by: Adam Miller <admiller@redhat.com>
Reviewed-by: Jeff Squyres
Add ANSIBLE_JSON_INDENT parameter to both the json and jsonl callback
plugins. The default values are different between the two modules to
maintain their existing behavior:
* json: indent==4, causing a prettyprint output
* jsonl: indent==0, causing a 1-line output
One specific use-case that is enabled by this feature: if a user
chooses to use the jsonl plugin so that they still get output at the
end of each task (vs. only at the end of the play), they may also want
human-readable output so that they can monitor the status of their
play. For example, setting the jsonl indent level to 4 gives a)
output at the end of each task, and b) making that output be both
machine readable and human readable.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Add the play and task path info (i.e., filename and line number) to
the JSON that is emitted from the json and jsonl callback plugins,
allowing more accurate post-mortem analysis.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
