Release 5.8.4.

htpasswd: improve documentation on crypt_scheme (#5741)

* htpasswd: improve documentation on crypt_scheme

* htpasswd: formatting in documentation

Co-authored-by: Felix Fontein <felix@fontein.de>

* htpasswd: formatting in documentation

Co-authored-by: Felix Fontein <felix@fontein.de>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 06d72dfed9)

Co-authored-by: bluikko <14869000+bluikko@users.noreply.github.com>

.azure-pipelines/azure-pipelines.yml

@@ -385,7 +385,7 @@ stages:
             - name: ArchLinux
               test: archlinux/3.10
             - name: CentOS Stream 8
-              test: centos-stream8/3.8
+              test: centos-stream8/3.9
           groups:
             - 1
             - 2

CHANGELOG.rst

@@ -6,6 +6,59 @@ Community General Release Notes
 
 This changelog describes changes after version 4.0.0.
 
+v5.8.4
+======
+
+Release Summary
+---------------
+
+Regular bugfix release.
+
+Bugfixes
+--------
+
+- gconftool2 - fix crash that prevents setting a key (https://github.com/ansible-collections/community.general/issues/5591, https://github.com/ansible-collections/community.general/pull/5687).
+- gitlab_group_variables - fix dropping variables accidentally when GitLab introduced new properties (https://github.com/ansible-collections/community.general/pull/5667).
+- gitlab_project_variables - fix dropping variables accidentally when GitLab introduced new properties (https://github.com/ansible-collections/community.general/pull/5667).
+- lxc_container - fix the arguments of the lxc command which broke the creation and cloning of containers (https://github.com/ansible-collections/community.general/issues/5578).
+- opkg - fix issue that ``force=reinstall`` would not reinstall an existing package (https://github.com/ansible-collections/community.general/pull/5705).
+- proxmox_disk - fixed possible issues with redundant ``vmid`` parameter (https://github.com/ansible-collections/community.general/issues/5492, https://github.com/ansible-collections/community.general/pull/5672).
+- proxmox_nic - fixed possible issues with redundant ``vmid`` parameter (https://github.com/ansible-collections/community.general/issues/5492, https://github.com/ansible-collections/community.general/pull/5672).
+- unixy callback plugin - fix typo introduced when updating to use Ansible's configuration manager for handling options (https://github.com/ansible-collections/community.general/issues/5600).
+
+v5.8.3
+======
+
+Release Summary
+---------------
+
+Regular bugfix release.
+
+Bugfixes
+--------
+
+- keycloak_client_rolemapping - calculate ``proposed`` and ``after`` return values properly (https://github.com/ansible-collections/community.general/pull/5619).
+- keycloak_client_rolemapping - remove only listed mappings with ``state=absent`` (https://github.com/ansible-collections/community.general/pull/5619).
+- proxmox inventory plugin - handle tags delimited by semicolon instead of comma, which happens from Proxmox 7.3 on (https://github.com/ansible-collections/community.general/pull/5602).
+- vdo - now uses ``yaml.safe_load()`` to parse command output instead of the deprecated ``yaml.load()`` which is potentially unsafe. Using ``yaml.load()`` without explicitely setting a ``Loader=`` is also an error in pyYAML 6.0 (https://github.com/ansible-collections/community.general/pull/5632).
+- vmadm - fix for index out of range error in ``get_vm_uuid`` (https://github.com/ansible-collections/community.general/pull/5628).
+
+v5.8.2
+======
+
+Release Summary
+---------------
+
+Regular bugfix release.
+
+Bugfixes
+--------
+
+- chroot connection plugin - add ``inventory_hostname`` to vars under ``remote_addr``. This is needed for compatibility with ansible-core 2.13 (https://github.com/ansible-collections/community.general/pull/5570).
+- cmd_runner module utils - fixed bug when handling default cases in ``cmd_runner_fmt.as_map()`` (https://github.com/ansible-collections/community.general/pull/5538).
+- cmd_runner module utils - formatting arguments ``cmd_runner_fmt.as_fixed()`` was expecting an non-existing argument (https://github.com/ansible-collections/community.general/pull/5538).
+- unixy callback plugin - fix plugin to work with ansible-core 2.14 by using Ansible's configuration manager for handling options (https://github.com/ansible-collections/community.general/issues/5600).
+
 v5.8.1
 ======
 

CONTRIBUTING.md

@@ -31,7 +31,7 @@ Also, consider taking up a valuable, reviewed, but abandoned pull request which
 * Try committing your changes with an informative but short commit message.
 * Do not squash your commits and force-push to your branch if not needed. Reviews of your pull request are much easier with individual commits to comprehend the pull request history. All commits of your pull request branch will be squashed into one commit by GitHub upon merge.
 * Do not add merge commits to your PR. The bot will complain and you will have to rebase ([instructions for rebasing](https://docs.ansible.com/ansible/latest/dev_guide/developing_rebasing.html)) to remove them before your PR can be merged. To avoid that git automatically does merges during pulls, you can configure it to do rebases instead by running `git config pull.rebase true` inside the repository checkout.
-* Make sure your PR includes a [changelog fragment](https://docs.ansible.com/ansible/devel/community/development_process.html#changelogs-how-to). (You must not include a fragment for new modules or new plugins, except for test and filter plugins. Also you shouldn't include one for docs-only changes. If you're not sure, simply don't include one, we'll tell you whether one is needed or not :) )
+* Make sure your PR includes a [changelog fragment](https://docs.ansible.com/ansible/devel/community/development_process.html#creating-changelog-fragments). (You must not include a fragment for new modules or new plugins, except for test and filter plugins. Also you shouldn't include one for docs-only changes. If you're not sure, simply don't include one, we'll tell you whether one is needed or not :) )
 * Avoid reformatting unrelated parts of the codebase in your PR. These types of changes will likely be requested for reversion, create additional work for reviewers, and may cause approval to be delayed.
 
 You can also read [our Quick-start development guide](https://github.com/ansible/community-docs/blob/main/create_pr_quick_start_guide.rst).

changelogs/changelog.yaml

@@ -1352,3 +1352,71 @@ releases:
     - 5543-dependent-template.yml
     - fqcn-warnings.yml
     release_date: '2022-11-15'
+  5.8.2:
+    changes:
+      bugfixes:
+      - chroot connection plugin - add ``inventory_hostname`` to vars under ``remote_addr``.
+        This is needed for compatibility with ansible-core 2.13 (https://github.com/ansible-collections/community.general/pull/5570).
+      - cmd_runner module utils - fixed bug when handling default cases in ``cmd_runner_fmt.as_map()``
+        (https://github.com/ansible-collections/community.general/pull/5538).
+      - cmd_runner module utils - formatting arguments ``cmd_runner_fmt.as_fixed()``
+        was expecting an non-existing argument (https://github.com/ansible-collections/community.general/pull/5538).
+      - unixy callback plugin - fix plugin to work with ansible-core 2.14 by using
+        Ansible's configuration manager for handling options (https://github.com/ansible-collections/community.general/issues/5600).
+      release_summary: Regular bugfix release.
+    fragments:
+    - 5.8.2.yml
+    - 5538-cmd-runner-as-fixed.yml
+    - 5570-chroot-plugin-fix-default-inventory_hostname.yml
+    - 5601-unixy-callback-use-config-manager.yml
+    release_date: '2022-11-29'
+  5.8.3:
+    changes:
+      bugfixes:
+      - keycloak_client_rolemapping - calculate ``proposed`` and ``after`` return
+        values properly (https://github.com/ansible-collections/community.general/pull/5619).
+      - keycloak_client_rolemapping - remove only listed mappings with ``state=absent``
+        (https://github.com/ansible-collections/community.general/pull/5619).
+      - proxmox inventory plugin - handle tags delimited by semicolon instead of comma,
+        which happens from Proxmox 7.3 on (https://github.com/ansible-collections/community.general/pull/5602).
+      - vdo - now uses ``yaml.safe_load()`` to parse command output instead of the
+        deprecated ``yaml.load()`` which is potentially unsafe. Using ``yaml.load()``
+        without explicitely setting a ``Loader=`` is also an error in pyYAML 6.0 (https://github.com/ansible-collections/community.general/pull/5632).
+      - vmadm - fix for index out of range error in ``get_vm_uuid`` (https://github.com/ansible-collections/community.general/pull/5628).
+      release_summary: Regular bugfix release.
+    fragments:
+    - 5.8.3.yml
+    - 5602-proxmox-tags.yml
+    - 5619-keycloak-improvements.yml
+    - 5628-fix-vmadm-off-by-one.yml
+    - 5632-vdo-Use-yaml-safe-load-instead-of-yaml-load.yml
+    release_date: '2022-12-05'
+  5.8.4:
+    changes:
+      bugfixes:
+      - gconftool2 - fix crash that prevents setting a key (https://github.com/ansible-collections/community.general/issues/5591,
+        https://github.com/ansible-collections/community.general/pull/5687).
+      - gitlab_group_variables - fix dropping variables accidentally when GitLab introduced
+        new properties (https://github.com/ansible-collections/community.general/pull/5667).
+      - gitlab_project_variables - fix dropping variables accidentally when GitLab
+        introduced new properties (https://github.com/ansible-collections/community.general/pull/5667).
+      - lxc_container - fix the arguments of the lxc command which broke the creation
+        and cloning of containers (https://github.com/ansible-collections/community.general/issues/5578).
+      - opkg - fix issue that ``force=reinstall`` would not reinstall an existing
+        package (https://github.com/ansible-collections/community.general/pull/5705).
+      - proxmox_disk - fixed possible issues with redundant ``vmid`` parameter (https://github.com/ansible-collections/community.general/issues/5492,
+        https://github.com/ansible-collections/community.general/pull/5672).
+      - proxmox_nic - fixed possible issues with redundant ``vmid`` parameter (https://github.com/ansible-collections/community.general/issues/5492,
+        https://github.com/ansible-collections/community.general/pull/5672).
+      - unixy callback plugin - fix typo introduced when updating to use Ansible's
+        configuration manager for handling options (https://github.com/ansible-collections/community.general/issues/5600).
+      release_summary: Regular bugfix release.
+    fragments:
+    - 5.8.4.yml
+    - 5659-fix-lxc_container-command.yml
+    - 5666-gitlab-variables.yml
+    - 5672-proxmox.yml
+    - 5687-gconftool2.yml
+    - 5705-opkg-fix-force-reinstall.yml
+    - 5744-unixy-callback-fix-config-manager-typo.yml
+    release_date: '2023-01-04'

galaxy.yml

@@ -5,7 +5,7 @@
 
 namespace: community
 name: general
-version: 5.8.1
+version: 5.8.4
 readme: README.md
 authors:
   - Ansible (https://github.com/ansible)

plugins/callback/unixy.py

@@ -63,7 +63,7 @@ class CallbackModule(CallbackModule_default):
 
     def _preprocess_result(self, result):
         self.delegated_vars = result._result.get('_ansible_delegated_vars', None)
-        self._handle_exception(result._result, use_stderr=self.display_failed_stderr)
+        self._handle_exception(result._result, use_stderr=self.get_option('display_failed_stderr'))
         self._handle_warnings(result._result)
 
     def _process_result_output(self, result, msg):
@@ -109,7 +109,7 @@ class CallbackModule(CallbackModule_default):
         self._display.display(msg)
 
     def v2_runner_on_skipped(self, result, ignore_errors=False):
-        if self.display_skipped_hosts:
+        if self.get_option('display_skipped_hosts'):
             self._preprocess_result(result)
             display_color = C.COLOR_SKIP
             msg = "skipped"
@@ -128,7 +128,7 @@ class CallbackModule(CallbackModule_default):
             msg += " | item: %s" % (item_value,)
 
         task_result = self._process_result_output(result, msg)
-        self._display.display("  " + task_result, display_color, stderr=self.display_failed_stderr)
+        self._display.display("  " + task_result, display_color, stderr=self.get_option('display_failed_stderr'))
 
     def v2_runner_on_ok(self, result, msg="ok", display_color=C.COLOR_OK):
         self._preprocess_result(result)
@@ -142,7 +142,7 @@ class CallbackModule(CallbackModule_default):
             display_color = C.COLOR_CHANGED
             task_result = self._process_result_output(result, msg)
             self._display.display("  " + task_result, display_color)
-        elif self.display_ok_hosts:
+        elif self.get_option('display_ok_hosts'):
             task_result = self._process_result_output(result, msg)
             self._display.display("  " + task_result, display_color)
 
@@ -162,7 +162,7 @@ class CallbackModule(CallbackModule_default):
         display_color = C.COLOR_UNREACHABLE
         task_result = self._process_result_output(result, msg)
 
-        self._display.display("  " + task_result, display_color, stderr=self.display_failed_stderr)
+        self._display.display("  " + task_result, display_color, stderr=self.get_option('display_failed_stderr'))
 
     def v2_on_file_diff(self, result):
         if result._task.loop and 'results' in result._result:
@@ -205,7 +205,7 @@ class CallbackModule(CallbackModule_default):
                 colorize(u'ignored', t['ignored'], None)),
                 log_only=True
             )
-        if stats.custom and self.show_custom_stats:
+        if stats.custom and self.get_option('show_custom_stats'):
             self._display.banner("CUSTOM STATS: ")
             # per host
             # TODO: come up with 'pretty format'

plugins/connection/chroot.py

@@ -22,6 +22,7 @@ DOCUMENTATION = '''
             - The path of the chroot you want to access.
         default: inventory_hostname
         vars:
+            - name: inventory_hostname
             - name: ansible_host
       executable:
         description:

plugins/inventory/proxmox.py

@@ -410,7 +410,7 @@ class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):
                     stripped_value = value.strip()
                     if stripped_value:
                         parsed_key = key + "_parsed"
-                        properties[parsed_key] = [tag.strip() for tag in stripped_value.split(",")]
+                        properties[parsed_key] = [tag.strip() for tag in stripped_value.replace(',', ';').split(";")]
 
                 # The first field in the agent string tells you whether the agent is enabled
                 # the rest of the comma separated string is extra config for the agent.

plugins/module_utils/cmd_runner.py

@@ -88,9 +88,10 @@ class FormatError(CmdRunnerException):
 
 
 class _ArgFormat(object):
-    def __init__(self, func, ignore_none=None):
+    def __init__(self, func, ignore_none=None, ignore_missing_value=False):
         self.func = func
         self.ignore_none = ignore_none
+        self.ignore_missing_value = ignore_missing_value
 
     def __call__(self, value, ctx_ignore_none):
         ignore_none = self.ignore_none if self.ignore_none is not None else ctx_ignore_none
@@ -127,7 +128,7 @@ class _Format(object):
 
     @staticmethod
     def as_fixed(args):
-        return _ArgFormat(lambda value: _ensure_list(args), ignore_none=False)
+        return _ArgFormat(lambda value: _ensure_list(args), ignore_none=False, ignore_missing_value=True)
 
     @staticmethod
     def as_func(func, ignore_none=None):
@@ -135,14 +136,15 @@ class _Format(object):
 
     @staticmethod
     def as_map(_map, default=None, ignore_none=None):
+        if default is None:
+            default = []
         return _ArgFormat(lambda value: _ensure_list(_map.get(value, default)), ignore_none=ignore_none)
 
     @staticmethod
     def as_default_type(_type, arg="", ignore_none=None):
         fmt = _Format
         if _type == "dict":
-            return fmt.as_func(lambda d: ["--{0}={1}".format(*a) for a in iteritems(d)],
-                               ignore_none=ignore_none)
+            return fmt.as_func(lambda d: ["--{0}={1}".format(*a) for a in iteritems(d)], ignore_none=ignore_none)
         if _type == "list":
             return fmt.as_func(lambda value: ["--{0}".format(x) for x in value], ignore_none=ignore_none)
         if _type == "bool":
@@ -261,10 +263,13 @@ class _CmdRunnerContext(object):
         for arg_name in self.args_order:
             value = None
             try:
-                value = named_args[arg_name]
+                if arg_name in named_args:
+                    value = named_args[arg_name]
+                elif not runner.arg_formats[arg_name].ignore_missing_value:
+                    raise MissingArgumentValue(self.args_order, arg_name)
                 self.cmd.extend(runner.arg_formats[arg_name](value, ctx_ignore_none=self.ignore_value_none))
-            except KeyError:
-                raise MissingArgumentValue(self.args_order, arg_name)
+            except MissingArgumentValue:
+                raise
             except Exception as e:
                 raise FormatError(arg_name, value, runner.arg_formats[arg_name], e)
 

plugins/module_utils/gitlab.py

@@ -110,3 +110,14 @@ def gitlab_authentication(module):
             GitLab remove Session API now that private tokens are removed from user API endpoints since version 10.2." % to_native(e))
 
     return gitlab_instance
+
+
+def filter_returned_variables(gitlab_variables):
+    # pop properties we don't know
+    existing_variables = [dict(x.attributes) for x in gitlab_variables]
+    KNOWN = ['key', 'value', 'masked', 'protected', 'variable_type', 'environment_scope']
+    for item in existing_variables:
+        for key in list(item.keys()):
+            if key not in KNOWN:
+                item.pop(key)
+    return existing_variables

plugins/module_utils/identity/keycloak/keycloak.py

@@ -606,7 +606,7 @@ class KeycloakAPI(object):
         """
         available_rolemappings_url = URL_CLIENT_GROUP_ROLEMAPPINGS.format(url=self.baseurl, realm=realm, id=gid, client=cid)
         try:
-            open_url(available_rolemappings_url, method="DELETE", http_agent=self.http_agent, headers=self.restheaders,
+            open_url(available_rolemappings_url, method="DELETE", http_agent=self.http_agent, headers=self.restheaders, data=json.dumps(role_rep),
                      validate_certs=self.validate_certs, timeout=self.connection_timeout)
         except Exception as e:
             self.module.fail_json(msg="Could not delete available rolemappings for client %s in group %s, realm %s: %s"

plugins/modules/cloud/lxc/lxc_container.py

@@ -677,7 +677,7 @@ class LxcContainerManagement(object):
 
         false_values = BOOLEANS_FALSE.union([None, ''])
         result = dict(
-            (k, v)
+            (v, self.module.params[k])
             for k, v in variables.items()
             if self.module.params[k] not in false_values
         )

plugins/modules/cloud/misc/proxmox_disk.py

@@ -699,7 +699,7 @@ def main():
                 module.exit_json(changed=False, vmid=vmid, msg='Disk %s already detached in VM %s' % (disk, vmid))
             if disk not in vm_config:
                 module.exit_json(changed=False, vmid=vmid, msg="Disk %s not present in VM %s config" % (disk, vmid))
-            proxmox.proxmox_api.nodes(vm['node']).qemu(vmid).unlink.put(vmid=vmid, idlist=disk, force=0)
+            proxmox.proxmox_api.nodes(vm['node']).qemu(vmid).unlink.put(idlist=disk, force=0)
             module.exit_json(changed=True, vmid=vmid, msg="Disk %s detached from VM %s" % (disk, vmid))
         except Exception as e:
             module.fail_json(msg="Failed to detach disk %s from VM %s with exception: %s" % (disk, vmid, str(e)))
@@ -734,7 +734,7 @@ def main():
         try:
             if disk not in vm_config:
                 module.exit_json(changed=False, vmid=vmid, msg="Disk %s is already absent in VM %s" % (disk, vmid))
-            proxmox.proxmox_api.nodes(vm['node']).qemu(vmid).unlink.put(vmid=vmid, idlist=disk, force=1)
+            proxmox.proxmox_api.nodes(vm['node']).qemu(vmid).unlink.put(idlist=disk, force=1)
             module.exit_json(changed=True, vmid=vmid, msg="Disk %s removed from VM %s" % (disk, vmid))
         except Exception as e:
             module.fail_json(vmid=vmid, msg='Unable to remove disk %s from VM %s: %s' % (disk, vmid, str(e)))

plugins/modules/cloud/misc/proxmox_nic.py

@@ -223,7 +223,7 @@ class ProxmoxNicAnsible(ProxmoxAnsible):
 
         if interface in vminfo:
             if not self.module.check_mode:
-                self.proxmox_api.nodes(vm['node']).qemu(vmid).config.set(vmid=vmid, delete=interface)
+                self.proxmox_api.nodes(vm['node']).qemu(vmid).config.set(delete=interface)
             return True
 
         return False

plugins/modules/cloud/smartos/vmadm.py

@@ -438,7 +438,7 @@ def get_vm_prop(module, uuid, prop):
 def get_vm_uuid(module, alias):
     # Lookup the uuid that goes with the given alias.
     # Returns the uuid or '' if not found.
-    cmd = [module.vmadm, 'lookup', '-j', '-o', 'uuid', 'alias={1}'.format(alias)]
+    cmd = [module.vmadm, 'lookup', '-j', '-o', 'uuid', 'alias={0}'.format(alias)]
 
     (rc, stdout, stderr) = module.run_command(cmd)
 

plugins/modules/identity/keycloak/keycloak_client_rolemapping.py

@@ -295,7 +295,7 @@ def main():
     assigned_roles_before = kc.get_client_group_composite_rolemappings(gid, cid, realm=realm)
 
     result['existing'] = assigned_roles_before
-    result['proposed'] = roles
+    result['proposed'] = list(assigned_roles_before) if assigned_roles_before else []
 
     update_roles = []
     for role_index, role in enumerate(roles, start=0):
@@ -307,6 +307,7 @@ def main():
                         'id': role['id'],
                         'name': role['name'],
                     })
+                    result['proposed'].append(available_role)
         # Fetch roles to remove if state absent
         else:
             for assigned_role in assigned_roles_before:
@@ -315,13 +316,15 @@ def main():
                         'id': role['id'],
                         'name': role['name'],
                     })
+                    if assigned_role in result['proposed']:  # Handle double removal
+                        result['proposed'].remove(assigned_role)
 
     if len(update_roles):
         if state == 'present':
             # Assign roles
             result['changed'] = True
             if module._diff:
-                result['diff'] = dict(before=assigned_roles_before, after=update_roles)
+                result['diff'] = dict(before=assigned_roles_before, after=result['proposed'])
             if module.check_mode:
                 module.exit_json(**result)
             kc.add_group_rolemapping(gid, cid, update_roles, realm=realm)
@@ -333,7 +336,7 @@ def main():
             # Remove mapping of role
             result['changed'] = True
             if module._diff:
-                result['diff'] = dict(before=assigned_roles_before, after=update_roles)
+                result['diff'] = dict(before=assigned_roles_before, after=result['proposed'])
             if module.check_mode:
                 module.exit_json(**result)
             kc.delete_group_rolemapping(gid, cid, update_roles, realm=realm)
@@ -344,7 +347,7 @@ def main():
     # Do nothing
     else:
         result['changed'] = False
-        result['msg'] = 'Nothing to do, roles %s are correctly mapped with group %s.' % (roles, group_name)
+        result['msg'] = 'Nothing to do, roles %s are %s with group %s.' % (roles, 'mapped' if state == 'present' else 'not mapped', group_name)
         module.exit_json(**result)
 
 

plugins/modules/identity/keycloak/keycloak_realm.py

@@ -519,6 +519,7 @@ EXAMPLES = '''
     auth_username: USERNAME
     auth_password: PASSWORD
     id: realm
+    realm: realm
     state: present
 
 - name: Delete a Keycloak realm

plugins/modules/packaging/os/opkg.py

@@ -154,7 +154,7 @@ def install_packages(module, opkg_path, packages):
     install_c = 0
 
     for package in packages:
-        if query_package(module, opkg_path, package):
+        if query_package(module, opkg_path, package) and (force != '--force-reinstall'):
             continue
 
         rc, out, err = module.run_command("%s install %s %s" % (opkg_path, force, package))

plugins/modules/packaging/os/redhat_subscription.py

@@ -34,15 +34,15 @@ options:
         type: str
     username:
         description:
-            - access.redhat.com or Sat6 username
+            - access.redhat.com or Red Hat Satellite or Katello username
         type: str
     password:
         description:
-            - access.redhat.com or Sat6 password
+            - access.redhat.com or Red Hat Satellite or Katello password
         type: str
     server_hostname:
         description:
-            - Specify an alternative Red Hat Subscription Management or Sat6 server
+            - Specify an alternative Red Hat Subscription Management or Red Hat Satellite or Katello server
         type: str
     server_insecure:
         description:
@@ -50,12 +50,12 @@ options:
         type: str
     server_prefix:
         description:
-            - Specify the prefix when registering to the Red Hat Subscription Management or Sat6 server.
+            - Specify the prefix when registering to the Red Hat Subscription Management or Red Hat Satellite or Katello server.
         type: str
         version_added: 3.3.0
     server_port:
         description:
-            - Specify the port when registering to the Red Hat Subscription Management or Sat6 server.
+            - Specify the port when registering to the Red Hat Subscription Management or Red Hat Satellite or Katello server.
         type: str
         version_added: 3.3.0
     rhsm_baseurl:
@@ -98,7 +98,7 @@ options:
         type: str
     environment:
         description:
-            - Register with a specific environment in the destination org. Used with Red Hat Satellite 6.x or Katello
+            - Register with a specific environment in the destination org. Used with Red Hat Satellite or Katello
         type: str
     pool:
         description:
@@ -229,7 +229,7 @@ EXAMPLES = '''
     org_id: 222333444
     pool: '^Red Hat Enterprise Server$'
 
-- name: Register as user credentials into given environment (against Red Hat Satellite 6.x), and auto-subscribe.
+- name: Register as user credentials into given environment (against Red Hat Satellite or Katello), and auto-subscribe.
   community.general.redhat_subscription:
     state: present
     username: joe_user
@@ -397,7 +397,9 @@ class Rhsm(RegistrationBase):
                  rhsm_baseurl, server_insecure, server_hostname, server_proxy_hostname,
                  server_proxy_port, server_proxy_user, server_proxy_password, release):
         '''
-            Register the current system to the provided RHSM or Sat6 server
+            Register the current system to the provided RHSM or Red Hat Satellite
+            or Katello server
+
             Raises:
               * Exception - if error occurs while running command
         '''

plugins/modules/source_control/github/github_release.py

@@ -108,17 +108,8 @@ EXAMPLES = '''
 '''
 
 RETURN = '''
-create_release:
-    description:
-    - Version of the created release
-    - "For Ansible version 2.5 and later, if specified release version already exists, then State is unchanged"
-    - "For Ansible versions prior to 2.5, if specified release version already exists, then State is skipped"
-    type: str
-    returned: success
-    sample: 1.1.0
-
-latest_release:
-    description: Version of the latest release
+tag:
+    description: Version of the created/latest release.
     type: str
     returned: success
     sample: 1.1.0

plugins/modules/source_control/gitlab/gitlab_group_variable.py

@@ -165,7 +165,7 @@ from ansible.module_utils.six import string_types
 from ansible.module_utils.six import integer_types
 
 from ansible_collections.community.general.plugins.module_utils.gitlab import (
-    auth_argument_spec, gitlab_authentication, ensure_gitlab_package
+    auth_argument_spec, gitlab_authentication, ensure_gitlab_package, filter_returned_variables
 )
 
 
@@ -296,11 +296,7 @@ def native_python_main(this_gitlab, purge, requested_variables, state, module):
     before = [x.attributes for x in gitlab_keys]
 
     gitlab_keys = this_gitlab.list_all_group_variables()
-    existing_variables = [x.attributes for x in gitlab_keys]
-
-    # preprocessing:filter out and enrich before compare
-    for item in existing_variables:
-        item.pop('group_id')
+    existing_variables = filter_returned_variables(gitlab_keys)
 
     for item in requested_variables:
         item['key'] = item.pop('name')
@@ -331,9 +327,7 @@ def native_python_main(this_gitlab, purge, requested_variables, state, module):
         if purge:
             # refetch and filter
             gitlab_keys = this_gitlab.list_all_group_variables()
-            existing_variables = [x.attributes for x in gitlab_keys]
-            for item in existing_variables:
-                item.pop('group_id')
+            existing_variables = filter_returned_variables(gitlab_keys)
 
             remove = [x for x in existing_variables if x not in requested_variables]
             for item in remove:

plugins/modules/source_control/gitlab/gitlab_project_variable.py

@@ -189,7 +189,7 @@ except Exception:
     HAS_GITLAB_PACKAGE = False
 
 from ansible_collections.community.general.plugins.module_utils.gitlab import (
-    auth_argument_spec, gitlab_authentication, ensure_gitlab_package
+    auth_argument_spec, gitlab_authentication, ensure_gitlab_package, filter_returned_variables
 )
 
 
@@ -255,9 +255,11 @@ class GitlabProjectVariables(object):
             return True
 
         var = {
-            "key": var_obj.get('key'), "value": var_obj.get('value'),
-            "masked": var_obj.get('masked'), "protected": var_obj.get('protected'),
-            "variable_type": var_obj.get('variable_type')
+            "key": var_obj.get('key'),
+            "value": var_obj.get('value'),
+            "masked": var_obj.get('masked'),
+            "protected": var_obj.get('protected'),
+            "variable_type": var_obj.get('variable_type'),
         }
 
         if var_obj.get('environment_scope') is not None:
@@ -319,12 +321,9 @@ def native_python_main(this_gitlab, purge, requested_variables, state, module):
     before = [x.attributes for x in gitlab_keys]
 
     gitlab_keys = this_gitlab.list_all_project_variables()
-    existing_variables = [x.attributes for x in gitlab_keys]
-
-    # preprocessing:filter out and enrich before compare
-    for item in existing_variables:
-        item.pop('project_id')
+    existing_variables = filter_returned_variables(gitlab_keys)
 
+    # filter out and enrich before compare
     for item in requested_variables:
         item['key'] = item.pop('name')
         item['value'] = str(item.get('value'))
@@ -354,9 +353,7 @@ def native_python_main(this_gitlab, purge, requested_variables, state, module):
         if purge:
             # refetch and filter
             gitlab_keys = this_gitlab.list_all_project_variables()
-            existing_variables = [x.attributes for x in gitlab_keys]
-            for item in existing_variables:
-                item.pop('project_id')
+            existing_variables = filter_returned_variables(gitlab_keys)
 
             remove = [x for x in existing_variables if x not in requested_variables]
             for item in remove:
@@ -409,7 +406,7 @@ def main():
             masked=dict(type='bool', default=False),
             protected=dict(type='bool', default=False),
             environment_scope=dict(type='str', default='*'),
-            variable_type=dict(type='str', default='env_var', choices=["env_var", "file"])
+            variable_type=dict(type='str', default='env_var', choices=["env_var", "file"]),
         )),
         state=dict(type='str', default="present", choices=["absent", "present"]),
     )

plugins/modules/system/gconftool2.py

@@ -125,7 +125,7 @@ class GConf2Preference(object):
             elif call_type == 'set':
                 cmd.extend(direct)
                 cmd.extend(config_source)
-                cmd.extend(["--type", self.value_type, "--{3}".format(call_type), self.key, self.value])
+                cmd.extend(["--type", self.value_type, "--{0}".format(call_type), self.key, self.value])
             elif call_type == 'unset':
                 cmd.extend(["--unset", self.key])
 

plugins/modules/system/vdo.py

@@ -332,7 +332,7 @@ def inventory_vdos(module, vdocmd):
     if rc != 0:
         module.fail_json(msg="Inventorying VDOs failed: %s" % vdostatusout, rc=rc, err=err)
 
-    vdostatusyaml = yaml.load(vdostatusout)
+    vdostatusyaml = yaml.safe_load(vdostatusout)
     if vdostatusyaml is None:
         return vdolist
 
@@ -548,7 +548,7 @@ def run_module():
     # Modify the current parameters of a VDO that exists.
     if desiredvdo in vdolist and state == 'present':
         rc, vdostatusoutput, err = module.run_command([vdocmd, "status"])
-        vdostatusyaml = yaml.load(vdostatusoutput)
+        vdostatusyaml = yaml.safe_load(vdostatusoutput)
 
         # An empty dictionary to contain dictionaries of VDO statistics
         processedvdos = {}

plugins/modules/web_infrastructure/htpasswd.py

@@ -41,9 +41,12 @@ options:
     description:
       - Encryption scheme to be used.  As well as the four choices listed
         here, you can also use any other hash supported by passlib, such as
-        md5_crypt and sha256_crypt, which are linux passwd hashes.  If you
-        do so the password file will not be compatible with Apache or Nginx
-      - 'Some of the available choices might be: C(apr_md5_crypt), C(des_crypt), C(ldap_sha1), C(plaintext)'
+        C(portable_apache22) and C(host_apache24); or C(md5_crypt) and C(sha256_crypt),
+        which are Linux passwd hashes.  Only some schemes in addition to
+        the four choices below will be compatible with Apache or Nginx, and
+        supported schemes depend on passlib version and its dependencies.
+      - See U(https://passlib.readthedocs.io/en/stable/lib/passlib.apache.html#passlib.apache.HtpasswdFile) parameter C(default_scheme).
+      - 'Some of the available choices might be: C(apr_md5_crypt), C(des_crypt), C(ldap_sha1), C(plaintext).'
   state:
     type: str
     required: false

tests/integration/targets/copr/tasks/main.yml

@@ -3,14 +3,27 @@
 # GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
 # SPDX-License-Identifier: GPL-3.0-or-later
 
-- when: ansible_distribution == 'Fedora'
+- when:
+    # Fedora or RHEL >= 8
+    # This module requires the dnf module which is not available on RHEL 7.
+    - >
+      ansible_distribution == 'Fedora'
+      or (ansible_os_family == 'RedHat' and ansible_distribution != 'Fedora'
+          and ansible_distribution_major_version | int >= 8)
+    # The copr module imports dnf which is only available for the system Python
+    # interpreter.
+    - >
+      not (ansible_distribution == 'CentOS' and
+           ansible_distribution_major_version | int == 8 and not
+           ansible_python_version.startswith('3.6'))
   block:
+  - debug: var=copr_chroot
   - name: enable copr project
     copr:
       host: copr.fedorainfracloud.org
       state: enabled
-      name: '@copr/integration_tests'
-      chroot: fedora-rawhide-x86_64
+      name: '{{ copr_fullname }}'
+      chroot: "{{ copr_chroot }}"
     register: result
 
   - name: assert that the copr project was enabled
@@ -24,8 +37,8 @@
     check_mode: yes
     copr:
       state: enabled
-      name: '@copr/integration_tests'
-      chroot: fedora-rawhide-x86_64
+      name: '{{ copr_fullname }}'
+      chroot: '{{ copr_chroot }}'
     register: result
 
   - name: assert that the copr project was enabled
@@ -34,10 +47,53 @@
         - result is not changed
         - result.msg == 'enabled'
 
+  - name: Ensure the repo is installed and enabled | slurp
+    register: result
+    ansible.builtin.slurp:
+      src: "{{ copr_repofile }}"
+
+  - name: Ensure the repo is installed and enabled
+    vars:
+      content: "{{ result.content | b64decode }}"
+      _baseurl: "{{ 'https://download.copr.fedorainfracloud.org/results/gotmax23/community.general.copr_integration_tests' | regex_escape }}"
+      baseurl: "{{ content | regex_search('baseurl=' ~ _baseurl) }}"
+    block:
+      - ansible.builtin.debug:
+          var: content
+      - ansible.builtin.debug:
+          var: baseurl
+      - name: Ensure the repo is installed and enabled
+        ansible.builtin.assert:
+          that:
+            - "'enabled=1' in content"
+            - baseurl | length > 0
+
+  - name: Install test package from Copr
+    when:
+      # Copr does not build new packages for EOL Fedoras.
+      - >
+        not (ansible_distribution == 'Fedora' and
+             ansible_distribution_major_version | int < 35)
+    block:
+      - name: install test package from the copr
+        ansible.builtin.package:
+          update_cache: true
+          name: copr-module-integration-dummy-package
+
+      - name: uninstall test package
+        register: result
+        ansible.builtin.package:
+          name: copr-module-integration-dummy-package
+          state: absent
+
+      - name: check uninstall test package
+        ansible.builtin.assert:
+          that: result.changed | bool
+
   - name: remove copr project
     copr:
       state: absent
-      name: '@copr/integration_tests'
+      name: '{{ copr_fullname }}'
     register: result
 
   - name: assert that the copr project was removed
@@ -46,11 +102,20 @@
         - 'result is changed'
         - result.msg == 'absent'
 
+  - name: Ensure the repo file was removed | stat
+    register: result
+    ansible.builtin.stat:
+      dest: "{{ copr_repofile }}"
+
+  - name: Ensure the repo file was removed
+    ansible.builtin.assert:
+      that: not result.stat.exists | bool
+
   - name: disable copr project
     copr:
       state: disabled
-      name: '@copr/integration_tests'
-      chroot: fedora-rawhide-x86_64
+      name: '{{ copr_fullname }}'
+      chroot: '{{ copr_chroot }}'
     register: result
 
   - name: assert that the copr project was disabled
@@ -59,10 +124,37 @@
         - 'result is changed'
         - result.msg == 'disabled'
 
+  - name: Ensure the repo is installed but disabled | slurp
+    register: result
+    ansible.builtin.slurp:
+      src: "{{ copr_repofile }}"
+
+  - name: Ensure the repo is installed but disabled
+    vars:
+      content: "{{ result.content | b64decode }}"
+      _baseurl: "{{ 'https://download.copr.fedorainfracloud.org/results/gotmax23/community.general.copr_integration_tests' | regex_escape }}"
+      baseurl: "{{ content | regex_search('baseurl=' ~ _baseurl) }}"
+    block:
+      - ansible.builtin.debug:
+          var: content
+      - ansible.builtin.debug:
+          var: baseurl
+      - name: Ensure the repo is installed but disabled
+        ansible.builtin.assert:
+          that:
+            - "'enabled=0' in content"
+            - baseurl | length > 0
+
   always:
     - name: clean up
+      ignore_errors: true
       copr:
         host: copr.fedorainfracloud.org
         state: absent
-        name: '@copr/integration_tests'
-        chroot: fedora-rawhide-x86_64
+        name: '{{ copr_fullname }}'
+        chroot: '{{ copr_chroot }}'
+
+    - name: cleanup test package
+      ansible.builtin.package:
+        name: copr-module-integration-dummy-package
+        state: absent

tests/integration/targets/copr/vars/main.yml

@@ -0,0 +1,15 @@
+# Copyright (c) 2022 Maxwell G <gotmax@e.email>
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or
+# https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+---
+copr_host: copr.fedorainfracloud.org
+copr_namespace: gotmax23
+copr_name: community.general.copr_integration_tests
+copr_fullname: '{{ copr_namespace }}/{{ copr_name }}'
+copr_repofile: '/etc/yum.repos.d/_copr:{{ copr_host }}:{{ copr_namespace }}:{{ copr_name }}.repo'
+
+# TODO: Fix chroot autodetection so this isn't necessary
+_copr_chroot_fedora: "fedora-rawhide-x86_64"
+_copr_chroot_rhelish: "epel-{{ ansible_distribution_major_version }}-x86_64"
+copr_chroot: "{{ _copr_chroot_fedora if ansible_distribution == 'Fedora' else _copr_chroot_rhelish }}"

tests/integration/targets/pipx_info/tasks/main.yml

@@ -56,7 +56,8 @@
       - info_all_deps.application|length == 1
       - info_all_deps.application[0].name == "tox"
       - "'version' in info_all_deps.application[0]"
-      - info_all_deps.application[0].dependencies == ["virtualenv"]
+      - info_all_deps.application[0].dependencies == ["chardet", "virtualenv"]
+        or info_all_deps.application[0].dependencies == ["virtualenv"]
       - "'injected' not in info_all.application[0]"
 
       - info_tox.application == info_all_deps.application

tests/unit/plugins/module_utils/test_cmd_runner.py

@@ -240,6 +240,60 @@ TC_RUNNER = dict(
             ),
         ),
     ),
+    aa_bb_fixed=(
+        dict(
+            args_bundle=dict(
+                aa=dict(type="int", value=11, fmt_func=fmt.as_opt_eq_val, fmt_arg="--answer"),
+                bb=dict(fmt_func=fmt.as_fixed, fmt_arg=["fixed", "args"]),
+            ),
+            runner_init_args=dict(),
+            runner_ctx_args=dict(args_order=['aa', 'bb']),
+        ),
+        dict(runner_ctx_run_args=dict(), rc=0, out="", err=""),
+        dict(
+            run_info=dict(
+                cmd=['/mock/bin/testing', '--answer=11', 'fixed', 'args'],
+                environ_update={'LANGUAGE': 'C', 'LC_ALL': 'C'},
+                args_order=('aa', 'bb'),
+            ),
+        ),
+    ),
+    aa_bb_map=(
+        dict(
+            args_bundle=dict(
+                aa=dict(type="int", value=11, fmt_func=fmt.as_opt_eq_val, fmt_arg="--answer"),
+                bb=dict(fmt_func=fmt.as_map, fmt_arg={"v1": 111, "v2": 222}),
+            ),
+            runner_init_args=dict(),
+            runner_ctx_args=dict(args_order=['aa', 'bb']),
+        ),
+        dict(runner_ctx_run_args=dict(bb="v2"), rc=0, out="", err=""),
+        dict(
+            run_info=dict(
+                cmd=['/mock/bin/testing', '--answer=11', '222'],
+                environ_update={'LANGUAGE': 'C', 'LC_ALL': 'C'},
+                args_order=('aa', 'bb'),
+            ),
+        ),
+    ),
+    aa_bb_map_default=(
+        dict(
+            args_bundle=dict(
+                aa=dict(type="int", value=11, fmt_func=fmt.as_opt_eq_val, fmt_arg="--answer"),
+                bb=dict(fmt_func=fmt.as_map, fmt_arg={"v1": 111, "v2": 222}),
+            ),
+            runner_init_args=dict(),
+            runner_ctx_args=dict(args_order=['aa', 'bb']),
+        ),
+        dict(runner_ctx_run_args=dict(bb="v123456789"), rc=0, out="", err=""),
+        dict(
+            run_info=dict(
+                cmd=['/mock/bin/testing', '--answer=11'],
+                environ_update={'LANGUAGE': 'C', 'LC_ALL': 'C'},
+                args_order=('aa', 'bb'),
+            ),
+        ),
+    ),
 )
 TC_RUNNER_IDS = sorted(TC_RUNNER.keys())
 
@@ -301,70 +355,16 @@ def test_runner_context(runner_input, cmd_execution, expected):
                 results = ctx.run(**cmd_execution['runner_ctx_run_args'])
                 _assert_run(runner_input, cmd_execution, expected, ctx, results)
 
+        with pytest.raises(exc):
+            with runner(**runner_input['runner_ctx_args']) as ctx2:
+                results2 = ctx2.run(**cmd_execution['runner_ctx_run_args'])
+                _assert_run(runner_input, cmd_execution, expected, ctx2, results2)
+
     else:
         with runner.context(**runner_input['runner_ctx_args']) as ctx:
             results = ctx.run(**cmd_execution['runner_ctx_run_args'])
             _assert_run(runner_input, cmd_execution, expected, ctx, results)
 
-
-@pytest.mark.parametrize('runner_input, cmd_execution, expected',
-                         (TC_RUNNER[tc] for tc in TC_RUNNER_IDS),
-                         ids=TC_RUNNER_IDS)
-def test_runner_callable(runner_input, cmd_execution, expected):
-    arg_spec = {}
-    params = {}
-    arg_formats = {}
-    for k, v in runner_input['args_bundle'].items():
-        try:
-            arg_spec[k] = {'type': v['type']}
-        except KeyError:
-            pass
-        try:
-            params[k] = v['value']
-        except KeyError:
-            pass
-        try:
-            arg_formats[k] = v['fmt_func'](v['fmt_arg'])
-        except KeyError:
-            pass
-
-    orig_results = tuple(cmd_execution[x] for x in ('rc', 'out', 'err'))
-
-    print("arg_spec={0}\nparams={1}\narg_formats={2}\n".format(
-        arg_spec,
-        params,
-        arg_formats,
-    ))
-
-    module = MagicMock()
-    type(module).argument_spec = PropertyMock(return_value=arg_spec)
-    type(module).params = PropertyMock(return_value=params)
-    module.get_bin_path.return_value = '/mock/bin/testing'
-    module.run_command.return_value = orig_results
-
-    runner = CmdRunner(
-        module=module,
-        command="testing",
-        arg_formats=arg_formats,
-        **runner_input['runner_init_args']
-    )
-
-    def _assert_run_info(actual, expected):
-        reduced = dict((k, actual[k]) for k in expected.keys())
-        assert reduced == expected, "{0}".format(reduced)
-
-    def _assert_run(runner_input, cmd_execution, expected, ctx, results):
-        _assert_run_info(ctx.run_info, expected['run_info'])
-        assert results == expected.get('results', orig_results)
-
-    exc = expected.get("exc")
-    if exc:
-        with pytest.raises(exc):
-            with runner(**runner_input['runner_ctx_args']) as ctx:
-                results = ctx.run(**cmd_execution['runner_ctx_run_args'])
-                _assert_run(runner_input, cmd_execution, expected, ctx, results)
-
-    else:
-        with runner(**runner_input['runner_ctx_args']) as ctx:
-            results = ctx.run(**cmd_execution['runner_ctx_run_args'])
-            _assert_run(runner_input, cmd_execution, expected, ctx, results)
+        with runner(**runner_input['runner_ctx_args']) as ctx2:
+            results2 = ctx2.run(**cmd_execution['runner_ctx_run_args'])
+            _assert_run(runner_input, cmd_execution, expected, ctx2, results2)