Commit Graph
87 Commits
Author SHA1 Message Date
Jakob Meng 114bc855c0 Refactored keystone_federation_protocol{,_info} modules
Change-Id: I6f9eef184538df4feff49dd6ed659a9b9ac1289f
2022-11-10 19:39:51 +01:00
Jakob Meng 5862d91615 Split project_access into {compute_flavor,volume_type}_access modules
Change-Id: I33fa4b3a08392feac702f45a2c47f8b04799ac0b
2022-11-10 19:35:49 +01:00
Jakob Meng cd002a5555 Use module defaults groups in ci integration tests to reduce code bloat
Change-Id: I1370f5bcde602f63b4763c487f027677e79c73b0
2022-11-09 14:35:17 +01:00
Jakob Meng d0ac32eefd Refactored integration test of openstacksdk logging feature
Instead of asserting that openstacksdk created a logging file,
we read the contents of that log file now. The benefit is, that
this integration test gives users an idea how to actually use the
logging feature and also read the contents of the log file.

Change-Id: I9e3c75dda4c1c920937c9780fbdcbe3caeb4d6ad
2022-11-09 14:35:09 +01:00
Jakob Meng d07778c24a Refactored config module
Change-Id: I7ba626e2ef2a108bf79cacd8d6ee0735b8a0bdeb
2022-11-09 09:32:27 +00:00
Jakob Meng a8f6dbd904 Renamed image->image_name and flavor->flavor_name to avoid collisions
Change-Id: I09a133b5c4f6c71e10d274be1c70b7edcce1c83c
2022-11-08 20:44:02 +01:00
Jakob Meng b3c2e8f1ce Refactored volume_snapshot{,_info} modules
Change-Id: I70fc744f786a9de654592c97188af48ddbe8751d
2022-11-08 13:48:57 +01:00
Jakob Meng 8c889f8eb3 Fixed check mode
Change-Id: I3a7a0b0567ac24fb9433352fdc99022bf366fc6a
2022-11-08 13:48:01 +01:00
Jakob Meng 764a8bff64 Refactored volume_backup{,_info} modules
Change-Id: I523fd25a11f8f39a346afc17ae1e3a4dfcb8bae2
2022-11-04 20:06:50 +01:00
Jakob Meng 8955110586 Refactored volume_info module
Change-Id: I5fd844fbb7b9831deb32594c4a587b7249f63d00
2022-11-03 10:31:54 +01:00
Jakob Meng 1e27b6b69c Revert openstack.cloud.server parameter user_data to userdata
openstacksdk's create_server() [1] takes cloud-init userdata as a
plaintext string (or other data types [2]) via argument 'userdata',
not 'user_data' [3]. This is 'userdata' argument of create_server()
is different from the 'user_data' value which is returned [4]. The
latter is base64 decoded, while the former is e.g. a plaintext string.
As a (tiny) visual indicator for this difference, this patch reverts
the module parameter 'user_data' to 'userdata' which was used before
this module got refactored [5].

[1] https://opendev.org/openstack/openstacksdk/src/commit/57fbb72e32351f795f5888306096e8d3d35a64ef/openstack/cloud/_compute.py#L678
[2] https://opendev.org/openstack/openstacksdk/src/commit/57fbb72e32351f795f5888306096e8d3d35a64ef/openstack/cloud/_compute.py#L1757
[3] https://opendev.org/openstack/openstacksdk/src/commit/57fbb72e32351f795f5888306096e8d3d35a64ef/openstack/cloud/_compute.py#L796
[4] https://opendev.org/openstack/openstacksdk/src/commit/57fbb72e32351f795f5888306096e8d3d35a64ef/openstack/compute/v2/server.py#L223
[5] https://opendev.org/openstack/ansible-collections-openstack/commit/ac401bb354c6a1d388848a3ebd18537beba080cc

Change-Id: I1f3b9314e4d82674cd2ae45f6209de5611e2b6cb
2022-10-28 18:19:24 +02:00
Jakob Meng 2f87448567 Always remove temporary files in volume's integration test
This will ensure proper cleanup when tests fail.

Change-Id: Ia5758ffeab43d92f670d3e159fe4f3747491ec94
2022-10-27 11:10:10 +02:00
Jakob Meng 4e1718db49 Refactored ci script with shellcheck suggestions and install collections
Change-Id: I071e50eadfaf0f17f413a0c5f86e5d6b96356b36
2022-10-26 13:05:07 +00:00
Jakob Meng 99074ccd12 Refactored baremetal_port and baremetal_port_info modules
Sorted argument specs and documentation of both modules.

Refactored both modules to be subclasses of OpenStackModule class.

Renamed baremetal_port's module attributes 'uuid' to 'id' and
'portgroup' to 'port_group' to match openstacksdk. Added the previous
attribute names as aliases to keep backward compatibility. Added
alias 'pxe_enabled' for 'is_pxe_enabled' which was previously set
programmatically.

Changed baremetal_port module to return attribute 'port' only when
state is present. It will return no values (except Ansible's default
values) when state is absent. Previous return value 'id' can be
retrieved from port's dictionary entry 'id'.
The non-standard return value 'result' has been dropped because its
content can easily be reconstructed with Ansible's is changed check.

The non-standard return value 'changes' has been dropped because its
content was only returned on updates, has no known uses and can
easily be reconstructed in Ansible by comparing the updated port
dictionary with a copy of the pre-updated port dictionary.

Module baremetal_port_info will no longer fail when no port with a
matching id or name or address could be found. Instead it will return
an empty list like other *_info modules.

baremetal_port_info's return attribute 'baremetal_ports' has been
renamed to 'ports' to be consistent with other modules. The former
name will keep to be available for now to keep backward compatibility.

Both modules convert their return values into dictionaries without
computed (redundant) values. They do not drop values such as links
anymore though, because we do not withhold information from users.

Updated DOCUMENTATION, EXAMPLES and RETURN docstrings in both modules.

Added integration tests for both modules. They will not run in CI atm,
because we do not have Ironic enabled in our DevStack environment.

Change-Id: I54b3ea9917fbbbdf381ef934a0d92e2857f6d51b
2022-10-25 11:31:07 +02:00
Jakob Meng 65a7e74b2b Refactored baremetal_node_action module
Sorted argument specs and documentation of the module.

Refactored baremetal_node_action module to be a subclass of the
OpenStackModule class.

Redefined baremetal_node_info's module attributes 'id' and 'uuid'
as aliases of the 'name' attribute because modules in this collection
do not differentiate between ids and names. The previous revision of
baremetal_node_info module had the same behaviour implemented but did
not make the relationship between 'id'/'uuid' and 'name' explicit.

Changed types and/or choices of module attributes 'deploy',
'maintenance', 'power' and 'state' to match what has been described
in DOCUMENTATION string and to get rid of the non-Ansible'ish and
inconsistent parsing of input values in _is_true() and _is_false()
functions. Ansible can handle argument types for us, no need to
implement it ourselfs.

Dropped deprecated ironic_url attribute from DOCUMENTATION docstring.
Dropped wait and timeout attributes from DOCUMENTATION because their
docstrings will be added via documentation fragment.

Dropped attribute 'result' from module results because in our modules
we consistently do not explain what we do in modules.

Updated DOCUMENTATION, EXAMPLES and added RETURN docstrings.

Refactored the change logic for maintenance, power state and state,
eliminating unreachable or broken code.

Dropped wait attribute from DOCUMENTATION because its docstring will
be added via documentation fragment.

Kept timeout attribute in DOCUMENTATION and argument_spec because
it has a high(er) default value, to account for long node
(de)activiation times, than what e.g. the generic doc fragment
specifies.

Change-Id: I991f23c16583da106105677d75b3651959280d98
2022-10-25 11:30:46 +02:00
Jakob Meng 902b2f8147 Refactored baremetal_node and baremetal_node_info modules
Added integration tests for both modules. They will not run in CI atm,
because we do not have Ironic enabled in our DevStack environment.

Sorted argument specs and documentation of both modules.

Refactored both modules to be subclasses of OpenStackModule class.

Renamed baremetal_node_info's module attribute 'node' to 'name' and
added the former as an alias to be consistent with other *_info
modules.

baremetal_node_info will no longer fetch port and portgroup details
because this requires extra api calls for each node. Users can use
the baremetal_port module to retrieve ports for each node on demand.

Refactored code for constructing node updates in baremetal_node module
which allowed us to drop the dependency on Python module jsonpatch.

Deprecated baremetal_node's skip_update_of_masked_password attribute.
Updating or even specificing passwords for nodes has not been
supported for a while now, rendering the attribute useless.

Renamed baremetal_node's module attributes 'chassis_uuid' to
'chassis_id', 'uuid' to 'id' as well as suboptions of
'properties' to match openstacksdk. Added the previous attribute
names as aliases to keep backward compatibility.
Marked nics attribute in baremetal_node as not updatable.

Changed baremetal_node module to return attribute 'node' only when
state is present. It will return no values (except Ansible's default
values) when state is absent. Previous return value 'uuid' can be
retrieved from node's dictionary entry 'id'.
The non-standard return value 'result' has been dropped because its
content can easily be reconstructed with Ansible's is changed check.
The non-standard return value 'changes' has been dropped because it
was only returned on updates, has no known uses and can easily be
reconstructed in Ansible by comparing the returned node dictionary
with a copy of a previous node dictionary.

Module baremetal_node_info will no longer fail when no node with a
matching id or name or mac could be found. Instead it will return
an empty list like other *_info modules.

baremetal_node_info's return attribute 'baremetal_nodes' has been
renamed to 'nodes' to be consistent with other modules. The former
name will keep to be available for now to keep backward
compatibility.

Both modules convert their return values into dictionaries without
computed (redundant) values. They do not drop values such as links
anymore though, because we do not withhold information from users.

Updated DOCUMENTATION, EXAMPLES and RETURN docstrings in both
modules.

Dropped deprecated ironic_url attribute from DOCUMENTATION docstring
in baremetal_info. Dropped wait attribute from DOCUMENTATION because
its docstring will be added via documentation fragment.

Kept timeout attribute in DOCUMENTATION and argument_spec because
it has a high(er) default value, to account for long provisioning
times, than what e.g. the generic doc fragment specifies.

Change-Id: If3044acf672295e9b61fa60d0969f47cd06dfdeb
2022-10-25 10:11:39 +02:00
Jakob Meng 7bad5cfd42 Refactored stack_info module and allow to filter by owner and project names
Allow to filter stacks by owner names and project names instead of
ids only, in order to be consistent with other *_info modules.

Renamed stack_info's module attributes 'owner_id' to 'owner' and
'project_id' to 'project' to account for the new filter by name
functionality. Added the *_id attributes as aliases to keep
backward compatibility.

Updated and extended DOCUMENTATION, EXAMPLES and RETURN docstrings.

The stack_info module will convert its return values into dictionaries
without computed (redundant) values. Thus dropping values such as
location is not required anymore.

Change-Id: I9cdfb44dd424f63c05943616cf5918ceb3a57b1f
2022-10-14 12:22:10 +02:00
Jakob Meng 0ade33eb6f Bump minimum required openstacksdk release to 0.102.0
Latest release candidate 0.102.0 of openstacksdk's first major release
brings new features and bug fixes, e.g. for floating ips [1], [2].

[1] https://review.opendev.org/c/openstack/openstacksdk/+/859672
[2] https://github.com/openstack/openstacksdk/commit/2535ba7a28ae081f064ae20f9ddb19f48a973625

Change-Id: I009c8515e853baf4df3455fab19571f28e02ab46
2022-10-14 12:08:43 +02:00
Jakob MengandRafael Castillo 41299b9666 Refactored stack module and use stack's tag(s) attribute
Dropped stack status checks for CREATE_COMPLETE and UPDATE_\
COMPLETE and instead pass wait=True to openstacksdk's create_stack()
and update_stack() calls because those will call event_utils.\
poll_for_events() which garantees that stack has reached those
states when they return [1],[2].

Check for duplicate keys in parameters which already have been
defined by regular module attributes. Raise an error to warn
users that they tried to overwrite parameters which they already
specified with module attributes.

Renamed stack's module attribute 'tag' to 'tags' to match both
openstacksdk and OpenStack API and clarify that more than a
single tag can be specified. Added 'tag' as an alias to keep
backward compatibility.

Actually pass the 'tags' aka 'tag' attribute to stack create
and update functions of the openstacksdk which was lost in [3].
Added tags to integration tests.

Sorted argument specs and documentation of the stack module and
marked attributes which are not updatable.

Dropped condition from self.conn.delete_stack() because the latter
will only return False when a stack could not be found which we
already. self.conn.delete_stack() will raise an exception when
stack deletion fails.

Renamed ci integration tests from 'orchestration' to 'stack' in
order to match module name and adapted tags accordingly.

Fixed and enabled ci integration tests for stack and stack_info
modules.

Dropped 'stack_name' from module return values in RETURN
docstring and ci integration tests because openstacksdk not
return this attribute.

[1] https://opendev.org/openstack/openstacksdk/src/commit/9b1c4333526862d0edfc8e5f7bd421b0065b06db/openstack/cloud/_orchestration.py#L92
[2] https://opendev.org/openstack/openstacksdk/src/commit/9b1c4333526862d0edfc8e5f7bd421b0065b06db/openstack/cloud/_orchestration.py#L148
[3] https://opendev.org/openstack/ansible-collections-openstack/commit/af79857bfb9f391e43eaab0f0c94d8c2b7b74c86

Change-Id: I4ace6012112bbcce9094353e27eb4066cf25f229
2022-10-07 19:08:28 +00:00
Jakob Meng a660a35d86 Replaced expensive get_server() and fixed issues in server module
openstacksdk's get_server() [1] calls add_server_interfaces() [2]
which queries OpenStack APIs several times to get all ports and
floating ips attached to a server.

Now we call openstacksdk's compute.find_server() [3] and compute.\
get_server() [4] which result in two API calls, in order to fill
server['addresses'] attribute which we later use to get floating ip
addresses attached to the server.

Do an extra call to compute.get_server() in order to return a pristine
server resource, because openstacksdk's create_server() might call
meta.add_server_interfaces() which alters server attributes such as
server['addresses'] [5].

Fail if options 'auto_ip', 'floating_ips' or 'floating_ip_pools'
are specified but 'wait' is not set to true, because openstacksdk
will add floating ip addresses only if we wait until the server
has been created [6]. This conditional fail will help users to not
shoot their foot.

Marked floating ip support unstable in this module due to various
unresolved issues in openstacksdk's add_ips_to_server() function
such as [9] and [10].

For Zuul CI job ansible-collections-openstack-functional-devstack-\
releases to pass, the minimum required openstacksdk release must be
0.101.0 because [7],[8] are available since that release only.

[1] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/cloud/_compute.py#L484
[2] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/cloud/meta.py#L439
[3] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/compute/v2/_proxy.py#L652
[4] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/compute/v2/_proxy.py#L666
[5] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/cloud/_compute.py#L942
[6] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/cloud/_compute.py#L945
[7] https://review.opendev.org/c/openstack/openstacksdk/+/851976
[8] https://github.com/openstack/openstacksdk/commit/0ded7ac398843b6b1ce46668eb3b45ce02628428
[9] https://storyboard.openstack.org/#!/story/2010352
[10] https://storyboard.openstack.org/#!/story/2010153

Change-Id: I6a5663433b1b9529f99d5eced22a28c692a1d288
2022-10-06 21:30:15 +02:00
Jakob Meng 6ab3d76696 Ensure openstacksdk compatibility in inventory plugin
Story: 2010337
Task: 46470

Change-Id: Ieb624b76627b5127d7a6c4d95233bbd5c2f16182
(cherry picked from commit e8bba38e2e)
2022-09-30 08:33:39 +02:00
Jakob Meng 40d913d2a6 Dropped minimum OpenStack SDK version in inventory plugin
We require openstacksdk>=0.99.0 for this collection but we do not
specify versions in module requirements. We probably should do so,
but then consistently across all modules and plugins.

Change-Id: If3d8cd1491266e3332f478267fc41771f280b9a8
2022-09-30 08:30:57 +02:00
Jakob Meng b416a3e242 Force major version bump in pbr
Sem-Ver: api-break

Ref.: https://docs.openstack.org/pbr/latest/user/features.html#version

Change-Id: Id0d5e5b389b3d2c194ff46ce91c3fe2e98e03c73
2022-09-23 10:43:22 +02:00
Jakob MengandRafael Castillo 57cc3e6aad Bump minimum required openstacksdk release to 0.101.0
Latest release candidate 0.101.0 of openstacksdk's first major release
brings new features and bug fixes, e.g. for floating ips [1], [2].

[1] https://review.opendev.org/c/openstack/openstacksdk/+/851976
[2] https://github.com/openstack/openstacksdk/commit/0ded7ac398843b6b1ce46668eb3b45ce02628428

Change-Id: I4f63e9edab19ba25918513b8c89fcd62913ae75a
2022-09-22 04:49:40 +00:00
Jakob MengandRafael Castillo 9f3f0837c8 Updated network module docs
The network module does support updates since Will's patch [1].

[1] https://review.opendev.org/c/openstack/ansible-collections-openstack/+/854170

Story: 2010024
Task: 45262
Change-Id: I9e4af2c6e685686e28e7f2ae56a615529a6d7b59
2022-09-21 20:09:01 +00:00
Jakob Meng 73827a3d57 Do not remove trailing spaces when reading public key in keypair module
Previously, openstack.cloud.keypair would remove trailing spaces after
reading a public key from a file. The openstack cli tool, python-\
openstackclient, does not do so, i.e. it does not use rstrip to remove
spaces at the end [1]. This breaks idempotency when using openstack
cli tool and our keypair module at the same time.

The rstrip code was introduced to keypair when our modules were still
part of ansible (non-core) in a completely unrelated change [2].

Now, keypair module does no longer alter the public key and instead
uploads it unchanged to OpenStack API.

[1] https://opendev.org/openstack/python-openstackclient/src/commit/7df94c9f821aa7fabef09cad9e6ab16ed66d46b6/openstackclient/compute/v2/keypair.py#L103
[2] https://github.com/ansible/ansible/commit/341efbf7ae078d9818b17036af5ccfed277db52d

Story: 2008574
Task: 41726
Change-Id: Ia09658467d98516ca1ea612e7301629b2f69d2d1
2022-09-07 10:38:40 +00:00
Jakob Meng b99218c143 Deprecate special value 'auto' for attribute 'id' of compute_flavor
Across our modules we use the None values to mark default values for
module attributes. This is in line with openstacksdk's behaviour.

Change-Id: I5920aeeb8eef2ee1c2066a71a273ba52f02305c3
2022-08-22 14:10:11 +02:00
Jakob MengandRafael Castillo d47cf4333c Remove debug statement leftover
Change-Id: I0d3d749b2f5b3474028c83c582c4fd7d1f07687d
2022-08-16 17:06:19 +00:00
Jakob Meng 0ec16fbecc Dropped extra interfaces_info attribute from routers_info module
routers_info's interfaces_info attribute is not provided by
openstacksdk, it is added to each router resource by the routers_info
module after retrieving the routers list. To get the required data
list_router_interfaces() [1] is being called for each router resource
which then retrieves all ports for each router. This requires extra
api calls which might be useless because we do not know whether the
user actually cares about the ports. For getting ports of a router
we have the openstack.cloud.ports module. So instead of proactively
retrieving the router ports we drop the interfaces_info attribute.

The interfaces_info attribute was introduced because retrieving
interfaces via openstacksdk and openstack.cloud modules was
complex in the past [2]. Nowadays, using openstack.cloud.ports
Ansible and Jinja2 filters retrieving ip addresses of a router
is straight forward. In case someone still needs the old
interfaces_info attribute, one can refer to the module example
to see how it could be reproduced. But in general, retrieving the
router interfaces is much easier as can be seen in the updated
integration tests.

[1] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/cloud/_network.py#L1926
[2] https://review.opendev.org/c/openstack/ansible-collections-openstack/+/703927/6/plugins/modules/os_routers_info.py

Change-Id: I7fbdf11d07c95421d3aee800bfeebb88ea829817
2022-08-11 08:37:10 +00:00
Jakob Meng d0eb83e934 Refactored port and port_info modules
Define port's module attribute 'name' as a required attribute because
this parameter is used to find, update and delete ports. Technically,
a name is not required to create a port, but idempotency cannot be
implemented without an identifier to refer to a port. In this
collection we use resource names to find and identify resources. We
do not offer a dedicated id attribute in most modules.

Use port's module attribute 'network' when finding, creating,
updating or deleting ports if the user provided this attribute.
This allows to reduce ambiguity when equal names are used across
different networks.

Added 'description' parameter to port module.

Renamed port's module attributes 'vnic_type' to 'binding_vnic_type'
and 'admin_state_up' to 'is_admin_state_up' to match openstacksdk's
attribute names which are used e.g. in module results. Added aliases
for the old attribute names to keep backward compatibility.

Renamed port_info's module attribute 'port' to 'name' and added
the former as an alias to be consistent with other *_info modules.

Dropped default=None and required=False from argument_spec of port
module because those are the default in Ansible [1][2].

Dropped 'id' field from port module's results to be consistent across
other modules. Use 'port.id' instead.

Sorted argument specs and documentation of the port module and
marked attributes which are not updatable.

Updated RETURN fields documentation for the module results of both
port and port_info modules.

Added integration tests to check the update mechanism of the port
module.

Added assertions for module results to catch future changes in the
openstacksdk and our Ansible modules.

Dropped openstacksdk version check since we require a recent release
anyway.

Fixed indentation in integration tests.

Merged integration tests of port_info module into port module,
because the former does not create any ports and assumes that
ports have been created earlier.

[1] https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html
[2] https://github.com/ansible/ansible/blob/61af59c8082cff068424d1012271daa0aac97288/lib/ansible/module_utils/common/parameters.py#L489

Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: Iacca78649f8e01ae95649d8d462f5d0a1740405e
2022-08-08 19:05:19 +00:00
Jakob Meng fdc67892e3 Refactored router module
Expanded and fixed module results docs.

Moved ext_ips_spec into the module class because global scope is not
necessary. Renamed it to external_fixed_ips_spec to explain its
purpose.

Sorted argument_spec and attribute docs by attribute name and fixed
indentations. Marked router attributes which cannot be updated.

Mark network attribute as required by enable_snat and
external_fixed_ips attributes. Fixed docstring of network attribute:
Module attribute interfaces does not require the network attribute,
its external_fixed_ips what requires network to be set.

Changed examples from deprecated ip to current ip_address attribute.

Dropped self.fail() calls and let openstacksdk handle missing networks,
subnets.. instead because less code means less code to maintain.

Limited line length to 80 chars to be consistent with other OpenStack
projects. Personally, I prefer a 120 chars limit but consistency is more
important.

Added explanation in code comment why we cannot update a router's name.

Moved upfront cleanup operations in router's integration tests to the
beginning of the role.

Assigned meaningful names to result variables
in router's ci role to easily identify which modules produced the
data.

Added tests for router ids, admin state and interfaces.

Change-Id: Icae77a43479fb4f0bae065d1c5d7942cb0f5fd6b
2022-08-03 06:16:42 +00:00
Jakob Meng 1bf22feb2d Fixed code violating Flake8 rule E275
assert [1] is a Python keyword hence Flake8 raised an error:
"E275 missing whitespace after keyword" [2].

[1] https://docs.python.org/3/reference/lexical_analysis.html#keywords
[2] https://www.flake8rules.com/rules/E275.html

Change-Id: I76bbe10b850c38a3fbb38c4a2f5ee17ca5b91b4e
2022-08-01 14:32:24 +02:00
Jakob Meng ac401bb354 Refactored server and server_info modules
Allow to update server attributes such as its description.

Changed default value of server attribute 'security_groups' from
['default'] to [] because the latter is the default in
python-openstackclient [1] and the former behavior causes issues
with existing servers [2]: Previously, when no 'security_groups'
parameter was given, the server module would change existing
servers to use the default security group, dropping all other
security groups assigned to the server.
Our (undocumented) guideline when writing modules is to only
add or change what has been requested by the user and to stick to
defaults from openstacksdk and python-openstackclient whenever
possible. Since we have to break backward compatibility with the
next release anyway, we take this opportunity to clean up this odd
behavior. Now, when no security groups are given, then security
groups of an existing server will not be touched.
Closes story #2007893 [2].
Note, Nova will create a server in the default security group,
if the security_groups parameter is omitted.

Dropped 'openstack' field from server module's results. This
variable expanded to additional server information which might
be useful for Ansible inventories and was filled from
openstacksdk's get_openstack_vars() function [3]. Variables in
this function can make additional cloud queries to retrieve
additional data, so calling this function can be expensive [4].
Users can use *_info modules to retrieve this data on-demand.

Dropped 'availabity_zone' attribute from generic OpenStackModule
arguments and inserted it into server and volume modules because
it is relevant to those two modules only. This is completes what
was started years ago [5] and is possible now since we have
breaking changes anyway.

Switched attribute name 'userdata' with its alias 'user_data' to
match openstacksdk's attribute names which are used e.g. in module
results. The previous attribute name 'userdata' is now used as an
alias and 'user_data' is used as the attribute name to keep backward
compatibility.

Wait for server to get into 'ACTIVE' state when creating a server
and attribute 'wait' has been set to true.

Sorted argument specs and documentation of the server module and
marked attributes which are not updatable. Changed unstable bash
script example in server module documentation.

Renamed server's module attribute 'delete_fip' to 'delete_ips' to
match openstacksdk and clarify that it includes all floating ip
addresses of the server.

Renamed server_info's module attribute 'server' to 'name' and added
the former as an alias to be consistent with other *_info modules.

Added RETURN fields documentation for the module results of both
server and server_info modules.

Added description and examples of how to use the 'filters' attribute
of the server_info module. Closes story #2007873 [6].

Removed 'openstack_' prefix from module results because the prefix is
not consistently used across modules, is more to type without any
benefit and removal of the prefix allows us to signal to users that
their code for handling module results has to be updated. Many modules
have different return values with openstacksdk >= 0.99.0 because it
consistently uses resource proxies now.

Added assertions for module results to catch future changes in the
openstacksdk and our Ansible modules.

Added integration tests to check the update mechanism of the server
module.

Fixed indentation in integration tests.

Ensure proper creation and deletion of resources such as networks,
subnets and servers in integration tests of server_action module.

Renamed ci/roles/server/defaults/main.yaml to main.yml, removing the
'a' in the file extension to be consistent with other filenames.

Dropped deprecated function openstack_find_nova_addresses() and
incorporated its code directly into the server module because it
is not used anywhere else.

[1] https://opendev.org/openstack/python-openstackclient/src/commit/e49ad1795b9dd57d5a82fb6f8f365fa20041cf29/openstackclient/compute/v2/server.py#L1070
[2] https://storyboard.openstack.org/#!/story/2007893
[3] https://opendev.org/openstack/openstacksdk/src/commit/9e9fc9879583943a08f854980cca5dfb3a5832f7/openstack/cloud/_compute.py#L1772
[4] https://opendev.org/openstack/openstacksdk/src/commit/9e9fc9879583943a08f854980cca5dfb3a5832f7/openstack/cloud/meta.py#L482
[5] https://github.com/ansible/ansible/commit/9bf33e56dd49e9478ba091a1ba12d17b7caeac24
[6] https://storyboard.openstack.org/#!/story/2007873

Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: I2f955519a7e8c782b1dab8f94f7a019ed384b81d
2022-07-28 21:56:30 +02:00
Jakob Meng 0b2b56765c Fixed Python shebang and UTF-8 coding in modules
Be consistent with Ansible docs [1], [2], [3].

[1] https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html
[2] https://docs.ansible.com/ansible-core/devel/dev_guide/testing/sanity/shebang.html
[3] https://github.com/ansible/ansible/blob/b86a18bd273499f5d10e581821a47571690660e1/test/lib/ansible_test/_util/controller/sanity/code-smell/shebang.py

Change-Id: Ia3976bb3fcca662cefa219edeef057bcb7143c01
2022-07-28 10:03:09 +02:00
Jakob Meng 0215e2a5d4 Dropped default module options
Removed default=None, type='str' and required=False from all module's
argument_specs with

sed -i \
    -e 's/default=None, //g' \
    -e 's/default=None,//g' \
    -e 's/default=None//g' \
    -e "s/type='str', //g" \
    -e "s/type='str',//g" \
    -e "s/type='str'//g" \
    -e 's/required=False, //g' \
    -e 's/required=False,//g' \
    -e 's/required=False//g' \
    plugins/modules/*.py plugins/module_utils/*.py

and manually cleaned the results because those options values are the
default in Ansible.

Ref.: https://docs.ansible.com/ansible/latest/dev_guide/developing_program_flow_modules.html#argument-spec

Change-Id: Icbc3bb84269d3b8205fac259300902ebdaf6a3ae
2022-07-27 12:39:44 +02:00
Jakob Meng ceae5c1030 Use master branch of openstacksdk in Bifrost jobs
Bifrost downgrades openstacksdk to a release <0.99.0 [1], which is
incompatible to our master branch of the Ansible OpenStack
collection. By adding openstacksdk to Zuul variable job.required-\
projects [2], Bifrost will install openstacksdk from its master
branch, overwriting any existing (older) openstacksdk releases.

[1] https://opendev.org/openstack/bifrost/commit/03ddd026567f7b61b89d83b0e7e7cb35ed466ae6
[2] https://review.opendev.org/c/openstack/bifrost/+/849563/5/playbooks/test-bifrost.yaml

Change-Id: I46dadc70338bb45d459e8176cc26e67c20c1d4ae
2022-07-25 20:20:46 +02:00
Jakob Meng b87ae7dcce Dropped symbolic links and plugin routing for deprecated module names
With Ansible OpenStack collection 2.0.0 we break backward
compatibility to older releases, mainly due to breaking changes
coming with openstacksdk >=0.99.0. For example, results will change
for most Ansible modules in this collection.

We take this opportunity to drop the symbolic links with prefix
os_ in plugins/modules and the plugin routing in meta/runtime.yml.
This means users have to call modules of the Ansible OpenStack
collection using their FQCN (Fully Qualified Collection Name) such
as openstack.cloud.server. Short module names such as os_server
will now raise an Ansible error. This also decreases the likelihood
of incompatible Ansible code going undetected.

Symbolic links were introduced to keep our collection backward
compatible to user code which was written for old(er) Ansible releases
which did not have support for collections and where OpenStack modules
where named with a prefix os_ such as os_server which is nowadays
known and stored as openstack.cloud.server.

In Ansible aka ansible-base 2.10, a internal routing table
lib/ansible/config/ansible_builtin_runtime.yml [1] was introduced which
Ansible uses to resolve deprecated module names missing the FQCN (Fully
Qualified Collection Name). Additionally, collections can define their
own plugin routing table in meta/runtime.yml [2] which we did.

In ansible-base 2.10 and ansible-core 2.11 or later, if a user uses a
short module name and the collections keyword is not used, Ansible
will first look in the internal routing table, get an FQCN, and then
looks in the collection for that FQCN. If there is another routing
entry for that new name in that collection's meta/runtime.yml,
Ansible will continue with that redirect. If it does not find another
redirect, Ansible will look for the plugin itself, so it will not
find a redirect in the collection before looking at its internal
redirects. Except if the user uses a FQCN, then it looks directly in
that collection.

Ansible 2.9 and 2.8 do not have any notion of these redirects with a
plugin routing table, backward compatibility with deprecated os_*
module names is solely achieved with symbolic links. Ansible releases
older than 2.11 are EOL [3], so usage of os_* symlinks should reduce
soon.

[1] https://github.com/ansible/ansible/blob/devel/lib/ansible/config/ansible_builtin_runtime.yml
[2] https://github.com/openstack/ansible-collections-openstack/blob/master/meta/runtime.yml
[3] https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html

Change-Id: I28cc05c95419b72552899c926721eb87fb6f0868
2022-07-22 09:48:20 +00:00
Jakob Meng eda2e301c3 Raise minimum OpenStack SDK version to 0.99.0 again
This reapplies commit 4bfa135b20 [1]
which got reverted in commit 1b59c19a24
[2] due to issues in how TripleO Quickstart installed the Ansible
OpenStack collection. TripleO Quickstart has now been fixed and
will install code from our stable/1.0.0 branch instead of master for
all RDO branches which have openstacksdk <0.99.0 [3][4].

[1] https://opendev.org/openstack/ansible-collections-openstack/commit/4bfa135b20caeb6d6c2717049059ac52fd08e0e1
[2] https://opendev.org/openstack/ansible-collections-openstack/commit/1b59c19a24c55aa236d80552dcbf70c9c7b5088e
[3] https://review.opendev.org/c/openstack/tripleo-ci/+/849500
[4] https://review.opendev.org/c/openstack/tripleo-quickstart/+/849620/

Change-Id: I918d776c1560f03a4a84df371feb013d47043aa0
2022-07-19 09:55:11 +00:00
Jakob Meng b250b76c33 Dropped CentOS8-based TripleO job
No RDO release for CentOS8 will get a OpenStackSDK 1.x.x release.

Change-Id: I728f0f282717c11a782be30859f999008f3fc8cb
2022-07-19 09:28:52 +02:00
Jakob Meng fc2dda1d86 Refactored TripleO jobs, e.g. to use latest openstacksdk RPM
Build and install latest RPM for openstacksdk from its master branch
instead of using the (pinned) RPM from RDO. This is necessary because
openstacksdk in RDO is currently pinned to <0.99.0 for all RDO
releases. Variable artg_change_list is used to define what code is
being build from source. The RPM of the Ansible OpenStack collection
does not have build from source because TripleO Quickstart installs
the collection from job.required-projects [1]. The latter shadows the
RPM release which is installed later by TripleO because it has a
higher precedence in ansible.cfg [2][3].

Changed the job hierarchy to other base jobs tripleo-ci-centos-8-\
standalone-build and tripleo-ci-centos-9-standalone-build. This reduces
the number of variables we have to define. It is also cleaner since our
CentOS9 job is no longer based on the CentOS8 job which prevents
issues with job variant collections due to our branched repository.

Added more Ansible modules to files which trigger TripleO jobs,
because Ansible role os_tempest [4] requires those modules and is
called in TripleO jobs. Modules which have been added include:
* openstack.cloud.compute_flavor
* openstack.cloud.image
* openstack.cloud.network
* openstack.cloud.router
* openstack.cloud.subnet

Added tripleo-ci-centos-8-standalone-osa to experimental jobs so that we
can actually run this job on demand easily.

[1] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/quickstart.sh#L123
[2] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/ansible.cfg#L19
[3] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/quickstart.sh#L595
[4] https://opendev.org/openstack/openstack-ansible-os_tempest.git

Change-Id: Ibde318678a3e44fdc297a6f29761eb0c7d77cbc9
2022-07-18 15:27:12 +02:00
Jakob Meng 1f5a2019a0 Replaced code in routers_info module with openstacksdk function
Replaced custom code for interface listing with call to openstacksdk.
The original idea was to reduce the number of calls to the OpenStack
API but this kind of optimization is better to be implemented in the
SDK itself [1]. Reimplementing code like this increases our
maintenance burden, does not help other SDK users and increases the
likeliness of bugs. For example, variable allowed_device_owners
introduced a bug, it is not 'network_router_interface_distributed'
but 'network:router_interface_distributed'.

[1] https://review.opendev.org/c/openstack/openstacksdk/+/849967

Change-Id: I9c52de03c53ef29d7cecdf26253c0c00a7cf3689
2022-07-15 12:34:39 +02:00
Jakob Meng 070e77feca Added assertions on endpoint module results
Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: If8befd62ac59ef19debf0d63abe76f11b47c2da3
2022-07-12 10:43:22 +02:00
Jakob Meng 0e675a9129 Split CI tests for security_group and security_group_rule modules
Previously, all security_group{,_info} and security_group_rule{,_info}
modules were tested in the same Ansible role. This patch splits
tests into two separate Ansible roles to increase readibility and
prevent variable name conflicts, e.g. for expected_fields.

Change-Id: Ifc28435147b3bfe88d4ee5e176469a53b7395dc0
2022-07-05 12:22:35 +02:00
Jakob Meng ad3a3a89f2 Refactored endpoint module and explained region attribute
Regions have IDs, but do not have names.
Ref.: https://docs.openstack.org/api-ref/identity/v3/#regions

Change-Id: I2512bbde6e96e2ab0f1fef0230295223f46105dd
2022-06-27 11:34:26 +00:00
Jakob Meng c6c1c6a070 Applied workaround in CI for issue #78017 in ansible-core
Module ansible.builtin.user in ansible-core 2.13.0 and 2.13.1 is
affected by #78017 [1] which results in an exception being raised
in ci/roles/keypair/tasks/main.yml [2]. Until this issue is fixed,
we will exclude the broken versions 2.13.0 and 2.13.1 in
requirements.txt [3].

[1] https://github.com/ansible/ansible/issues/78017
[2] https://opendev.org/openstack/ansible-collections-openstack/src/commit/802e46d554a1f899ad56be5574d2f7e2323ee6d6/ci/roles/keypair/tasks/main.yml#L72
[3] https://opendev.org/openstack/ansible-collections-openstack/src/branch/master/tests/requirements.txt

Change-Id: I61bec4e62ecbcf357f3c1279a7373049077cb8d4
Signed-off-by: Jakob Meng <code@jakobmeng.de>
2022-06-26 13:07:34 +02:00
Jakob Meng a9fa496ebe Changed our DevStack based Zuul CI jobs to voting
Keep jobs with devel branch of Ansible non-voting to prevent
Ansible from blocking our Zuul CI gates.

Change-Id: I92668b37d42db758e2bae8e791357b72c131a899
2022-06-01 20:44:27 +02:00
Jakob Meng dee39a71b6 Warn users about us breaking backward compatibility
Change-Id: I7a2867329f65af6330abccb1954bf49b92cd8721
2022-05-27 08:06:52 +02:00
Jakob Meng 1b59c19a24 Revert "Raise minimum OpenStack SDK version to 0.99.0"
This reverts commit 4bfa135b20 [1] because
it broke TripleO [2] and possibly other users which were using code from
master branch with incompatible SDK releases. We will reapply this
safety check later once dust has settled.

Ref.:
[1] https://opendev.org/openstack/ansible-collections-openstack/commit/4bfa135b20caeb6d6c2717049059ac52fd08e0e1
[2] https://bugs.launchpad.net/tripleo/+bug/1975646

Change-Id: I637f1b7c1b792adf6d3d17a27ccb42179f56a83b
2022-05-25 08:34:29 +02:00
Jakob Meng 4bfa135b20 Raise minimum OpenStack SDK version to 0.99.0
Alongside OpenStack SDK 1.0.0 we will release a new collection version
2.0.0 which is compatible to OpenStack SDK 1.x.x series only. Code in
branch stable/1.0.0 will remain compatible to OpenStack SDK 0.x.x
series only. Release candidates for the first major release of
OpenStackSDK 1.x.x will be numbered using 0.99.x versions.

At Ansible OpenStack modules PTG on 2022-04-07 it was decided to raise
an error if one is using a incompatible releases of the OpenStack SDK
with our collection. We decided against showing warnings only because
they can be missed easily and functionality  will be broken but
probably hardly detectable when using the wrong SDK.

This patch bumps the minimum required SDK versions to 0.99.0, so that
an error will be raised when users try to use our collection with an
incompatible SDK release, e.g. use code from our master branch with
a OpenStack SDK 0.x.x release.

Change-Id: I3974deabc516379745794806886352279dc4f4a7
2022-05-24 14:36:19 +02:00
Jakob Meng c7ae7a5f98 Updated pip constraints for release candidates of OpenStackSDK's first major release
A release candidate for the first major release of OpenStackSDK 1.x.x
has been published with version number 0.99.0. Release candidates will
be numbered using 0.99.x versions.

Ref.: https://meetings.opendev.org/irclogs/%23openstack-release/%23openstack-release.2022-05-19.log.html

Change-Id: I51a5b803f6646d3b1c5e198072cb3da19925fc3f
2022-05-23 14:07:41 +02:00
Jakob Meng 4db7a6238b Changed TripleO jobs to use correct release files
Commit bc27851617 [1] in opendev.org/openstack/tripleo-ci changed the
release filename for periodic jobs to promotion-testing-hash-*.yml [2]
instead of using e.g. train.yml for OpenStack Train based releases [3].
Due to this change, container images were not pulled from the local
image registry but from trunk.registry.rdoproject.org instead [2]. This
caused our periodic jobs to fail during container image build because
when using a local registry a different namespace prefix "openstack"
(instead of *-binary such as centos-binary) is used which is not valid
on trunk.registry.rdoproject.org.

As a workaround, we force TripleO jobs to run as check jobs, no matter
what pipeline they run in [4].

Thanks to Sandeep Yadav <sandyada@redhat.com> for discovering the root
cause of this issue and providing a workaround!

Ref.:
[1] https://opendev.org/openstack/tripleo-ci/commit/bc278516174cd1133ca910bb4978397499645a34
[2] https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/promotion-testing-hash-train.yml#L7
[3] https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/train.yml
[4] https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/ci-common-vars/vars/main.yaml#L24

Change-Id: Ib7d8fc9e6781e43e04f0a9feee261b9f3f29e1fe
2022-05-11 11:47:31 +02:00
Jakob Meng fdf4999747 Reverted identity_user_info from identity.users() to search_users()
Commit 2df07f3523 changed module identity_user_info to use function
identity.users() instead of search_users(). The first does not allow
to search for id with parameter name while the previous and current
search_users() function has a name_or_id parameter which allows to
search by name and id.

Ref.: https://opendev.org/openstack/ansible-collections-openstack/commit/2df07f3523ba3281d84c93083bc3b5d1791afb4b

Change-Id: I71226e578a234d24e068a256cf4a5533ccd4c201
2022-05-09 10:42:54 +02:00
Jakob Meng a8589e9f4d Added description to RETURN doc of identity_user_info module
Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: I000a12df18c66816b2b2de177530ffc46f9dc9ae
2022-05-05 11:05:57 +02:00
Jakob Meng c83e0e39b1 Removed job definitions for stable/1.0.0 branch
Previously, all job definitions where shared across each .zuul.yaml in
both branches. When a job definition was changed in one branch, Zuul CI
could pick the job definition from the other branch, which was not
intended.

The problem arises when mixing explicit job.branches matchers with
implicit branch matching, when defining same jobs on multiple branches.
Zuul CI expects that jobs to be defined in one or the other branch, not
both at the same time. One should only use job.branches matchers from
single-branched projects, e.g. trusted config repos. When defining jobs
in branched repositories one selects which job definition to use by the
branch associated with the triggering event instead.
Each trigger has a branch associated with it, whether it is the branch
targeted by the change being proposed, the branch to which a commit
merged, a branch attached to a timer trigger etc. This branch name is
searched across involved projects in order to determine what job
definition should be used.

The job.branches directive is rarely applied to a job which will be
copied to multiple branches. When you have multiple copies of a job
with the job.branches attribute, Zuul CI could pick any of the job
definitions which might not be the one you expected.
The job.branches attribute is useful in single branch config
repositories where a specific job definition has to be applied to a
specific branch of the repository. Another definition of the job
will exist in another branch of the config repository.

This patch removes job definitions which are specific to other
branches, except for parent jobs which are shared across branches.

Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: Idb8720bd96843b7807dd5cb62b30c1edf3a7a37c
2022-05-04 10:17:09 +02:00
Jakob Meng 92566dedae Fixed return type of identity_role module
Change-Id: Ifcf55749919fd59c990611e8c4537de787ba4005
2022-05-04 09:25:59 +02:00
Jakob Meng 4a7330364e Fixed return values in compute_service_info module again
OpenStack SDK 0.53 added parameters is_forced_down and updated_at
in openstack/compute/v2/service.py, hence our compute_service_info
module will return different values depending on which release of
the OpenStack SDK is used.

Ref.: https://github.com/openstack/openstacksdk/commit/5450c4525313581b2ebc0e1ab6ac6252964c595f

Change-Id: I4b055266555cb91681d0ab6edcaa850e061f3afb
2022-05-02 20:49:39 +02:00
Jakob Meng cf5007d478 Fixed return value disable{d,s}_reason in compute_service_info module
OpenStack SDK 0.53 renamed parameter disables_reason to disabled_reason
in openstack/compute/v2/service.py, hence our compute_service_info
module will return different values depending on which release of
the OpenStack SDK is used.

Ref.: https://github.com/openstack/openstacksdk/commit/5450c4525313581b2ebc0e1ab6ac6252964c595f

Change-Id: I1c0f787f7f67c92f92dd106fc8d55580461e4aa3
2022-05-02 11:50:32 +02:00
Jakob Meng c10cc9dd8c Temporarily run passing tests in our Zuul CI jobs only
With merging the code for 1.0.0 of OpenStack SDK into the master branch
several of our modules and CI tests broke. To be able to merge patches
we had to set most of our jobs to non-voting, so atm we do not have a
good code coverage in Zuul CI.

This patch temporarily skips broken tests so that we can set our jobs
back to voting and get some basic code coverage from CI. Follow up
patches which fix modules are supposed to readd skipped tests on
occasion.

Change-Id: Ice42d0bdc12c24227a323ad9c5d3fd33870975c4
2022-04-28 13:19:05 +02:00
Jakob Meng 3ead86904a Changed our Zuul CI *-octavia job to non-voting
Our *-octavia jobs pull OpenStack SDK from PyPI and PyPI is still
serving the 0.x.x series of the SDK because 1.0.0 has not been
released yet. A recent commit bb25330ddc [1][2] broke compatibility
to older SDK releases prior to 1.0.0 and since then our *-octavia
job on our master branch is failing.

Ref.:
[1] https://review.opendev.org/c/openstack/ansible-collections-openstack/+/839033
[2] https://opendev.org/openstack/ansible-collections-openstack/commit/bb25330ddc07d9b433c9f8db7c52baed7020f0cf

Change-Id: I4bacc358cca08a71694c590202066c8565a96f02
2022-04-28 09:59:18 +02:00
Jakob Meng 9f60f0f26d Restricted galaxy-importer script to Python 3.6+
Ansible Galaxy content importer is using format strings [1] which
are supported since Python 3.6. Our Zuul CI job for OpenStack Queens
uses Ubuntu 16.04 LTS (Xenial Xerus) as its base image which has
Python 3.5 only.

Ref.:
[1] https://github.com/ansible/galaxy-importer/blob/b7140d6b3b5e172553788ae9aaa7526f8ff49ea2/galaxy_importer/main.py#L117

Change-Id: I5d3b2f71937a0e4ab9a8d49df10744f7d95a7de2
2022-04-28 08:36:12 +02:00
Jakob Meng e85a0b809a Constrain filters in compute_service_info to SDK >= 0.53.0
Older releases of OpenStack SDK do not support filtering services.

Change-Id: I613c76b8f794ea2024939e07250a50edc5b9e49a
2022-04-27 18:16:55 +00:00
Jakob Meng bba1da17c9 Removed object tags from ci role server
Tag object was introduced to server role in commit c8a5be6b30 [1] to
allow skipping server tests when volumes are not available.

Whenever tag object is specified, Ansible will run those three tasks
in role server. But as our server module has not been ported to
OpenStack SDK 1.0.0 series it will fail even if someone only wants to
test our object ci role.

This patch removes all occurrences of the object tag in the ci server
role. Since it is not used anywhere in our code it will not break ci.

Ref.:
[1] https://opendev.org/openstack/ansible-collections-openstack/commit/c8a5be6b3022140da41dc10a2d0a6a147ad170cf

Change-Id: I222fac499c9a3cb16c4581fb4347170a4d97f833
2022-04-27 09:30:21 +02:00
Jakob Meng bc6622e0e7 Added support for specifying a maximum version of the OpenStack SDK
Alongside OpenStack SDK 1.0.0 we will release a new collection version
2.0.0 which is compatible to OpenStack SDK 1.x.x series only. Code in
branch stable/1.0.0 will remain compatible to OpenStack SDK 0.x.x
series only.

At Ansible OpenStack modules PTG on 2022-04-07 it was decided to raise
an error if one is using a incompatible releases of the OpenStack SDK
with our collection. We decided against showing warnings only because
they can be missed easily and functionality  will be broken but
probably hardly detectable when using the wrong SDK.

This patch implements the code to raise errors when users are trying
to use our collection with an incompatible SDK release, e.g. use code
from our stable/1.0.0 branch with a OpenStack SDK 1.x.x release.

It does not yet change the minimum and maximum required SDK versions
because OpenStack SDK 1.0.0 has not yet been released to PyPI and
SDK's master branch still does not return a 1.x.x version number.

Change-Id: I1052d21cf8f108dbc99619cd4c4072488645b855
2022-04-27 06:45:30 +00:00
Jakob Meng e869564e3c Use Rocky release of Heat in Queens job
The oldest branch in Heat repository is stable/rocky. Previously,
Zuul CI would use the master branch of Heat since it could not find
stable/queens branch but master branch has incompatibilities with
our Queens job.

Change-Id: Iaeca759cad641d4923fc63489fd65f57d9f1345a
2022-04-26 13:14:14 +00:00
Jakob Meng 5a43bdb873 Follow up to bump of minimum required OpenStack SDK release to SDK 0.36.0 (Train)
Commit 879270aa47 [1] bumped the required minimum SDK release
but missed to update two locations in code and docs.

Ref.:
[1] https://opendev.org/openstack/ansible-collections-openstack/commit/879270aa47f4b4e1977637af678ed03b4c6a2c2e

Change-Id: I725a26b07484619f6f2c460e974821f81d60b153
2022-04-22 12:09:26 +02:00
Jakob Meng bb25330ddc Renamed image property protected to is_protected
OpenStack SDK removed support for the deprecated image property
protected in commit afb49692 [1][2]. This replaces the usage of
protected with is_protected in openstack.cloud.image but adds
protected as an alias to not break backward compatibility for
user code. This breaks backward compatibility to older OpenStack
SDK releases though.

Ref.:
[1] https://github.com/openstack/openstacksdk/commit/afb49692f5bfab259af8c594eedbddcb52cbde81
[2] https://review.opendev.org/c/openstack/openstacksdk/+/820926

Change-Id: I5044c927e90c650234fbc22375f44e6841a485a1
2022-04-22 11:38:14 +02:00
Jakob Meng f09cccdb9e Dropped broken linter job openstack-tox-linters-ansible-2.9
Running older linter releases while we have a more up to date
Ansible 2.12 based linter does not provide value and thus wastes
ci resources. Our Ansible 2.9 based linter is broken atm and since
it is EOL soon anyway we drop this job. We still have a linter job
based on the last stable release and one based on Ansible's
devel branch.

Our tox environment for Ansible 2.9 will be dropped in a later
patch once all Ansible 2.9 based jobs have been removed.

Change-Id: I9cd3f729b06516bbd9a3c7985b65fcf294c8bdd7
2022-04-20 09:03:30 +02:00
Jakob Meng 2bf82c2669 Temporarily set job openstack-tox-linters-ansible-2.9 to non-voting
Python module rstcheck which is used by ansible-test deprecated
Python versions prior 3.7 in version 3.5.0 and removed support
in 4.0.0 [1]. Ubuntu 18.04 LTS (Bionic Beaver) comes with Python
3.6 by default, so rstcheck prints a FutureWarning which confuses
ansible-test:

  Run command: ***/python -m rstcheck --report warning
  --ignore-substitutions _,br,release,today,version
  docs/openstack_guidelines.rst
  Traceback (most recent call last):
    File "***/ansible-test", line 28, in <module>
      main()
    File "***/ansible-test", line 24, in main
      cli_main()
    File "***/ansible_test/_internal/cli.py", line 130, in main
      args.func(config)
    File "***/ansible_test/_internal/sanity/__init__.py", line 193,
    in command_sanity
      result = test.test(args, sanity_targets, version)
    File "***/ansible_test/_internal/sanity/rstcheck.py", line 80,
    in test
      results = parse_to_list_of_dict(pattern, stderr)
    File "***/ansible_test/_internal/util.py", line 799, in
    parse_to_list_of_dict
      raise Exception('Pattern "%s" did not match values:\n%s' %
      (pattern, '\n'.join(unmatched)))
  Exception: Pattern "^(?P<path>[^:]*):(?P<line>[0-9]+):
  \((?P<level>INFO|WARNING|ERROR|SEVERE)/[0-4]\) (?P<message>.*)$"
  did not match values:
  ***/rstcheck.py:51: FutureWarning: Python versions prior 3.7 are
  deprecated. Please update your python version.
    FutureWarning
  ERROR: Command "/usr/bin/env ANSIBLE_TEST_CONTENT_ROOT=***/
  ansible_collections/openstack/cloud LC_ALL=en_US.UTF-8 ***/python3.6
  ***/ansible-test sanity -v --python 3.6 --skip-test
  metaclass-boilerplate --skip-test future-import-boilerplate plugins/
  docs/ meta/ scripts/ --metadata ***.json --truncate 0 --redact
  --color no --requirements" returned exit status 1.

We cannot constrain the version of rstcheck which ansible-test installs
into its Python virtual environment. This has to be fixed in Ansible
itself, a pull request against Ansible 2.9 has been opened [2].

Ref.:
[1] https://github.com/myint/rstcheck/blob/master/README.rst
[2] https://github.com/ansible/ansible/pull/77568

As a workaround we temporarily set our Ubuntu 18.04 based linter job
openstack-tox-linters-ansible-2.9 to non-voting and remove it from
check dependencies and as a gate job.

Thanks to Arx Cruz and Jesper Schmitz Mouridsen for pointing out this
issue and its root cause ☺️

Change-Id: Ic6f2febc5a40a29534ac4c7f41f714865099086a
2022-04-19 21:59:31 +02:00
Jakob Meng 5fc8fca06b Drop username from return values of identity_user_info
Users would have a non-null username only with Identity API v2 which
was available in Keystone of OpenStack Pike. Identity API v2 has been
removed since OpenStack Queens [1].

Function search_users() from OpenStack SDK still returns the username
attribute because of [2][3] but it will always be None since
Keystone's API does not return username since OpenStack Pike.

[1] https://docs.openstack.org/releasenotes/keystone/queens.html
[2] https://github.com/openstack/openstacksdk/blob/975cabbdd84baa8e05dc5b107b673607e7c0af1a/openstack/cloud/_utils.py#L246
[3] https://github.com/openstack/openstacksdk/commit/76b081efe45f1f2ab8e09a1fd5a23d9fc54361bc

Change-Id: I2054dd55662698dabd0f2b3565c31dcd3bf24e5a
2022-04-13 12:28:49 +02:00
Jakob Meng 4c3dfec59d Fixed Ansible branch in base job openstack-tox-linters-ansible
Ansible's git repository has no master branch but a devel and several
stable-* branches instead.

Change-Id: I43844c8a1cefceb7337e5a6ff1e8b8df451efb26
2022-04-04 08:30:33 +00:00
Jakob Meng a03dd05465 Refactored tox requirements for different Ansible releases
Sorted pip requirements file to improve readability.

Moved pip requirements for tests into tests subdirectory and dropped
'pip-' prefix to shorten filenames and conform with common naming
scheme for pip requirements files.

Added constrains on OpenStack SDK 1.*.* to job ansible-collections-\
openstack-functional-devstack-releases on master branch because only
stable/1.0.0 branch is compatible to the OpenStack SDK 0.*.* series.

Changed job ansible-collections-openstack-functional-devstack-releases
on master branch to non-voting because OpenStack SDK 1.*.* has not
been released to PyPI yet, so tests on master branch are expected to
fail once we introduce breaking changes from stable/1.0.0 branch.

Change-Id: I6b6bb8c6900f7c8341bbf3f9a24999fbf693ba4b
2022-03-31 10:35:16 +00:00
Jakob Meng 938abd0d84 Reenabled check-import.sh which tests imports to Ansible Galaxy
Reverted commit 1f3417cdef [1] which disabled check-import.sh script.
Python module galaxy_importer will return a non-zero return value on
errors since commit 4f5fd0f29c [2].

Use galaxy-importer 0.3.1 for Ansible 2.9 and galaxy-importer 0.3.2
for later Ansible releases because galaxy-importer moved from ansible
2.9 to ansible-core 2.11 in 0.3.2 [3].

Ref.:
[1] https://opendev.org/openstack/ansible-collections-openstack/commit/1f3417cdef4b55fa6cfb241815a2531358fa286f
[2] https://github.com/ansible/galaxy-importer/commit/4f5fd0f29c30c354cd92837ac71109abc0b48082
[3] https://github.com/ansible/galaxy-importer/commit/98933547831922c45243f39d85eefe150b55fc36

Change-Id: I898149727d80cd7effe6a04ca77a13ef1774e781
2022-03-29 13:20:41 +00:00
Jakob Meng 749f04bcfb Added job variants for stable/1.0.0 branch to Zuul CI config
Releases of OpenStack SDK on PyPI will soon switch to the 1.*.* series
which our stable/1.0.0 branch is incompatible with. Previously, in
commit 0f532d10f3, several jobs have been changed to run on master
branch only.

This patch adds siblings jobs for our stable/1.0.0 branch which
pull the latest SDK releases of the 0.*.* series from PyPI instead.

Ref.: https://opendev.org/openstack/ansible-collections-openstack/commit/0f532d10f373a3c6cf26be09045da5a934214729

Change-Id: Iefc6acfa4c25eb5d9ab062a3bfa655be2188cb77
2022-03-29 07:32:06 +00:00
Jakob Meng 27e113780f Set Zuul CI job for OpenStack Train to voting
Job *-train-ansible-2.11 failed since commit 031475d42e because that
patch caused most jobs to install the latest Python packages of
OpenStack SDK and other requirements from PyPI to tox' virtualenv
instead of respecting the override-checkout keywords and using
releases of OpenStack Train. Commit 87858ab976 and its follow ups
has fixed this issue.

Ref.:
https://opendev.org/openstack/ansible-collections-openstack/commit/031475d42e306808fa1beefdaaf615fa540662a4
https://opendev.org/openstack/ansible-collections-openstack/commit/87858ab976160e08536b936bb7a3e1498d38c1d2

Change-Id: Ib12e6195db9bb232735ea5a785ccc88bc749ea17
2022-03-28 11:56:38 +00:00
Jakob Meng 19b05f689c Fixed job hierarchy for Zuul CI
Added a parent job *-functional-devstack-base which defines basic job
attributes such as job.required-projects. It does not restrict branches
because else Zuul would not find a matching parent job variant during
job freeze when child jobs are on other branches. It does not define
attributes job.override-checkout and job.required-projects.override-\
checkout because else Zuul would use this branch when matching variants
for parent jobs during job freeze.

Jobs *-devstack-{xena,wallaby,train}-ansible-2.{11,12} have been
changed to inherit from *-devstack-base instead of *-devstack-ansible-\
2.{11,12}. The latter do not run for branch stable/1.0.0 which caused
Zuul to dismiss the *-devstack-{xena,wallaby,train}-ansible-2.{11,12}
when collecting parent job variants during job freeze:
The previous parent jobs *-devstack-ansible-2.{11,12} set job.branches
to master so Zuul cannot match that job when it collects job variants
and thus would ignore the child jobs.

Likewise, jobs *-devstack-{xena,wallaby,train}-ansible-2.{11,12} cannot
inherit from any parent job which sets job.required-projects.override-\
checkout on openstack/devstack because Zuul would use that git ref
instead of stable branch defined below to checkout projects of parent
devstack jobs when collecting variants for parent jobs.

Added a warning to the beginning of .zuul.yaml to keep this file in
sync between branches to avoid issues e.g. with job scheduling. Zuul CI
will search in master branch first when collecting job variants during
job freeze which can have unwanted side effects. For example, when
parent job *-base has been changed in stable/1.0.0 branch, Zuul could
still use *-base variants from master branch during job freeze on child
jobs such as *-ussuri-ansible-2.11 etc.

Change-Id: I3ca4ed5795c45a5565a374f04a1ddb29816bf114
2022-03-28 09:11:06 +00:00
Jakob Meng d44915caba Synchronized galaxy.yml and galaxy.yml.in files
Change-Id: Ic93a0d1f93e2d86358085c7442ad73d52b9a55ba
(cherry picked from commit 26bc8a0666)
2022-03-25 19:49:55 +00:00
Jakob Meng e2ee2d0f5d Fixed branch matching for parent jobs when on stable/1.0.0 branch
When a patch is submitted against a branch, Zuul CI will collect job
variants for each ci job and all its parent jobs. If both job.\
override-checkout and job.required-projects.override-checkout
attributes are not defined, then Zuul will for each (parent) job match
the current branch of the patch against all branches of the project in
which the job is defined. If no such branch exist in a project, then no
job variant matches and this job will be ignored [1].
For example, if a patch is submitted for our stable/1.0.0 branch, then
Zuul CI will try to match 'stable/1.0.0' against all branches in the
projects where job ansible-collections-openstack-functional-devstack
and its parent job openstacksdk-functional-devstack are defined. The
first is defined in openstack/ansible-collections-openstack/.zuul.yaml,
so a match for stable/1.0.0 will be found. But openstacksdk-functional-\
devstack is defined in openstack/openstacksdk/.zuul.yaml which has no
branch stable/1.0.0 defined. So Zuul will not schedule job ansible-\
collections-openstack-functional-devstack at all.

The solution is twofold. First, the base jobs such as ansible-\
collections-openstack-functional-devstack have to be changed to always
checkout existing branches in projects which define (parent) jobs.
Using job.override-checkout might have unintended sideeffects because
it will checkout the specified branch for all required projects which
also includes the project which the patch was submitted for. Instead
we set job.required-projects.override-checkout for all projects which
define parent jobs. For example, in ansible-collections-openstack-\
functional-devstack we set job.required-projects.override-checkout to
master for opendev.org/openstack/devstack which defines parent job
openstack-functional-devstack.

In child jobs which (re)define job.override-checkout, we have to change
job.required-projects.override-checkout for all projects which define
parent jobs to the value of job.override-checkout. If we fail to update
job.required-projects.override-checkout then Zuul will checkout the
branch which was defined in the base jobs because job.\
required-projects.override-checkout has higher precedence than job.\
override-checkout.

Setting attribute project.<pipeline>.debug to true helps with debugging
these job scheduling issues. "If this is set to true, Zuul will include
debugging information in reports it makes about items in the pipeline.
This should not normally be set, but in situations were it is difficult
to determine why Zuul did or did not run a certain job, the additional
information this provides may help" [2].

Note, once job scheduling has been completed, Zuul will use a different
algorithm to checkout projects which are listed in job.\
required-projects [3][4]. It will fallback to a default branch if no
matching branch can be found in projects [5].

Ref.:
[1] https://opendev.org/zuul/zuul/src/branch/master/zuul/model.py#L6996
[2] https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.%3Cpipeline%3E.debug
[3] https://zuul-ci.org/docs/zuul/latest/job-content.html#git-repositories
[4] https://opendev.org/zuul/zuul/src/branch/master/zuul/executor/server.py#L1648
[5] https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.default-branch

Change-Id: I31f9607ab7e2e2ae8534429da7f5e5f235560c56
2022-03-24 20:07:07 +00:00
Jakob Meng 771fb68f99 Run jobs for older OpenStack releases on stable/1.0.0 branch only
OpenStack SDK 0.* releases, from OpenStack Zed and earlier, are only
supported by our stable/1.0.0 branch. Our master branch does not
support old SDK releases anymore, so we restrict Zuul CI jobs
which run older OpenStack releases to patches against our
stable/1.0.0 branch.

Change-Id: I45bf5f90ba2265ab3b9faab77b75babf693b52bb
(cherry picked from commit 8708167b5f)
2022-03-24 15:46:20 +00:00
Jakob Meng 5b53433348 Changed jobs against master branch of OpenStack SDK to non-voting
Our collection has not been ported to the new OpenStack SDK 1.* yet.
Until the migration has been completed successfully, we have to set
all jobs which use the master branch of the SDK to non-voting.

Change-Id: I8fbf568ab87360bf34125dbf1d939c075d3764ae
2022-03-24 12:22:38 +01:00
Jakob Meng 7b26b82890 Synchronized Zuul CI configuration across all branches
A unified .zuul.yaml file allows for easier cherry-picking of
patches for Zuul CI jobs across branches.

Change-Id: Iafea23e6be3bbe82a6e71e52b78509a4908a4580
2022-03-23 15:18:24 +01:00
Jakob Meng e28ab9acec Dropped redundant job *-new_sdk
Dropped job ansible-collections-openstack-functional-devstack-ansible-\
new_sdk because its a duplicate of job ansible-collections-openstack-\
functional-devstack-ansible-2.12.

Change-Id: Ide63ac659f07f90cc1663a8f6409f72eb0ae05fa
2022-03-22 11:50:13 +01:00
Jakob Meng 7820d9ca60 Fixed python packages in tox virtualenvs
Reverted commit 031475d42e which changed tox_install_siblings to
false in base job ansible-collections-openstack-functional-devstack.
This caused most jobs to install the latest Python packages of
OpenStack SDK and other requirements from PyPI to tox' virtualenv
instead of respecting the override-checkout keywords.

Ref.: https://opendev.org/openstack/ansible-collections-openstack/commit/031475d42e306808fa1beefdaaf615fa540662a4

Change-Id: Ide693ef95c613454e1cfb2ee1880793a49f6524e
2022-03-21 11:19:52 +01:00
Jakob Meng a162a02536 Added warning about the upcoming release and its breaking changes
Change-Id: I729df817e9893abb483d7397359e23026adbb4ff
2022-03-15 15:30:12 +01:00
Jakob Meng 61e20bb945 Fix assertion after stack deletion
After a stack has been removed with module stack, a call to module
stack_info might still return this stack with its status set to
'DELETE_COMPLETE' and its status_reason defined as 'Stack DELETE
completed successfully'.

Change-Id: Ice843c403669b4a4e1b12ec73db1fb00d1405980
2022-03-14 10:54:08 +01:00
Jakob Meng 879270aa47 Bumped minimum required OpenStack SDK release to SDK 0.36.0 (Train)
For example, to_dict's computed parameter is available since
SDK 0.18 (Stein) only.

Overview on OpenStack SDK versions in various distributions:
* ArchLinux has SDK 0.59.0
* CentOS 7 has SDK 0.36 (Train)
* CentOS 8 has SDK 0.36 (Train), SDK 0.46 (Ussuri),
  SDK 0.50 (Victoria) and SDK 0.55 (Wallaby)
* Debian 10 (Buster) has SDK 0.17.2 and Ansible 2.7.7 which
  does not support Ansible collections anyway. Debian's
  buster-backports repository has Ansible 2.9.16 but backports
  are provided on an as-is basis, with risk of incompatibilities.
* Debian 11 (Bullseye) has SDK 0.50.0
* Ubuntu 18.04 LTS has SDK 0.11.3 which is not supported by
  this collection since the lowest supported version so far
  is 0.13
* Ubuntu 20.04 LTS has SDK 0.46.0
* Red Hat OpenStack (RHOSP) 16.0-16.2 have SDK 0.36 (Train)

Change-Id: I45d3c05c2ec983993aacc7414213b394b59f5552
2022-03-10 13:13:35 +00:00
Jakob Meng 002e55e269 Revert changes to Zuul config which accidentally got merged from stable branch
Change-Id: I83958370433e3fd5161170032e4adf08a812af72
2022-03-09 21:34:01 +01:00
Jakob Meng b640e6207c Release 1.7.1 version
Change-Id: I958ba6890a54c59e0ccdb9249d959c745acfb8e9
2022-03-09 20:17:52 +01:00