Instead of asserting that openstacksdk created a logging file,
we read the contents of that log file now. The benefit is, that
this integration test gives users an idea how to actually use the
logging feature and also read the contents of the log file.
Change-Id: I9e3c75dda4c1c920937c9780fbdcbe3caeb4d6ad
Sorted argument specs and documentation of both modules.
Refactored both modules to be subclasses of OpenStackModule class.
Renamed baremetal_port's module attributes 'uuid' to 'id' and
'portgroup' to 'port_group' to match openstacksdk. Added the previous
attribute names as aliases to keep backward compatibility. Added
alias 'pxe_enabled' for 'is_pxe_enabled' which was previously set
programmatically.
Changed baremetal_port module to return attribute 'port' only when
state is present. It will return no values (except Ansible's default
values) when state is absent. Previous return value 'id' can be
retrieved from port's dictionary entry 'id'.
The non-standard return value 'result' has been dropped because its
content can easily be reconstructed with Ansible's is changed check.
The non-standard return value 'changes' has been dropped because its
content was only returned on updates, has no known uses and can
easily be reconstructed in Ansible by comparing the updated port
dictionary with a copy of the pre-updated port dictionary.
Module baremetal_port_info will no longer fail when no port with a
matching id or name or address could be found. Instead it will return
an empty list like other *_info modules.
baremetal_port_info's return attribute 'baremetal_ports' has been
renamed to 'ports' to be consistent with other modules. The former
name will keep to be available for now to keep backward compatibility.
Both modules convert their return values into dictionaries without
computed (redundant) values. They do not drop values such as links
anymore though, because we do not withhold information from users.
Updated DOCUMENTATION, EXAMPLES and RETURN docstrings in both modules.
Added integration tests for both modules. They will not run in CI atm,
because we do not have Ironic enabled in our DevStack environment.
Change-Id: I54b3ea9917fbbbdf381ef934a0d92e2857f6d51b
Sorted argument specs and documentation of the module.
Refactored baremetal_node_action module to be a subclass of the
OpenStackModule class.
Redefined baremetal_node_info's module attributes 'id' and 'uuid'
as aliases of the 'name' attribute because modules in this collection
do not differentiate between ids and names. The previous revision of
baremetal_node_info module had the same behaviour implemented but did
not make the relationship between 'id'/'uuid' and 'name' explicit.
Changed types and/or choices of module attributes 'deploy',
'maintenance', 'power' and 'state' to match what has been described
in DOCUMENTATION string and to get rid of the non-Ansible'ish and
inconsistent parsing of input values in _is_true() and _is_false()
functions. Ansible can handle argument types for us, no need to
implement it ourselfs.
Dropped deprecated ironic_url attribute from DOCUMENTATION docstring.
Dropped wait and timeout attributes from DOCUMENTATION because their
docstrings will be added via documentation fragment.
Dropped attribute 'result' from module results because in our modules
we consistently do not explain what we do in modules.
Updated DOCUMENTATION, EXAMPLES and added RETURN docstrings.
Refactored the change logic for maintenance, power state and state,
eliminating unreachable or broken code.
Dropped wait attribute from DOCUMENTATION because its docstring will
be added via documentation fragment.
Kept timeout attribute in DOCUMENTATION and argument_spec because
it has a high(er) default value, to account for long node
(de)activiation times, than what e.g. the generic doc fragment
specifies.
Change-Id: I991f23c16583da106105677d75b3651959280d98
Added integration tests for both modules. They will not run in CI atm,
because we do not have Ironic enabled in our DevStack environment.
Sorted argument specs and documentation of both modules.
Refactored both modules to be subclasses of OpenStackModule class.
Renamed baremetal_node_info's module attribute 'node' to 'name' and
added the former as an alias to be consistent with other *_info
modules.
baremetal_node_info will no longer fetch port and portgroup details
because this requires extra api calls for each node. Users can use
the baremetal_port module to retrieve ports for each node on demand.
Refactored code for constructing node updates in baremetal_node module
which allowed us to drop the dependency on Python module jsonpatch.
Deprecated baremetal_node's skip_update_of_masked_password attribute.
Updating or even specificing passwords for nodes has not been
supported for a while now, rendering the attribute useless.
Renamed baremetal_node's module attributes 'chassis_uuid' to
'chassis_id', 'uuid' to 'id' as well as suboptions of
'properties' to match openstacksdk. Added the previous attribute
names as aliases to keep backward compatibility.
Marked nics attribute in baremetal_node as not updatable.
Changed baremetal_node module to return attribute 'node' only when
state is present. It will return no values (except Ansible's default
values) when state is absent. Previous return value 'uuid' can be
retrieved from node's dictionary entry 'id'.
The non-standard return value 'result' has been dropped because its
content can easily be reconstructed with Ansible's is changed check.
The non-standard return value 'changes' has been dropped because it
was only returned on updates, has no known uses and can easily be
reconstructed in Ansible by comparing the returned node dictionary
with a copy of a previous node dictionary.
Module baremetal_node_info will no longer fail when no node with a
matching id or name or mac could be found. Instead it will return
an empty list like other *_info modules.
baremetal_node_info's return attribute 'baremetal_nodes' has been
renamed to 'nodes' to be consistent with other modules. The former
name will keep to be available for now to keep backward
compatibility.
Both modules convert their return values into dictionaries without
computed (redundant) values. They do not drop values such as links
anymore though, because we do not withhold information from users.
Updated DOCUMENTATION, EXAMPLES and RETURN docstrings in both
modules.
Dropped deprecated ironic_url attribute from DOCUMENTATION docstring
in baremetal_info. Dropped wait attribute from DOCUMENTATION because
its docstring will be added via documentation fragment.
Kept timeout attribute in DOCUMENTATION and argument_spec because
it has a high(er) default value, to account for long provisioning
times, than what e.g. the generic doc fragment specifies.
Change-Id: If3044acf672295e9b61fa60d0969f47cd06dfdeb
Allow to filter stacks by owner names and project names instead of
ids only, in order to be consistent with other *_info modules.
Renamed stack_info's module attributes 'owner_id' to 'owner' and
'project_id' to 'project' to account for the new filter by name
functionality. Added the *_id attributes as aliases to keep
backward compatibility.
Updated and extended DOCUMENTATION, EXAMPLES and RETURN docstrings.
The stack_info module will convert its return values into dictionaries
without computed (redundant) values. Thus dropping values such as
location is not required anymore.
Change-Id: I9cdfb44dd424f63c05943616cf5918ceb3a57b1f
Dropped stack status checks for CREATE_COMPLETE and UPDATE_\
COMPLETE and instead pass wait=True to openstacksdk's create_stack()
and update_stack() calls because those will call event_utils.\
poll_for_events() which garantees that stack has reached those
states when they return [1],[2].
Check for duplicate keys in parameters which already have been
defined by regular module attributes. Raise an error to warn
users that they tried to overwrite parameters which they already
specified with module attributes.
Renamed stack's module attribute 'tag' to 'tags' to match both
openstacksdk and OpenStack API and clarify that more than a
single tag can be specified. Added 'tag' as an alias to keep
backward compatibility.
Actually pass the 'tags' aka 'tag' attribute to stack create
and update functions of the openstacksdk which was lost in [3].
Added tags to integration tests.
Sorted argument specs and documentation of the stack module and
marked attributes which are not updatable.
Dropped condition from self.conn.delete_stack() because the latter
will only return False when a stack could not be found which we
already. self.conn.delete_stack() will raise an exception when
stack deletion fails.
Renamed ci integration tests from 'orchestration' to 'stack' in
order to match module name and adapted tags accordingly.
Fixed and enabled ci integration tests for stack and stack_info
modules.
Dropped 'stack_name' from module return values in RETURN
docstring and ci integration tests because openstacksdk not
return this attribute.
[1] https://opendev.org/openstack/openstacksdk/src/commit/9b1c4333526862d0edfc8e5f7bd421b0065b06db/openstack/cloud/_orchestration.py#L92
[2] https://opendev.org/openstack/openstacksdk/src/commit/9b1c4333526862d0edfc8e5f7bd421b0065b06db/openstack/cloud/_orchestration.py#L148
[3] https://opendev.org/openstack/ansible-collections-openstack/commit/af79857bfb9f391e43eaab0f0c94d8c2b7b74c86
Change-Id: I4ace6012112bbcce9094353e27eb4066cf25f229
We require openstacksdk>=0.99.0 for this collection but we do not
specify versions in module requirements. We probably should do so,
but then consistently across all modules and plugins.
Change-Id: If3d8cd1491266e3332f478267fc41771f280b9a8
Across our modules we use the None values to mark default values for
module attributes. This is in line with openstacksdk's behaviour.
Change-Id: I5920aeeb8eef2ee1c2066a71a273ba52f02305c3
routers_info's interfaces_info attribute is not provided by
openstacksdk, it is added to each router resource by the routers_info
module after retrieving the routers list. To get the required data
list_router_interfaces() [1] is being called for each router resource
which then retrieves all ports for each router. This requires extra
api calls which might be useless because we do not know whether the
user actually cares about the ports. For getting ports of a router
we have the openstack.cloud.ports module. So instead of proactively
retrieving the router ports we drop the interfaces_info attribute.
The interfaces_info attribute was introduced because retrieving
interfaces via openstacksdk and openstack.cloud modules was
complex in the past [2]. Nowadays, using openstack.cloud.ports
Ansible and Jinja2 filters retrieving ip addresses of a router
is straight forward. In case someone still needs the old
interfaces_info attribute, one can refer to the module example
to see how it could be reproduced. But in general, retrieving the
router interfaces is much easier as can be seen in the updated
integration tests.
[1] https://opendev.org/openstack/openstacksdk/src/commit/3f81d0001dd994cde990d38f6e2671ee0694d7d5/openstack/cloud/_network.py#L1926
[2] https://review.opendev.org/c/openstack/ansible-collections-openstack/+/703927/6/plugins/modules/os_routers_info.py
Change-Id: I7fbdf11d07c95421d3aee800bfeebb88ea829817
Define port's module attribute 'name' as a required attribute because
this parameter is used to find, update and delete ports. Technically,
a name is not required to create a port, but idempotency cannot be
implemented without an identifier to refer to a port. In this
collection we use resource names to find and identify resources. We
do not offer a dedicated id attribute in most modules.
Use port's module attribute 'network' when finding, creating,
updating or deleting ports if the user provided this attribute.
This allows to reduce ambiguity when equal names are used across
different networks.
Added 'description' parameter to port module.
Renamed port's module attributes 'vnic_type' to 'binding_vnic_type'
and 'admin_state_up' to 'is_admin_state_up' to match openstacksdk's
attribute names which are used e.g. in module results. Added aliases
for the old attribute names to keep backward compatibility.
Renamed port_info's module attribute 'port' to 'name' and added
the former as an alias to be consistent with other *_info modules.
Dropped default=None and required=False from argument_spec of port
module because those are the default in Ansible [1][2].
Dropped 'id' field from port module's results to be consistent across
other modules. Use 'port.id' instead.
Sorted argument specs and documentation of the port module and
marked attributes which are not updatable.
Updated RETURN fields documentation for the module results of both
port and port_info modules.
Added integration tests to check the update mechanism of the port
module.
Added assertions for module results to catch future changes in the
openstacksdk and our Ansible modules.
Dropped openstacksdk version check since we require a recent release
anyway.
Fixed indentation in integration tests.
Merged integration tests of port_info module into port module,
because the former does not create any ports and assumes that
ports have been created earlier.
[1] https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html
[2] https://github.com/ansible/ansible/blob/61af59c8082cff068424d1012271daa0aac97288/lib/ansible/module_utils/common/parameters.py#L489
Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: Iacca78649f8e01ae95649d8d462f5d0a1740405e
Expanded and fixed module results docs.
Moved ext_ips_spec into the module class because global scope is not
necessary. Renamed it to external_fixed_ips_spec to explain its
purpose.
Sorted argument_spec and attribute docs by attribute name and fixed
indentations. Marked router attributes which cannot be updated.
Mark network attribute as required by enable_snat and
external_fixed_ips attributes. Fixed docstring of network attribute:
Module attribute interfaces does not require the network attribute,
its external_fixed_ips what requires network to be set.
Changed examples from deprecated ip to current ip_address attribute.
Dropped self.fail() calls and let openstacksdk handle missing networks,
subnets.. instead because less code means less code to maintain.
Limited line length to 80 chars to be consistent with other OpenStack
projects. Personally, I prefer a 120 chars limit but consistency is more
important.
Added explanation in code comment why we cannot update a router's name.
Moved upfront cleanup operations in router's integration tests to the
beginning of the role.
Assigned meaningful names to result variables
in router's ci role to easily identify which modules produced the
data.
Added tests for router ids, admin state and interfaces.
Change-Id: Icae77a43479fb4f0bae065d1c5d7942cb0f5fd6b
Allow to update server attributes such as its description.
Changed default value of server attribute 'security_groups' from
['default'] to [] because the latter is the default in
python-openstackclient [1] and the former behavior causes issues
with existing servers [2]: Previously, when no 'security_groups'
parameter was given, the server module would change existing
servers to use the default security group, dropping all other
security groups assigned to the server.
Our (undocumented) guideline when writing modules is to only
add or change what has been requested by the user and to stick to
defaults from openstacksdk and python-openstackclient whenever
possible. Since we have to break backward compatibility with the
next release anyway, we take this opportunity to clean up this odd
behavior. Now, when no security groups are given, then security
groups of an existing server will not be touched.
Closes story #2007893 [2].
Note, Nova will create a server in the default security group,
if the security_groups parameter is omitted.
Dropped 'openstack' field from server module's results. This
variable expanded to additional server information which might
be useful for Ansible inventories and was filled from
openstacksdk's get_openstack_vars() function [3]. Variables in
this function can make additional cloud queries to retrieve
additional data, so calling this function can be expensive [4].
Users can use *_info modules to retrieve this data on-demand.
Dropped 'availabity_zone' attribute from generic OpenStackModule
arguments and inserted it into server and volume modules because
it is relevant to those two modules only. This is completes what
was started years ago [5] and is possible now since we have
breaking changes anyway.
Switched attribute name 'userdata' with its alias 'user_data' to
match openstacksdk's attribute names which are used e.g. in module
results. The previous attribute name 'userdata' is now used as an
alias and 'user_data' is used as the attribute name to keep backward
compatibility.
Wait for server to get into 'ACTIVE' state when creating a server
and attribute 'wait' has been set to true.
Sorted argument specs and documentation of the server module and
marked attributes which are not updatable. Changed unstable bash
script example in server module documentation.
Renamed server's module attribute 'delete_fip' to 'delete_ips' to
match openstacksdk and clarify that it includes all floating ip
addresses of the server.
Renamed server_info's module attribute 'server' to 'name' and added
the former as an alias to be consistent with other *_info modules.
Added RETURN fields documentation for the module results of both
server and server_info modules.
Added description and examples of how to use the 'filters' attribute
of the server_info module. Closes story #2007873 [6].
Removed 'openstack_' prefix from module results because the prefix is
not consistently used across modules, is more to type without any
benefit and removal of the prefix allows us to signal to users that
their code for handling module results has to be updated. Many modules
have different return values with openstacksdk >= 0.99.0 because it
consistently uses resource proxies now.
Added assertions for module results to catch future changes in the
openstacksdk and our Ansible modules.
Added integration tests to check the update mechanism of the server
module.
Fixed indentation in integration tests.
Ensure proper creation and deletion of resources such as networks,
subnets and servers in integration tests of server_action module.
Renamed ci/roles/server/defaults/main.yaml to main.yml, removing the
'a' in the file extension to be consistent with other filenames.
Dropped deprecated function openstack_find_nova_addresses() and
incorporated its code directly into the server module because it
is not used anywhere else.
[1] https://opendev.org/openstack/python-openstackclient/src/commit/e49ad1795b9dd57d5a82fb6f8f365fa20041cf29/openstackclient/compute/v2/server.py#L1070
[2] https://storyboard.openstack.org/#!/story/2007893
[3] https://opendev.org/openstack/openstacksdk/src/commit/9e9fc9879583943a08f854980cca5dfb3a5832f7/openstack/cloud/_compute.py#L1772
[4] https://opendev.org/openstack/openstacksdk/src/commit/9e9fc9879583943a08f854980cca5dfb3a5832f7/openstack/cloud/meta.py#L482
[5] https://github.com/ansible/ansible/commit/9bf33e56dd49e9478ba091a1ba12d17b7caeac24
[6] https://storyboard.openstack.org/#!/story/2007873
Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: I2f955519a7e8c782b1dab8f94f7a019ed384b81d
With Ansible OpenStack collection 2.0.0 we break backward
compatibility to older releases, mainly due to breaking changes
coming with openstacksdk >=0.99.0. For example, results will change
for most Ansible modules in this collection.
We take this opportunity to drop the symbolic links with prefix
os_ in plugins/modules and the plugin routing in meta/runtime.yml.
This means users have to call modules of the Ansible OpenStack
collection using their FQCN (Fully Qualified Collection Name) such
as openstack.cloud.server. Short module names such as os_server
will now raise an Ansible error. This also decreases the likelihood
of incompatible Ansible code going undetected.
Symbolic links were introduced to keep our collection backward
compatible to user code which was written for old(er) Ansible releases
which did not have support for collections and where OpenStack modules
where named with a prefix os_ such as os_server which is nowadays
known and stored as openstack.cloud.server.
In Ansible aka ansible-base 2.10, a internal routing table
lib/ansible/config/ansible_builtin_runtime.yml [1] was introduced which
Ansible uses to resolve deprecated module names missing the FQCN (Fully
Qualified Collection Name). Additionally, collections can define their
own plugin routing table in meta/runtime.yml [2] which we did.
In ansible-base 2.10 and ansible-core 2.11 or later, if a user uses a
short module name and the collections keyword is not used, Ansible
will first look in the internal routing table, get an FQCN, and then
looks in the collection for that FQCN. If there is another routing
entry for that new name in that collection's meta/runtime.yml,
Ansible will continue with that redirect. If it does not find another
redirect, Ansible will look for the plugin itself, so it will not
find a redirect in the collection before looking at its internal
redirects. Except if the user uses a FQCN, then it looks directly in
that collection.
Ansible 2.9 and 2.8 do not have any notion of these redirects with a
plugin routing table, backward compatibility with deprecated os_*
module names is solely achieved with symbolic links. Ansible releases
older than 2.11 are EOL [3], so usage of os_* symlinks should reduce
soon.
[1] https://github.com/ansible/ansible/blob/devel/lib/ansible/config/ansible_builtin_runtime.yml
[2] https://github.com/openstack/ansible-collections-openstack/blob/master/meta/runtime.yml
[3] https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html
Change-Id: I28cc05c95419b72552899c926721eb87fb6f0868
Build and install latest RPM for openstacksdk from its master branch
instead of using the (pinned) RPM from RDO. This is necessary because
openstacksdk in RDO is currently pinned to <0.99.0 for all RDO
releases. Variable artg_change_list is used to define what code is
being build from source. The RPM of the Ansible OpenStack collection
does not have build from source because TripleO Quickstart installs
the collection from job.required-projects [1]. The latter shadows the
RPM release which is installed later by TripleO because it has a
higher precedence in ansible.cfg [2][3].
Changed the job hierarchy to other base jobs tripleo-ci-centos-8-\
standalone-build and tripleo-ci-centos-9-standalone-build. This reduces
the number of variables we have to define. It is also cleaner since our
CentOS9 job is no longer based on the CentOS8 job which prevents
issues with job variant collections due to our branched repository.
Added more Ansible modules to files which trigger TripleO jobs,
because Ansible role os_tempest [4] requires those modules and is
called in TripleO jobs. Modules which have been added include:
* openstack.cloud.compute_flavor
* openstack.cloud.image
* openstack.cloud.network
* openstack.cloud.router
* openstack.cloud.subnet
Added tripleo-ci-centos-8-standalone-osa to experimental jobs so that we
can actually run this job on demand easily.
[1] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/quickstart.sh#L123
[2] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/ansible.cfg#L19
[3] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/quickstart.sh#L595
[4] https://opendev.org/openstack/openstack-ansible-os_tempest.git
Change-Id: Ibde318678a3e44fdc297a6f29761eb0c7d77cbc9
Replaced custom code for interface listing with call to openstacksdk.
The original idea was to reduce the number of calls to the OpenStack
API but this kind of optimization is better to be implemented in the
SDK itself [1]. Reimplementing code like this increases our
maintenance burden, does not help other SDK users and increases the
likeliness of bugs. For example, variable allowed_device_owners
introduced a bug, it is not 'network_router_interface_distributed'
but 'network:router_interface_distributed'.
[1] https://review.opendev.org/c/openstack/openstacksdk/+/849967
Change-Id: I9c52de03c53ef29d7cecdf26253c0c00a7cf3689
Previously, all security_group{,_info} and security_group_rule{,_info}
modules were tested in the same Ansible role. This patch splits
tests into two separate Ansible roles to increase readibility and
prevent variable name conflicts, e.g. for expected_fields.
Change-Id: Ifc28435147b3bfe88d4ee5e176469a53b7395dc0
Keep jobs with devel branch of Ansible non-voting to prevent
Ansible from blocking our Zuul CI gates.
Change-Id: I92668b37d42db758e2bae8e791357b72c131a899
Alongside OpenStack SDK 1.0.0 we will release a new collection version
2.0.0 which is compatible to OpenStack SDK 1.x.x series only. Code in
branch stable/1.0.0 will remain compatible to OpenStack SDK 0.x.x
series only. Release candidates for the first major release of
OpenStackSDK 1.x.x will be numbered using 0.99.x versions.
At Ansible OpenStack modules PTG on 2022-04-07 it was decided to raise
an error if one is using a incompatible releases of the OpenStack SDK
with our collection. We decided against showing warnings only because
they can be missed easily and functionality will be broken but
probably hardly detectable when using the wrong SDK.
This patch bumps the minimum required SDK versions to 0.99.0, so that
an error will be raised when users try to use our collection with an
incompatible SDK release, e.g. use code from our master branch with
a OpenStack SDK 0.x.x release.
Change-Id: I3974deabc516379745794806886352279dc4f4a7
Previously, all job definitions where shared across each .zuul.yaml in
both branches. When a job definition was changed in one branch, Zuul CI
could pick the job definition from the other branch, which was not
intended.
The problem arises when mixing explicit job.branches matchers with
implicit branch matching, when defining same jobs on multiple branches.
Zuul CI expects that jobs to be defined in one or the other branch, not
both at the same time. One should only use job.branches matchers from
single-branched projects, e.g. trusted config repos. When defining jobs
in branched repositories one selects which job definition to use by the
branch associated with the triggering event instead.
Each trigger has a branch associated with it, whether it is the branch
targeted by the change being proposed, the branch to which a commit
merged, a branch attached to a timer trigger etc. This branch name is
searched across involved projects in order to determine what job
definition should be used.
The job.branches directive is rarely applied to a job which will be
copied to multiple branches. When you have multiple copies of a job
with the job.branches attribute, Zuul CI could pick any of the job
definitions which might not be the one you expected.
The job.branches attribute is useful in single branch config
repositories where a specific job definition has to be applied to a
specific branch of the repository. Another definition of the job
will exist in another branch of the config repository.
This patch removes job definitions which are specific to other
branches, except for parent jobs which are shared across branches.
Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: Idb8720bd96843b7807dd5cb62b30c1edf3a7a37c
OpenStack SDK 0.53 added parameters is_forced_down and updated_at
in openstack/compute/v2/service.py, hence our compute_service_info
module will return different values depending on which release of
the OpenStack SDK is used.
Ref.: https://github.com/openstack/openstacksdk/commit/5450c4525313581b2ebc0e1ab6ac6252964c595f
Change-Id: I4b055266555cb91681d0ab6edcaa850e061f3afb
OpenStack SDK 0.53 renamed parameter disables_reason to disabled_reason
in openstack/compute/v2/service.py, hence our compute_service_info
module will return different values depending on which release of
the OpenStack SDK is used.
Ref.: https://github.com/openstack/openstacksdk/commit/5450c4525313581b2ebc0e1ab6ac6252964c595f
Change-Id: I1c0f787f7f67c92f92dd106fc8d55580461e4aa3
With merging the code for 1.0.0 of OpenStack SDK into the master branch
several of our modules and CI tests broke. To be able to merge patches
we had to set most of our jobs to non-voting, so atm we do not have a
good code coverage in Zuul CI.
This patch temporarily skips broken tests so that we can set our jobs
back to voting and get some basic code coverage from CI. Follow up
patches which fix modules are supposed to readd skipped tests on
occasion.
Change-Id: Ice42d0bdc12c24227a323ad9c5d3fd33870975c4
Tag object was introduced to server role in commit c8a5be6b30 [1] to
allow skipping server tests when volumes are not available.
Whenever tag object is specified, Ansible will run those three tasks
in role server. But as our server module has not been ported to
OpenStack SDK 1.0.0 series it will fail even if someone only wants to
test our object ci role.
This patch removes all occurrences of the object tag in the ci server
role. Since it is not used anywhere in our code it will not break ci.
Ref.:
[1] https://opendev.org/openstack/ansible-collections-openstack/commit/c8a5be6b3022140da41dc10a2d0a6a147ad170cf
Change-Id: I222fac499c9a3cb16c4581fb4347170a4d97f833
Alongside OpenStack SDK 1.0.0 we will release a new collection version
2.0.0 which is compatible to OpenStack SDK 1.x.x series only. Code in
branch stable/1.0.0 will remain compatible to OpenStack SDK 0.x.x
series only.
At Ansible OpenStack modules PTG on 2022-04-07 it was decided to raise
an error if one is using a incompatible releases of the OpenStack SDK
with our collection. We decided against showing warnings only because
they can be missed easily and functionality will be broken but
probably hardly detectable when using the wrong SDK.
This patch implements the code to raise errors when users are trying
to use our collection with an incompatible SDK release, e.g. use code
from our stable/1.0.0 branch with a OpenStack SDK 1.x.x release.
It does not yet change the minimum and maximum required SDK versions
because OpenStack SDK 1.0.0 has not yet been released to PyPI and
SDK's master branch still does not return a 1.x.x version number.
Change-Id: I1052d21cf8f108dbc99619cd4c4072488645b855
The oldest branch in Heat repository is stable/rocky. Previously,
Zuul CI would use the master branch of Heat since it could not find
stable/queens branch but master branch has incompatibilities with
our Queens job.
Change-Id: Iaeca759cad641d4923fc63489fd65f57d9f1345a
Running older linter releases while we have a more up to date
Ansible 2.12 based linter does not provide value and thus wastes
ci resources. Our Ansible 2.9 based linter is broken atm and since
it is EOL soon anyway we drop this job. We still have a linter job
based on the last stable release and one based on Ansible's
devel branch.
Our tox environment for Ansible 2.9 will be dropped in a later
patch once all Ansible 2.9 based jobs have been removed.
Change-Id: I9cd3f729b06516bbd9a3c7985b65fcf294c8bdd7
Python module rstcheck which is used by ansible-test deprecated
Python versions prior 3.7 in version 3.5.0 and removed support
in 4.0.0 [1]. Ubuntu 18.04 LTS (Bionic Beaver) comes with Python
3.6 by default, so rstcheck prints a FutureWarning which confuses
ansible-test:
Run command: ***/python -m rstcheck --report warning
--ignore-substitutions _,br,release,today,version
docs/openstack_guidelines.rst
Traceback (most recent call last):
File "***/ansible-test", line 28, in <module>
main()
File "***/ansible-test", line 24, in main
cli_main()
File "***/ansible_test/_internal/cli.py", line 130, in main
args.func(config)
File "***/ansible_test/_internal/sanity/__init__.py", line 193,
in command_sanity
result = test.test(args, sanity_targets, version)
File "***/ansible_test/_internal/sanity/rstcheck.py", line 80,
in test
results = parse_to_list_of_dict(pattern, stderr)
File "***/ansible_test/_internal/util.py", line 799, in
parse_to_list_of_dict
raise Exception('Pattern "%s" did not match values:\n%s' %
(pattern, '\n'.join(unmatched)))
Exception: Pattern "^(?P<path>[^:]*):(?P<line>[0-9]+):
\((?P<level>INFO|WARNING|ERROR|SEVERE)/[0-4]\) (?P<message>.*)$"
did not match values:
***/rstcheck.py:51: FutureWarning: Python versions prior 3.7 are
deprecated. Please update your python version.
FutureWarning
ERROR: Command "/usr/bin/env ANSIBLE_TEST_CONTENT_ROOT=***/
ansible_collections/openstack/cloud LC_ALL=en_US.UTF-8 ***/python3.6
***/ansible-test sanity -v --python 3.6 --skip-test
metaclass-boilerplate --skip-test future-import-boilerplate plugins/
docs/ meta/ scripts/ --metadata ***.json --truncate 0 --redact
--color no --requirements" returned exit status 1.
We cannot constrain the version of rstcheck which ansible-test installs
into its Python virtual environment. This has to be fixed in Ansible
itself, a pull request against Ansible 2.9 has been opened [2].
Ref.:
[1] https://github.com/myint/rstcheck/blob/master/README.rst
[2] https://github.com/ansible/ansible/pull/77568
As a workaround we temporarily set our Ubuntu 18.04 based linter job
openstack-tox-linters-ansible-2.9 to non-voting and remove it from
check dependencies and as a gate job.
Thanks to Arx Cruz and Jesper Schmitz Mouridsen for pointing out this
issue and its root cause ☺️
Change-Id: Ic6f2febc5a40a29534ac4c7f41f714865099086a
Sorted pip requirements file to improve readability.
Moved pip requirements for tests into tests subdirectory and dropped
'pip-' prefix to shorten filenames and conform with common naming
scheme for pip requirements files.
Added constrains on OpenStack SDK 1.*.* to job ansible-collections-\
openstack-functional-devstack-releases on master branch because only
stable/1.0.0 branch is compatible to the OpenStack SDK 0.*.* series.
Changed job ansible-collections-openstack-functional-devstack-releases
on master branch to non-voting because OpenStack SDK 1.*.* has not
been released to PyPI yet, so tests on master branch are expected to
fail once we introduce breaking changes from stable/1.0.0 branch.
Change-Id: I6b6bb8c6900f7c8341bbf3f9a24999fbf693ba4b
Releases of OpenStack SDK on PyPI will soon switch to the 1.*.* series
which our stable/1.0.0 branch is incompatible with. Previously, in
commit 0f532d10f3, several jobs have been changed to run on master
branch only.
This patch adds siblings jobs for our stable/1.0.0 branch which
pull the latest SDK releases of the 0.*.* series from PyPI instead.
Ref.: https://opendev.org/openstack/ansible-collections-openstack/commit/0f532d10f373a3c6cf26be09045da5a934214729
Change-Id: Iefc6acfa4c25eb5d9ab062a3bfa655be2188cb77
Added a parent job *-functional-devstack-base which defines basic job
attributes such as job.required-projects. It does not restrict branches
because else Zuul would not find a matching parent job variant during
job freeze when child jobs are on other branches. It does not define
attributes job.override-checkout and job.required-projects.override-\
checkout because else Zuul would use this branch when matching variants
for parent jobs during job freeze.
Jobs *-devstack-{xena,wallaby,train}-ansible-2.{11,12} have been
changed to inherit from *-devstack-base instead of *-devstack-ansible-\
2.{11,12}. The latter do not run for branch stable/1.0.0 which caused
Zuul to dismiss the *-devstack-{xena,wallaby,train}-ansible-2.{11,12}
when collecting parent job variants during job freeze:
The previous parent jobs *-devstack-ansible-2.{11,12} set job.branches
to master so Zuul cannot match that job when it collects job variants
and thus would ignore the child jobs.
Likewise, jobs *-devstack-{xena,wallaby,train}-ansible-2.{11,12} cannot
inherit from any parent job which sets job.required-projects.override-\
checkout on openstack/devstack because Zuul would use that git ref
instead of stable branch defined below to checkout projects of parent
devstack jobs when collecting variants for parent jobs.
Added a warning to the beginning of .zuul.yaml to keep this file in
sync between branches to avoid issues e.g. with job scheduling. Zuul CI
will search in master branch first when collecting job variants during
job freeze which can have unwanted side effects. For example, when
parent job *-base has been changed in stable/1.0.0 branch, Zuul could
still use *-base variants from master branch during job freeze on child
jobs such as *-ussuri-ansible-2.11 etc.
Change-Id: I3ca4ed5795c45a5565a374f04a1ddb29816bf114
When a patch is submitted against a branch, Zuul CI will collect job
variants for each ci job and all its parent jobs. If both job.\
override-checkout and job.required-projects.override-checkout
attributes are not defined, then Zuul will for each (parent) job match
the current branch of the patch against all branches of the project in
which the job is defined. If no such branch exist in a project, then no
job variant matches and this job will be ignored [1].
For example, if a patch is submitted for our stable/1.0.0 branch, then
Zuul CI will try to match 'stable/1.0.0' against all branches in the
projects where job ansible-collections-openstack-functional-devstack
and its parent job openstacksdk-functional-devstack are defined. The
first is defined in openstack/ansible-collections-openstack/.zuul.yaml,
so a match for stable/1.0.0 will be found. But openstacksdk-functional-\
devstack is defined in openstack/openstacksdk/.zuul.yaml which has no
branch stable/1.0.0 defined. So Zuul will not schedule job ansible-\
collections-openstack-functional-devstack at all.
The solution is twofold. First, the base jobs such as ansible-\
collections-openstack-functional-devstack have to be changed to always
checkout existing branches in projects which define (parent) jobs.
Using job.override-checkout might have unintended sideeffects because
it will checkout the specified branch for all required projects which
also includes the project which the patch was submitted for. Instead
we set job.required-projects.override-checkout for all projects which
define parent jobs. For example, in ansible-collections-openstack-\
functional-devstack we set job.required-projects.override-checkout to
master for opendev.org/openstack/devstack which defines parent job
openstack-functional-devstack.
In child jobs which (re)define job.override-checkout, we have to change
job.required-projects.override-checkout for all projects which define
parent jobs to the value of job.override-checkout. If we fail to update
job.required-projects.override-checkout then Zuul will checkout the
branch which was defined in the base jobs because job.\
required-projects.override-checkout has higher precedence than job.\
override-checkout.
Setting attribute project.<pipeline>.debug to true helps with debugging
these job scheduling issues. "If this is set to true, Zuul will include
debugging information in reports it makes about items in the pipeline.
This should not normally be set, but in situations were it is difficult
to determine why Zuul did or did not run a certain job, the additional
information this provides may help" [2].
Note, once job scheduling has been completed, Zuul will use a different
algorithm to checkout projects which are listed in job.\
required-projects [3][4]. It will fallback to a default branch if no
matching branch can be found in projects [5].
Ref.:
[1] https://opendev.org/zuul/zuul/src/branch/master/zuul/model.py#L6996
[2] https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.%3Cpipeline%3E.debug
[3] https://zuul-ci.org/docs/zuul/latest/job-content.html#git-repositories
[4] https://opendev.org/zuul/zuul/src/branch/master/zuul/executor/server.py#L1648
[5] https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.default-branch
Change-Id: I31f9607ab7e2e2ae8534429da7f5e5f235560c56
OpenStack SDK 0.* releases, from OpenStack Zed and earlier, are only
supported by our stable/1.0.0 branch. Our master branch does not
support old SDK releases anymore, so we restrict Zuul CI jobs
which run older OpenStack releases to patches against our
stable/1.0.0 branch.
Change-Id: I45bf5f90ba2265ab3b9faab77b75babf693b52bb
(cherry picked from commit 8708167b5f)
Our collection has not been ported to the new OpenStack SDK 1.* yet.
Until the migration has been completed successfully, we have to set
all jobs which use the master branch of the SDK to non-voting.
Change-Id: I8fbf568ab87360bf34125dbf1d939c075d3764ae
A unified .zuul.yaml file allows for easier cherry-picking of
patches for Zuul CI jobs across branches.
Change-Id: Iafea23e6be3bbe82a6e71e52b78509a4908a4580
Dropped job ansible-collections-openstack-functional-devstack-ansible-\
new_sdk because its a duplicate of job ansible-collections-openstack-\
functional-devstack-ansible-2.12.
Change-Id: Ide63ac659f07f90cc1663a8f6409f72eb0ae05fa
Reverted commit 031475d42e which changed tox_install_siblings to
false in base job ansible-collections-openstack-functional-devstack.
This caused most jobs to install the latest Python packages of
OpenStack SDK and other requirements from PyPI to tox' virtualenv
instead of respecting the override-checkout keywords.
Ref.: https://opendev.org/openstack/ansible-collections-openstack/commit/031475d42e306808fa1beefdaaf615fa540662a4
Change-Id: Ide693ef95c613454e1cfb2ee1880793a49f6524e
After a stack has been removed with module stack, a call to module
stack_info might still return this stack with its status set to
'DELETE_COMPLETE' and its status_reason defined as 'Stack DELETE
completed successfully'.
Change-Id: Ice843c403669b4a4e1b12ec73db1fb00d1405980
For example, to_dict's computed parameter is available since
SDK 0.18 (Stein) only.
Overview on OpenStack SDK versions in various distributions:
* ArchLinux has SDK 0.59.0
* CentOS 7 has SDK 0.36 (Train)
* CentOS 8 has SDK 0.36 (Train), SDK 0.46 (Ussuri),
SDK 0.50 (Victoria) and SDK 0.55 (Wallaby)
* Debian 10 (Buster) has SDK 0.17.2 and Ansible 2.7.7 which
does not support Ansible collections anyway. Debian's
buster-backports repository has Ansible 2.9.16 but backports
are provided on an as-is basis, with risk of incompatibilities.
* Debian 11 (Bullseye) has SDK 0.50.0
* Ubuntu 18.04 LTS has SDK 0.11.3 which is not supported by
this collection since the lowest supported version so far
is 0.13
* Ubuntu 20.04 LTS has SDK 0.46.0
* Red Hat OpenStack (RHOSP) 16.0-16.2 have SDK 0.36 (Train)
Change-Id: I45d3c05c2ec983993aacc7414213b394b59f5552