OpenStack SDK 0.53 added parameters is_forced_down and updated_at
in openstack/compute/v2/service.py, hence our compute_service_info
module will return different values depending on which release of
the OpenStack SDK is used.
Ref.: 5450c45253
Change-Id: I4b055266555cb91681d0ab6edcaa850e061f3afb
(cherry picked from commit 4a7330364e)
OpenStack SDK 0.53 renamed parameter disables_reason to disabled_reason
in openstack/compute/v2/service.py, hence our compute_service_info
module will return different values depending on which release of
the OpenStack SDK is used.
Ref.: 5450c45253
Change-Id: I1c0f787f7f67c92f92dd106fc8d55580461e4aa3
(cherry picked from commit cf5007d478)
With merging the code for 1.0.0 of OpenStack SDK into the master branch
several of our modules and CI tests broke. To be able to merge patches
we had to set most of our jobs to non-voting, so atm we do not have a
good code coverage in Zuul CI.
This patch temporarily skips broken tests so that we can set our jobs
back to voting and get some basic code coverage from CI. Follow up
patches which fix modules are supposed to readd skipped tests on
occasion.
Change-Id: Ice42d0bdc12c24227a323ad9c5d3fd33870975c4
(cherry picked from commit c10cc9dd8c)
Our *-octavia jobs pull OpenStack SDK from PyPI and PyPI is still
serving the 0.x.x series of the SDK because 1.0.0 has not been
released yet. A recent commit bb25330ddc [1][2] broke compatibility
to older SDK releases prior to 1.0.0 and since then our *-octavia
job on our master branch is failing.
Ref.:
[1] https://review.opendev.org/c/openstack/ansible-collections-openstack/+/839033
[2] bb25330ddc
Change-Id: I4bacc358cca08a71694c590202066c8565a96f02
(cherry picked from commit 3ead86904a)
Ansible Galaxy content importer is using format strings [1] which
are supported since Python 3.6. Our Zuul CI job for OpenStack Queens
uses Ubuntu 16.04 LTS (Xenial Xerus) as its base image which has
Python 3.5 only.
Ref.:
[1] b7140d6b3b/galaxy_importer/main.py (L117)
Change-Id: I5d3b2f71937a0e4ab9a8d49df10744f7d95a7de2
(cherry picked from commit 9f60f0f26d)
Older releases of OpenStack SDK do not support filtering services.
Change-Id: I613c76b8f794ea2024939e07250a50edc5b9e49a
(cherry picked from commit e85a0b809a)
Tag object was introduced to server role in commit c8a5be6b30 [1] to
allow skipping server tests when volumes are not available.
Whenever tag object is specified, Ansible will run those three tasks
in role server. But as our server module has not been ported to
OpenStack SDK 1.0.0 series it will fail even if someone only wants to
test our object ci role.
This patch removes all occurrences of the object tag in the ci server
role. Since it is not used anywhere in our code it will not break ci.
Ref.:
[1] c8a5be6b30
Change-Id: I222fac499c9a3cb16c4581fb4347170a4d97f833
(cherry picked from commit bba1da17c9)
Alongside OpenStack SDK 1.0.0 we will release a new collection version
2.0.0 which is compatible to OpenStack SDK 1.x.x series only. Code in
branch stable/1.0.0 will remain compatible to OpenStack SDK 0.x.x
series only.
At Ansible OpenStack modules PTG on 2022-04-07 it was decided to raise
an error if one is using a incompatible releases of the OpenStack SDK
with our collection. We decided against showing warnings only because
they can be missed easily and functionality will be broken but
probably hardly detectable when using the wrong SDK.
This patch implements the code to raise errors when users are trying
to use our collection with an incompatible SDK release, e.g. use code
from our stable/1.0.0 branch with a OpenStack SDK 1.x.x release.
It does not yet change the minimum and maximum required SDK versions
because OpenStack SDK 1.0.0 has not yet been released to PyPI and
SDK's master branch still does not return a 1.x.x version number.
Change-Id: I1052d21cf8f108dbc99619cd4c4072488645b855
(cherry picked from commit bc6622e0e7)
The oldest branch in Heat repository is stable/rocky. Previously,
Zuul CI would use the master branch of Heat since it could not find
stable/queens branch but master branch has incompatibilities with
our Queens job.
Change-Id: Iaeca759cad641d4923fc63489fd65f57d9f1345a
(cherry picked from commit e869564e3c)
Make it compatible with new SDK.
Although this one was already using self.con.dns.zones to retrieve the
zones, it wasn't using the to_dict(computed=False) and was still
removing the location (which is obsolate when you use to_dict.
Change-Id: Ie2a5b772acc0c8c8338f6f1da877564a077e3b7a
(cherry picked from commit 0c6e8bed69)
Commit 879270aa47 [1] bumped the required minimum SDK release
but missed to update two locations in code and docs.
Ref.:
[1] 879270aa47
Change-Id: I725a26b07484619f6f2c460e974821f81d60b153
(cherry picked from commit 5a43bdb873)
Users would have a non-null username only with Identity API v2 which
was available in Keystone of OpenStack Pike. Identity API v2 has been
removed since OpenStack Queens [1].
Function search_users() from OpenStack SDK still returns the username
attribute because of [2][3] but it will always be None since
Keystone's API does not return username since OpenStack Pike.
[1] https://docs.openstack.org/releasenotes/keystone/queens.html
[2] 975cabbdd8/openstack/cloud/_utils.py (L246)
[3] 76b081efe4
Change-Id: I2054dd55662698dabd0f2b3565c31dcd3bf24e5a
(cherry picked from commit 5fc8fca06b)
Running older linter releases while we have a more up to date
Ansible 2.12 based linter does not provide value and thus wastes
ci resources. Our Ansible 2.9 based linter is broken atm and since
it is EOL soon anyway we drop this job. We still have a linter job
based on the last stable release and one based on Ansible's
devel branch.
Our tox environment for Ansible 2.9 will be dropped in a later
patch once all Ansible 2.9 based jobs have been removed.
Change-Id: I9cd3f729b06516bbd9a3c7985b65fcf294c8bdd7
(cherry picked from commit f09cccdb9e)
Python module rstcheck which is used by ansible-test deprecated
Python versions prior 3.7 in version 3.5.0 and removed support
in 4.0.0 [1]. Ubuntu 18.04 LTS (Bionic Beaver) comes with Python
3.6 by default, so rstcheck prints a FutureWarning which confuses
ansible-test:
Run command: ***/python -m rstcheck --report warning
--ignore-substitutions _,br,release,today,version
docs/openstack_guidelines.rst
Traceback (most recent call last):
File "***/ansible-test", line 28, in <module>
main()
File "***/ansible-test", line 24, in main
cli_main()
File "***/ansible_test/_internal/cli.py", line 130, in main
args.func(config)
File "***/ansible_test/_internal/sanity/__init__.py", line 193,
in command_sanity
result = test.test(args, sanity_targets, version)
File "***/ansible_test/_internal/sanity/rstcheck.py", line 80,
in test
results = parse_to_list_of_dict(pattern, stderr)
File "***/ansible_test/_internal/util.py", line 799, in
parse_to_list_of_dict
raise Exception('Pattern "%s" did not match values:\n%s' %
(pattern, '\n'.join(unmatched)))
Exception: Pattern "^(?P<path>[^:]*):(?P<line>[0-9]+):
\((?P<level>INFO|WARNING|ERROR|SEVERE)/[0-4]\) (?P<message>.*)$"
did not match values:
***/rstcheck.py:51: FutureWarning: Python versions prior 3.7 are
deprecated. Please update your python version.
FutureWarning
ERROR: Command "/usr/bin/env ANSIBLE_TEST_CONTENT_ROOT=***/
ansible_collections/openstack/cloud LC_ALL=en_US.UTF-8 ***/python3.6
***/ansible-test sanity -v --python 3.6 --skip-test
metaclass-boilerplate --skip-test future-import-boilerplate plugins/
docs/ meta/ scripts/ --metadata ***.json --truncate 0 --redact
--color no --requirements" returned exit status 1.
We cannot constrain the version of rstcheck which ansible-test installs
into its Python virtual environment. This has to be fixed in Ansible
itself, a pull request against Ansible 2.9 has been opened [2].
Ref.:
[1] https://github.com/myint/rstcheck/blob/master/README.rst
[2] https://github.com/ansible/ansible/pull/77568
As a workaround we temporarily set our Ubuntu 18.04 based linter job
openstack-tox-linters-ansible-2.9 to non-voting and remove it from
check dependencies and as a gate job.
Thanks to Arx Cruz and Jesper Schmitz Mouridsen for pointing out this
issue and its root cause ☺️
Change-Id: Ic6f2febc5a40a29534ac4c7f41f714865099086a
(cherry picked from commit 2bf82c2669)
The list of fixed ips on the routers info was wrong, adding only the
last one instead of the all of them.
Change-Id: I0bb352ea1845d25cff3aeae507aa55ba473b0a45
Attributes such as location and tenant_id are computed by OpenStack
SDK. We do not return these fields in floating_ip_info because e.g.
tenant_id has been marked as deprecated and is a copy of the
project_id field.
Ref.: 70a06d9990
Added an assertion to CI which checks that all advertised fields
are returned by floating_ip_info. This helps with detecting breaking
changes in future updates.
Change-Id: I62e4681cd57f82054f68efe1dc59be2cca118135
This allows to execute actions on servers outside of the current auth
scoped project if the user has permission to do so.
Change-Id: Ifb3f40973a76ad8c57bcbcbcb8e73c917681096b
Sorted pip requirements file to improve readability.
Moved pip requirements for tests into tests subdirectory and dropped
'pip-' prefix to shorten filenames and conform with common naming
scheme for pip requirements files.
Added constrains on OpenStack SDK 1.*.* to job ansible-collections-\
openstack-functional-devstack-releases on master branch because only
stable/1.0.0 branch is compatible to the OpenStack SDK 0.*.* series.
Changed job ansible-collections-openstack-functional-devstack-releases
on master branch to non-voting because OpenStack SDK 1.*.* has not
been released to PyPI yet, so tests on master branch are expected to
fail once we introduce breaking changes from stable/1.0.0 branch.
Change-Id: I6b6bb8c6900f7c8341bbf3f9a24999fbf693ba4b
Reverted commit 1f3417cdef [1] which disabled check-import.sh script.
Python module galaxy_importer will return a non-zero return value on
errors since commit 4f5fd0f29c [2].
Use galaxy-importer 0.3.1 for Ansible 2.9 and galaxy-importer 0.3.2
for later Ansible releases because galaxy-importer moved from ansible
2.9 to ansible-core 2.11 in 0.3.2 [3].
Ref.:
[1] 1f3417cdef
[2] 4f5fd0f29c
[3] 9893354783
Change-Id: I898149727d80cd7effe6a04ca77a13ef1774e781
Releases of OpenStack SDK on PyPI will soon switch to the 1.*.* series
which our stable/1.0.0 branch is incompatible with. Previously, in
commit 0f532d10f3, several jobs have been changed to run on master
branch only.
This patch adds siblings jobs for our stable/1.0.0 branch which
pull the latest SDK releases of the 0.*.* series from PyPI instead.
Ref.: 0f532d10f3
Change-Id: Iefc6acfa4c25eb5d9ab062a3bfa655be2188cb77
Job *-train-ansible-2.11 failed since commit 031475d42e because that
patch caused most jobs to install the latest Python packages of
OpenStack SDK and other requirements from PyPI to tox' virtualenv
instead of respecting the override-checkout keywords and using
releases of OpenStack Train. Commit 87858ab976 and its follow ups
has fixed this issue.
Ref.:
031475d42e87858ab976
Change-Id: Ib12e6195db9bb232735ea5a785ccc88bc749ea17
Added a parent job *-functional-devstack-base which defines basic job
attributes such as job.required-projects. It does not restrict branches
because else Zuul would not find a matching parent job variant during
job freeze when child jobs are on other branches. It does not define
attributes job.override-checkout and job.required-projects.override-\
checkout because else Zuul would use this branch when matching variants
for parent jobs during job freeze.
Jobs *-devstack-{xena,wallaby,train}-ansible-2.{11,12} have been
changed to inherit from *-devstack-base instead of *-devstack-ansible-\
2.{11,12}. The latter do not run for branch stable/1.0.0 which caused
Zuul to dismiss the *-devstack-{xena,wallaby,train}-ansible-2.{11,12}
when collecting parent job variants during job freeze:
The previous parent jobs *-devstack-ansible-2.{11,12} set job.branches
to master so Zuul cannot match that job when it collects job variants
and thus would ignore the child jobs.
Likewise, jobs *-devstack-{xena,wallaby,train}-ansible-2.{11,12} cannot
inherit from any parent job which sets job.required-projects.override-\
checkout on openstack/devstack because Zuul would use that git ref
instead of stable branch defined below to checkout projects of parent
devstack jobs when collecting variants for parent jobs.
Added a warning to the beginning of .zuul.yaml to keep this file in
sync between branches to avoid issues e.g. with job scheduling. Zuul CI
will search in master branch first when collecting job variants during
job freeze which can have unwanted side effects. For example, when
parent job *-base has been changed in stable/1.0.0 branch, Zuul could
still use *-base variants from master branch during job freeze on child
jobs such as *-ussuri-ansible-2.11 etc.
Change-Id: I3ca4ed5795c45a5565a374f04a1ddb29816bf114
When a patch is submitted against a branch, Zuul CI will collect job
variants for each ci job and all its parent jobs. If both job.\
override-checkout and job.required-projects.override-checkout
attributes are not defined, then Zuul will for each (parent) job match
the current branch of the patch against all branches of the project in
which the job is defined. If no such branch exist in a project, then no
job variant matches and this job will be ignored [1].
For example, if a patch is submitted for our stable/1.0.0 branch, then
Zuul CI will try to match 'stable/1.0.0' against all branches in the
projects where job ansible-collections-openstack-functional-devstack
and its parent job openstacksdk-functional-devstack are defined. The
first is defined in openstack/ansible-collections-openstack/.zuul.yaml,
so a match for stable/1.0.0 will be found. But openstacksdk-functional-\
devstack is defined in openstack/openstacksdk/.zuul.yaml which has no
branch stable/1.0.0 defined. So Zuul will not schedule job ansible-\
collections-openstack-functional-devstack at all.
The solution is twofold. First, the base jobs such as ansible-\
collections-openstack-functional-devstack have to be changed to always
checkout existing branches in projects which define (parent) jobs.
Using job.override-checkout might have unintended sideeffects because
it will checkout the specified branch for all required projects which
also includes the project which the patch was submitted for. Instead
we set job.required-projects.override-checkout for all projects which
define parent jobs. For example, in ansible-collections-openstack-\
functional-devstack we set job.required-projects.override-checkout to
master for opendev.org/openstack/devstack which defines parent job
openstack-functional-devstack.
In child jobs which (re)define job.override-checkout, we have to change
job.required-projects.override-checkout for all projects which define
parent jobs to the value of job.override-checkout. If we fail to update
job.required-projects.override-checkout then Zuul will checkout the
branch which was defined in the base jobs because job.\
required-projects.override-checkout has higher precedence than job.\
override-checkout.
Setting attribute project.<pipeline>.debug to true helps with debugging
these job scheduling issues. "If this is set to true, Zuul will include
debugging information in reports it makes about items in the pipeline.
This should not normally be set, but in situations were it is difficult
to determine why Zuul did or did not run a certain job, the additional
information this provides may help" [2].
Note, once job scheduling has been completed, Zuul will use a different
algorithm to checkout projects which are listed in job.\
required-projects [3][4]. It will fallback to a default branch if no
matching branch can be found in projects [5].
Ref.:
[1] https://opendev.org/zuul/zuul/src/branch/master/zuul/model.py#L6996
[2] https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.%3Cpipeline%3E.debug
[3] https://zuul-ci.org/docs/zuul/latest/job-content.html#git-repositories
[4] https://opendev.org/zuul/zuul/src/branch/master/zuul/executor/server.py#L1648
[5] https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.default-branch
Change-Id: I31f9607ab7e2e2ae8534429da7f5e5f235560c56
Our collection has not been ported to the new OpenStack SDK 1.* yet.
Until the migration has been completed successfully, we have to set
all jobs which use the master branch of the SDK to non-voting.
Change-Id: I8fbf568ab87360bf34125dbf1d939c075d3764ae
(cherry picked from commit 5b53433348)
OpenStack SDK 0.* releases, from OpenStack Zed and earlier, are only
supported by our stable/1.0.0 branch. Our master branch does not
support old SDK releases anymore, so we restrict Zuul CI jobs
which run older OpenStack releases to patches against our
stable/1.0.0 branch.
Change-Id: I45bf5f90ba2265ab3b9faab77b75babf693b52bb
Reparented ansible-collections-openstack-functional-devstack-{xena,\
wallaby,train}-ansible-2.{11,12} jobs from ansible-collections-\
openstack-functional-devstack-ansible-devel to corresponding ansible-\
collections-openstack-functional-devstack-ansible-2.{9,11,12} jobs
because the previous inheritance hierarchy had no benefits.
The new hierarchy has been straightened, i.e. redundant voting
overrides have been removed and jobs now properly inherit variables
such as tox_envlist and required projects such as github.com/ansible/\
ansible for the specified ansible releases.
Change-Id: I33addad110f4f15ec56dfea0fd18954c55d24b82
Signed-off-by: Kevin Carter <kecarter@redhat.com>
Signed-off-by: Jakob Meng <code@jakobmeng.de>
Our master branch is compatible to OpenStack SDK 1.* releases only,
while our stable/1.0.0 branch works with OpenStack SDK 0.* releases
only. Users of train or wallaby branches of TripleO will use packages
which are provided by RDO and we will pin them to our stable/1.0.0
anyway. Using OpenStack SDK 1.* from PyPI with TripleO train/wallaby
is not supported.
Change-Id: Ia6bcd3809e2a63ec9c11db4eeeca837839c9c0a9
Latest releases of OpenStack SDK will soon be 1.* versioned which
our stable/1.0.0 branch is incompatible with. We will have to add
a siblings job for *-releases later which pulls the latest release
of the 0.* series of the OpenStack SDK and runs on commits for our
stable/1.0.0 branch.
Change-Id: I77f6730e8c9e5840be460795a17c9d48aeecccd0
Reverted commit 031475d42e which changed tox_install_siblings to
false in base job ansible-collections-openstack-functional-devstack.
This caused most jobs to install the latest Python packages of
OpenStack SDK and other requirements from PyPI to tox' virtualenv
instead of respecting the override-checkout keywords.
Ref.: 031475d42e
Change-Id: Ide693ef95c613454e1cfb2ee1880793a49f6524e
Latest releases of OpenStack SDK will soon be 1.* versioned which
our stable/1.0.0 branch is incompatible with. We will have to add
a siblings job for octavia later which pulls the latest release
of the 0.* series of the OpenStack SDK and runs on commits for our
stable/1.0.0 branch.
Change-Id: I8cfc7cf609d10ff96a03ae7f1f407ee0ec2da20c
Run Zuul jobs which test against the master branch of OpenStack SDK
only for commits on our master branch because branch stable/1.0.0 is
compatible only to OpenStack SDK 0.* releases.
Change-Id: I35849df166f3f36f4e3e2870ccd8d5fe9e33e50f
This patch removes "deleted" and "deleted_at" from the doc in image_info module.
These params were never returned since image_info started using openstacksdk.
Change-Id: Id5aa9164bacf7808fd21235bd7327569344295ab
A freshly created host aggregate can have the host list set to None,
consequently you'd hit:
```
failed: [localhost] (item={'name': 'gpu', 'hosts': [], 'metadata': {'type': 'gpu'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"hosts": [], "metadata": {"type": "gpu"}, "name": "gpu"}, "module_stderr": "Traceback (most recent call last):\n File \"/var/lib/home/stackhpc/.ansible/tmp/ansible-tmp-1642696576.6728637-1456290-187052400642084/Ansiba
llZ_host_aggregate.py\", line 100, in <module>\n _ansiballz_main()\n File \"/var/lib/home/stackhpc/.ansible/tmp/ansible-tmp-1642696576.6728637-1456290-187052400642084/AnsiballZ_host_aggregate.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/var/lib/home/stackhpc/.ansible/tmp/ansible-tmp-1642696576.672
8637-1456290-187052400642084/AnsiballZ_host_aggregate.py\", line 41, in invoke_module\n run_name='__main__', alter_sys=True)\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, p
kg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_os_nova_host_aggregate_payload_qwjtdtjj/ansible_os_nova_host_aggregate_payload.zip/ansible_collections/openstack/cloud/plugins/modules/host_aggregate.py\", line 214, in <module>\n File \"/tmp/ansible_os_nova_host_aggregate
_payload_qwjtdtjj/ansible_os_nova_host_aggregate_payload.zip/ansible_collections/openstack/cloud/plugins/modules/host_aggregate.py\", line 210, in main\n File \"/tmp/ansible_os_nova_host_aggregate_payload_qwjtdtjj/ansible_os_nova_host_aggregate_payload.zip/ansible_collections/openstack/cloud/plugins/module_utils/openstack.py\", line 407, in __call__\n File \
"/tmp/ansible_os_nova_host_aggregate_payload_qwjtdtjj/ansible_os_nova_host_aggregate_payload.zip/ansible_collections/openstack/cloud/plugins/modules/host_aggregate.py\", line 176, in run\n File \"/tmp/ansible_os_nova_host_aggregate_payload_qwjtdtjj/ansible_os_nova_host_aggregate_payload.zip/ansible_collections/openstack/cloud/plugins/modules/host_aggregate.py
\", line 138, in _update_hosts\nTypeError: 'NoneType' object is not iterable\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
```
I've not investigated which API and library combinations elict this
behaviour, but it does seem to occur. We can safely handle this
possibility in backwards compatible way.
It is possible to workaround this issue by invoking the module a second
time.
Change-Id: Ie14391f18c0f65833d00a4b4f6b1b314a0903d2b
After a stack has been removed with module stack, a call to module
stack_info might still return this stack with its status set to
'DELETE_COMPLETE' and its status_reason defined as 'Stack DELETE
completed successfully'.
Change-Id: Ice843c403669b4a4e1b12ec73db1fb00d1405980
For example, to_dict's computed parameter is available since
SDK 0.18 (Stein) only.
Overview on OpenStack SDK versions in various distributions:
* ArchLinux has SDK 0.59.0
* CentOS 7 has SDK 0.36 (Train)
* CentOS 8 has SDK 0.36 (Train), SDK 0.46 (Ussuri),
SDK 0.50 (Victoria) and SDK 0.55 (Wallaby)
* Debian 10 (Buster) has SDK 0.17.2 and Ansible 2.7.7 which
does not support Ansible collections anyway. Debian's
buster-backports repository has Ansible 2.9.16 but backports
are provided on an as-is basis, with risk of incompatibilities.
* Debian 11 (Bullseye) has SDK 0.50.0
* Ubuntu 18.04 LTS has SDK 0.11.3 which is not supported by
this collection since the lowest supported version so far
is 0.13
* Ubuntu 20.04 LTS has SDK 0.46.0
* Red Hat OpenStack (RHOSP) 16.0-16.2 have SDK 0.36 (Train)
Change-Id: I45d3c05c2ec983993aacc7414213b394b59f5552
Primary reason for this new release is to fix the mess we did yesterday
when we accidentially merged release 1.7.1 and all changes from the
stable branch into the master branch. This release 1.7.2 does not really
change anything except fixing the guidelines.
Change-Id: I1ad23bd36746a180f8851df294df4e01213bdf2f
