Commit Graph

827 Commits

Author SHA1 Message Date
rip777
4e54df71a2 Further iam_group improvements
added `purge_users` and `purge_policy` option
added `convert_friendly_names_to_arns` function
flake8 fixes
2017-06-29 11:38:49 +10:00
Nick Aslanidis
dc8a05dfe2 New AWS module for managing IAM groups 2017-06-29 11:38:48 +10:00
MaciejCetler
0d179c77a7 fix for iam (#23347)
* fix for iam

* Update iam.py

removed extra space
2017-06-28 11:11:19 -04:00
Sloane Hertel
3ba4fc2d54 boto only returns tags in the described security group if they exist (#26053) 2017-06-28 09:14:38 +10:00
Will Thames
0e0b91534a [cloud] Return target_group_names where sensible in ec2_asg_facts module (#26078)
If target_group_arns is an empty list, then return
an empty target_group_names list.

If a connection to elbv2 is not obtainable, then it is
not possible to return target_group_names
2017-06-27 17:04:15 -04:00
Josh Souza
e49f15d6e4 [cloud] Fix logical flaw in route53_health_check, use string ports everywhere (#25706)
* Fix logical flaw (update when diff), use string ports everywhere

* Change port comparison to integer vs. string

The comparison works either way as long as it's consistent. Boto docs
state that it takes in an integer, but if given a string apparently
keeps it as such. This change just ensures that when we compare, we
specifically deal with integers.
2017-06-27 15:58:21 -04:00
Toshio Kuratomi
e238ae999b Cyptography pr 20566 rebase (#25560)
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto

pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.

This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.

* Handle wrong password given for VaultAES format

* Do not display deprecation warning for cryptography on python-2.6

* Namespace all of the pycrypto imports and always import them

  Makes unittests better and the code less likely to get stupid mistakes
  (like using HMAC from cryptogrpahy when the one from pycrypto is needed)

* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko

* contrib/inventory/gce: Remove spurious require on pycrypto

(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)

* Add cryptography to ec2_win_password module requirements
  * Fix python3 bug which would pass text strings to a function which
    requires byte strings.

* Attempt to add pycrypto version to setup deps

* Change hacking README for dual pycrypto/cryptography

* update dependencies for various CI scripts

* additional CI dockerfile/script updates

* add paramiko to the windows and sanity requirement set

  This is needed because ansible lists it as a requirement. Previously
  the missing dep wasn't enforced, but cryptography imports pkg_resources
  so you can't ignore a requirement any more

* Add integration test cases for old vault and for wrong passwords

* helper script for manual testing of pycrypto/cryptography

* Skip the pycrypto tests so that users without it installed can still run the unittests

* Run unittests for vault with both cryptography and pycrypto backend
2017-06-27 06:00:15 -07:00
Sloane Hertel
dc2d9a2134 [cloud] Python3 fixes: use string_types instead of basestring (#26102)
Don't modify a dict while iterating over it
2017-06-26 15:06:53 -04:00
Mark Szymanski
91c1d8d8e2 Fixing bug introduced in 20622 2017-06-26 11:38:47 -04:00
mihu
6b76bc924f [cloud] New feature for ec2_group: allow deleting groups by id (#26022) 2017-06-26 09:07:29 -04:00
Ryan Fitzpatrick
3e52f47b51 [cloud] Fixes #26045 Ensure instance.instance_profile is not None before casting to dict (#26047) 2017-06-23 12:33:55 -04:00
Mark Szymanski
1703db4211 Allow iam_cert to be renamed or moved without needing to specify cert or key data (#20622)
Generally Pep8 and flake8 compliant so removing from legacy-files.txt
2017-06-23 15:32:15 +10:00
Constantin
9cc35b094f Fix of argument error in #25940 (#25979) 2017-06-22 16:59:36 +10:00
mihu
60afe93521 amazon: allow NACL to be removed by its id (#25925) 2017-06-22 16:46:46 +10:00
Wouter de Geus
dd07d11ae5 [cloud] Add ChangeSet support to cloudformation module (#23490) (#24497)
*  * Implements Change Sets on updating a cloudformation stack when create_changeset=true (#23490)

*  * Silence test complaints ;)

*  * Added optional changeset_name parameter.
 * Check if changeset with the requested name already exist.
 * Documentation fix

*  * Added warning when cloudformation stack has pending changesets.
 * Fix documentation
2017-06-21 16:05:17 -04:00
Tom Paine
f831be680e [cloud] Update ec2.py default for public IP assignment (#25896)
* Update ec2.py

Update docs to reflect default.

* Update ec2.py

Further default update.
2017-06-21 14:58:26 -04:00
Sloane Hertel
04e93e22cb add traceback and more helpful errors for ec2_vpc_route_table (#25704) 2017-06-21 12:01:52 +10:00
Yang Liu
b41b4fe2fd use proper pagination variable (#25838) 2017-06-21 12:00:41 +10:00
Damir Suleymanov
74f6b64fb0 Handle rate limits on every iteration (#23726) 2017-06-20 16:49:05 -04:00
Will Thames
f3c98f482a [cloud] New module ec2_vpc_endpoint_facts for AWS EC2 VPC Endpoints (#20211)
* New facts module for AWS EC2 VPC Endpoints

* ec2_vpc_endpoint_facts - meet latest Ansible standards

Fix exception syntax and use of `iteritems` for python3

Fix undefined `ec2` variable (should have been `connection`
Address various flake8 issues

Use `ansible_dict_to_boto3_filter_list` rather than
duplicating its implementation

* Remove max_items and next_token from vpc_endpoint_facts

max_items and next_token should be a module concern, not
a caller concern. It would be very difficult for a module
consumer to use next_token properly, whereas it's easy for
the module to handle it.

* ec2_vpc_endpoint_facts trivially supports check mode

Add supports_check_mode=True to the argument spec.

* Improve RETURN documentation for ec2_vpc_endpoint_facts

Fix bug in EXAMPLE documentation too

* fix return type for validate-modules
2017-06-20 08:31:31 -04:00
Will Rouesnel
3adf08e10f [cloud] iam_cert return arn and allow use with ansible vault (#20787)
* iam_cert.py Fix duplicate certificate detection with included chains.

The iam_cert module would fail to detect certificates as duplicates
if the certificate body included the authority chain directly.

This commit fixes the problem by checking if a given certificate
matches the start of the data returned by AWS, since in all cases
where they would match the certificate will come first.

* iam_cert.py Return certificate ARN in all success cases.

When uploading certificates or interacting with IAM, the certificate ARN
is needed for other operations with AWS such as provisioning elastic load
balancers.

This commit returns the certificate ARN in all success cases, which allows
it to be used to idempotently provision other Amazon services depending on
it (ELBs being an immediate example).
2017-06-20 08:01:56 -04:00
Sloane Hertel
06c21b4bec [cloud] fix dynamodb_ttl exception handling for ParamValidationError (#25870)
Also adds tracebacks to exceptions and imports `HAS_BOTO3` instead of checking directly.
2017-06-19 15:29:14 -04:00
Sloane Hertel
e079498485 [cloud] document route53 state choices added in 2.4 (#25866) 2017-06-19 15:16:58 -04:00
Ted Timmons
346cbf7d7a [cloud] New module: Add module to set Amazon AWS DynamoDB TTL (cloud/amazon/dynamodb_ttl) (#22588)
* add aws dynamo_ttl module, small parameter setter

 - New Module Pull Request

`dynamodb_ttl`

2.3.0/devel

Very self-contained TTL setter. This is independent of the dynamodb_table module
as it's really designed to be a helper for tables that may be created in other
ways (say, CloudFormation, which doesn't support setting TTL).

* committer is no longer a valid value

* bump version_added, catch common exceptions

* pep8 fixes

* one more pep8
2017-06-19 09:57:04 -04:00
Abhijeet Kasurde
6bc1e802e3 Rectify traceback.format_exc() call (#25773)
Fix adds correct call to traceback.format_exc method

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-16 11:38:36 -04:00
Sloane Hertel
938d60472a [cloud] ec2.py: add region to the aws_connect_kwargs before creating vpc connection Fixes #25673 (#25749)
* add region to the aws_connect_kwargs before creating vpc connection

* remove unused import
2017-06-15 15:52:56 -04:00
Robin Miller
540e1bbd69 [cloud] Route53 Avoid throttling errors and unnecessary processing when checking rrsets. (#22104)
The boto Route53 get_all_rrsets method will return the record set
matching the name, type, and identifier specified, followed by ALL
subsequent sets in alphabetical order based on name. If the specified
set does not exist, the method will still return all the sets that
_would_ have come after it. Searching through sets we know will not
match is not just a waste of resources but, more importantly, often
triggers AWS API throttling when used on zones with large numbers of
records.
2017-06-15 15:06:16 -04:00
Rob
6d34ce3f2a [cloud] elb_target_group_facts.pyFix tags being snaked (#25742) 2017-06-15 13:19:28 -04:00
Rob
f8d027b1ba [cloud] New module elb_target_group_facts (#24583) 2017-06-14 12:22:50 -04:00
Sloane Hertel
dfcf43efa9 ec2: Allow EC2-VPC instances to modify security groups - fixes #18928 (#23831)
* Allow EC2-VPC instances to update SG

make ec2 pep8

* use sets instead of loop and a break

* bring things in an indentation level

* Use to_text instead of str, text_type instead of basestring, - instead of difference

* basestrings not unicode

* simplifying syntax
2017-06-14 11:10:29 -04:00
Sloane Hertel
bd4f08d434 [cloud] Fix Route53 module value option, defaulting to empty list (#24555) 2017-06-13 15:55:19 -04:00
Sloane Hertel
3bba43a487 [cloud] ec2.py: region is documented as optional; allow endpoints to be used - fixes #24382 (#24470)
* region isn't required for ec2.py; allow endpoints to be used

* move where aws_connect_kwargs is set

* remove camel_dict_to_snake_dict and display error message
2017-06-13 15:12:47 -04:00
Sloane Hertel
00d5de3dec fix return value (#25611)
document the returned
2017-06-13 09:18:58 -04:00
Simon Staszkiewicz
eb7530df95 Correct link to DescribeInternetGateway docs (#25622)
Old link pointed to DescribeRoutingTable documentation instead of DescribeInternetGateway documentation.
2017-06-12 20:30:30 -04:00
Sloane Hertel
3f0c47196e [cloud] s3: deleting a nonexistent bucket should not give a traceback Fixes #25445 (#25487)
* trying to delete a nonexistent bucket should not fail

* Improve error handling for deleting s3 bucket

* Allow successful deletion

* Add test for deleting a nonexistent bucket

rename integration test target from s3 to aws_s3
2017-06-12 13:52:25 -04:00
Alan Loi
3ac9273507 Fix hash_key_name to be optional for delete (#25009) 2017-06-12 10:25:43 -04:00
Alan Loi
c23e3c1913 Fix version_added for tags and wait_for_active_timeout params (#24993) 2017-06-12 10:19:41 -04:00
willricardo
dd1f5d7acd ec2 module conver basestring to string_types (#25503) 2017-06-12 08:48:38 -04:00
Will Thames
16b877e2b3 ec2_asg and ec2_asg_facts module improvements (#25166)
* ec2_asg and ec2_asg_facts module improvements

Return target group information for both ec2_asg and ec2_asg_facts
modules

Provide RETURN documentation for ec2_asg module

PEP8 fixes for ec2_asg_facts

* ec2_asg: use pagination when describing target groups

In case an ASG has 100s of target groups, ensure that
we get the full result using build_full_result
2017-06-12 07:15:04 -04:00
Abhijeet Kasurde
b89cb95609 Fix spelling mistakes (comments only) (#25564)
Original Author : klemens <ka7@github.com>

Taking over previous PR as per
https://github.com/ansible/ansible/pull/23644#issuecomment-307334525

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-12 07:55:19 +01:00
Chris Atkinson
62fa2019c6 [cloud] Allow IAM certificate secrets, body, chains to be strings (e.g. sourced from ansible-vault) (#24206)
* Allow cert and private pem information to be passed in via string, eg when obtaining sensitive key details from anisble-vault at runtime

* Allow cert chain body to be passed as a string

* Ensure the new options are set in parameters

* Dont publish the private key in logs

* Set the version_added documentation

* Update documentation inline with review

* Removes file based certificates in favour of string only as suggested in feature review

* Documentation changes as suggested by review
2017-06-09 11:32:22 -04:00
Sloane Hertel
b376bde4d7 [cloud] ec2_eni module - allow attaching a new instance - fixes #19452 (#22377)
* fix bug attaching instances

* pep8
2017-06-09 11:31:42 -04:00
Will Thames
3549391673 New facts module for AWS VPC Peering (#19910)
* New facts module for AWS VPC Peering

* Update documentation and region catch for boto3

* Update ec2_vpc_peering_facts for python3 and flake8

Update version to 2.3
Update for python3 exception handling
Use ansible_dict_to_boto3_filter_list rather than duplicating
implementation
Use camel_dict_to_snake_dict on results for ansible standard results
Tidy up flake8 compliance

* Support check_mode for ec2_vpc_peering_facts

Add supports_check_mode to the module declaration

* Fix latest Ansible standards

Update version to 2.4
Add ANSIBLE_METADATA
Move imports
2017-06-06 14:42:58 -04:00
Sloane Hertel
7bebc83dcf module_utils/ec2: fix boto3 tags to remove (#25277)
* Fix boto3 tags to remove; unable to remove a list of strings, need list of {'Key': key}

* pep8 fix

* cannot bugfix due to differences in AWS APIs - just document

* add more info to guidelines for using compare_aws_tags
2017-06-06 11:02:15 -04:00
Michael De La Rue
e28845018d [cloud] New module: AWS API Gageway module (#20230)
* Ultra basic api-gateway module based of lambda.py

* Ultra basic deployment added to api-gateway module

* ApiGateway module Allow creation of APIs, more documentation and better return value

* ApiGateway module incorporate review feedback

* ApiGateway module flake8 cleanup

* APIGateway module - more review fixes.

* slightly better messages in api_gateway module

* AWS api_gateway module - try to improve messages in case of exceptions

* rename api_gateway module to aws_api_gateway as discussed in PR 20230

* aws_api_gateway - Allow delivery of swagger either as text or dictionary.

* aws_api_gateway module - introduce 'unit' tests, improve imports using them and small fixes

* aws_api_gateway module - move path expand_user to avoid early typecheck

* aws_api_gateway - version means version of metadata not module - fix to 1.0

* aws_api_gateway module - Rely on module_utils.ec2 for imports & path type for expanduser / cleanups

* aws_api_gateway module - heavy cleanup and refactor of code + cloud retry functionality.

* api_gateway_module - failing test case for handling more than one deployment in succession and API deletion

* add TooManyRequestsException to AWSRetry exception list - makes API deployment work.

* api_gateway_module - Fixes for various review comments + errors from various linters

* api_gateway_module - Fixes for more review comments + linter error

* api_gateway_module - Major refactor into sensible functions - create_response becomes configure_response

* api_gateway_module - should be working under python3; remove test exclusion

* api_gateway_module - finish off remaining review fixes - use ansible defaults and fix mutually exclusive

* api_gateway_module - attempt to improve handling of botocore errors in python3

* api_gateway_module - implement state=absent / API deletion
2017-06-05 10:25:56 -04:00
Sloane Hertel
77008707f5 ensure port is an int (#25310) 2017-06-02 15:15:47 -04:00
Will Thames
84eea2a7e3 Support check_mode in ec2_vpc_nacl (#23112)
* Support check_mode in ec2_vpc_nacl

Ensure that all API calls that make changes are guarded by
`if not module.check_mode`.

* Update ec2_vpc_nacl_facts to latest pep8 standards
2017-06-02 13:15:38 -04:00
Silvio Tomatis
2ee2c8c1ab [cloud] ec2_elb_lb module Python 3 fix for string types (#24925)
* Python 3 compatibility for ec2_elb_lb module

* Use string_types from ansible.module_utils.six for python2/3 compatibility
2017-06-02 07:40:56 -04:00
Brandon Davidson
0e0c5328fa amazon/ec2_vpc_igw: Allow internet gateways to be tagged (#23782) 2017-06-02 07:38:50 -04:00
Sloane Hertel
7160a491c2 elasticache: add requirements to parameter_group and snapshot (#24904)
* Add requirements to elasticache parameter group

* add requirements for elasticache_snapshot

* pep8 elasticache_snapshot

* remove elasticache_snapshot from legacy files
2017-06-02 00:18:02 +02:00