Add unit tests for ansible.module_utils.urls (#38059)

* Start of tests for ansible.module_utils.urls

* Start adding file for generic functions throughout urls

* Add tests for maybe_add_ssl_handler

* Remove commented out line

* Improve coverage of maybe_add_ssl_handler, test basic_auth_header

* Start tests for open_url

* pep8 and ignore urlopen in test_url_open.py tests

* Extend auth tests, add test for validate_certs=False

* Finish tests for open_url

* Add tests for fetch_url

* Add fetch_url tests to replace-urlopen ignore

* dummy instead of _

* Add BadStatusLine test

* Reorganize/rename tests

* Add tests for RedirectHandlerFactory

* Add POST test to confirm behavior is to convert to GET

* Update tests to handle recent changes to RedirectHandlerFactory

* Special test, just to confirm that aliasing http_error_308 to http_error_307 does not cause issues with urllib2 type redirects

test/units/module_utils/urls/fixtures/client.key

@@ -0,0 +1,28 @@
+-----BEGIN PRIVATE KEY-----
+MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDTyiVxrsSyZ+Qr
+iMT6sFYCqQtkLqlIWfbpTg9B6fZc793uoMzLUGq3efiZUhhxI78dQ3gNPgs1sK3W
+heFpk1n4IL8ll1MS1uJKk2vYqzZVhjgcvQpeV9gm7bt0ndPzGj5h4fh7proPntSy
+eBvMKVoqTT7tEnapRKy3anbwRPgTt7B5jEvJkPazuIc+ooMsYOHWfvj4oVsev0N2
+SsP0o6cHcsRujFMhz/JTJ1STQxacaVuyKpXacX7Eu1MJgGt/jU/QKNREcV9LdneO
+NgqY9tNv0h+9s7DfHYXm8U3POr+bdcW6Yy4791KGCaUNtiNqT1lvu/4yd4WRkXbF
+Fm5hJUUpAgMBAAECggEBAJYOac1MSK0nEvENbJM6ERa9cwa+UM6kf176IbFP9XAP
+u6zxXWjIR3RMBSmMkyjGbQhs30hypzqZPfH61aUZ8+rsOMKHnyKAAcFZBlZzqIGc
+IXGrNwd1Mf8S/Xg4ww1BkOWFV6s0jCu5G3Z/xyI2Ql4qcOVD6bMwpzclRbQjCand
+dvqyCdMD0sRDyeOIK5hBhUY60JnWbMCu6pBU+qPoRukbRieaeDLIN1clwEqIQV78
+LLnv4n9fuGozH0JdHHfyXFytCgIJvEspZUja/5R4orADhr3ZB010RLzYvs2ndE3B
+4cF9RgxspJZeJ/P+PglViZuzj37pXy+7GAcJLR9ka4kCgYEA/l01XKwkCzMgXHW4
+UPgl1+on42BsN7T9r3S5tihOjHf4ZJWkgYzisLVX+Nc1oUI3HQfM9PDJZXMMNm7J
+ZRvERcopU26wWqr6CFPblGv8oqXHqcpeta8i3xZKoPASsTW6ssuPCEajiLZbQ1rH
+H/HP+OZIVLM/WCPgA2BckTU9JnsCgYEA1SbXllXnlwGqmjitmY1Z07rUxQ3ah/fB
+iccbbg3E4onontYXIlI5zQms3u+qBdi0ZuwaDm5Y4BetOq0a3UyxAsugqVFnzTba
+1w/sFb3fw9KeQ/il4CXkbq87nzJfDmEyqHGCCYXbijHBxnq99PkqwVpaAhHHEW0m
+vWyMUvPRY6sCgYAbtUWR0cKfYbNdvwkT8OQWcBBmSWOgcdvMmBd+y0c7L/pj4pUn
+85PiEe8CUVcrOM5OIEJoUC5wGacz6r+PfwXTYGE+EGmvhr5z18aslVLQ2OQ2D7Bf
+dDOFP6VjgKNYoHS0802iZid8RfkNDj9wsGOqRlOMvnXhAQ9u7rlGrBj8LwKBgFfo
+ph99nH8eE9N5LrfWoUZ+loQS258aInsFYB26lgnsYMEpgO8JxIb4x5BGffPdVUHh
+fDmZbxQ1D5/UhvDgUVzayI8sYMg1KHpsOa0Z2zCzK8zSvu68EgNISCm3J5cRpUft
+UHlG+K19KfMG6lMfdG+8KMUTuetI/iI/o3wOzLvzAoGAIrOh30rHt8wit7ELARyx
+wPkp2ARYXrKfX3NES4c67zSAi+3dCjxRqywqTI0gLicyMlj8zEu9YE9Ix/rl8lRZ
+nQ9LZmqv7QHzhLTUCPGgZYnemvBzo7r0eW8Oag52dbcJO6FBszfWrxskm/fX25Rb
+WPxih2vdRy814dNPW25rgdw=
+-----END PRIVATE KEY-----

test/units/module_utils/urls/fixtures/client.pem

@@ -0,0 +1,81 @@
+Certificate:
+    Data:
+        Version: 3 (0x2)
+        Serial Number: 4099 (0x1003)
+    Signature Algorithm: sha256WithRSAEncryption
+        Issuer: C=US, ST=North Carolina, L=Durham, O=Ansible, CN=ansible.http.tests
+        Validity
+            Not Before: Mar 21 18:22:47 2018 GMT
+            Not After : Mar 18 18:22:47 2028 GMT
+        Subject: C=US, ST=North Carolina, O=Ansible, CN=client.ansible.http.tests
+        Subject Public Key Info:
+            Public Key Algorithm: rsaEncryption
+                Public-Key: (2048 bit)
+                Modulus:
+                    00:d3:ca:25:71:ae:c4:b2:67:e4:2b:88:c4:fa:b0:
+                    56:02:a9:0b:64:2e:a9:48:59:f6:e9:4e:0f:41:e9:
+                    f6:5c:ef:dd:ee:a0:cc:cb:50:6a:b7:79:f8:99:52:
+                    18:71:23:bf:1d:43:78:0d:3e:0b:35:b0:ad:d6:85:
+                    e1:69:93:59:f8:20:bf:25:97:53:12:d6:e2:4a:93:
+                    6b:d8:ab:36:55:86:38:1c:bd:0a:5e:57:d8:26:ed:
+                    bb:74:9d:d3:f3:1a:3e:61:e1:f8:7b:a6:ba:0f:9e:
+                    d4:b2:78:1b:cc:29:5a:2a:4d:3e:ed:12:76:a9:44:
+                    ac:b7:6a:76:f0:44:f8:13:b7:b0:79:8c:4b:c9:90:
+                    f6:b3:b8:87:3e:a2:83:2c:60:e1:d6:7e:f8:f8:a1:
+                    5b:1e:bf:43:76:4a:c3:f4:a3:a7:07:72:c4:6e:8c:
+                    53:21:cf:f2:53:27:54:93:43:16:9c:69:5b:b2:2a:
+                    95:da:71:7e:c4:bb:53:09:80:6b:7f:8d:4f:d0:28:
+                    d4:44:71:5f:4b:76:77:8e:36:0a:98:f6:d3:6f:d2:
+                    1f:bd:b3:b0:df:1d:85:e6:f1:4d:cf:3a:bf:9b:75:
+                    c5:ba:63:2e:3b:f7:52:86:09:a5:0d:b6:23:6a:4f:
+                    59:6f:bb:fe:32:77:85:91:91:76:c5:16:6e:61:25:
+                    45:29
+                Exponent: 65537 (0x10001)
+        X509v3 extensions:
+            X509v3 Basic Constraints: 
+                CA:FALSE
+            Netscape Comment: 
+                OpenSSL Generated Certificate
+            X509v3 Subject Key Identifier: 
+                AF:F3:E5:2A:EB:CF:C7:7E:A4:D6:49:92:F9:29:EE:6A:1B:68:AB:0F
+            X509v3 Authority Key Identifier: 
+                keyid:13:2E:30:F0:04:EA:41:5F:B7:08:BD:34:31:D7:11:EA:56:A6:99:F0
+
+    Signature Algorithm: sha256WithRSAEncryption
+         29:62:39:25:79:58:eb:a4:b3:0c:ea:aa:1d:2b:96:7c:6e:10:
+         ce:16:07:b7:70:7f:16:da:fd:20:e6:a2:d9:b4:88:e0:f9:84:
+         87:f8:b0:0d:77:8b:ae:27:f5:ee:e6:4f:86:a1:2d:74:07:7c:
+         c7:5d:c2:bd:e4:70:e7:42:e4:14:ee:b9:b7:63:b8:8c:6d:21:
+         61:56:0b:96:f6:15:ba:7a:ae:80:98:ac:57:99:79:3d:7a:a9:
+         d8:26:93:30:17:53:7c:2d:02:4b:64:49:25:65:e7:69:5a:08:
+         cf:84:94:8e:6a:42:a7:d1:4f:ba:39:4b:7c:11:67:31:f7:1b:
+         2b:cd:79:c2:28:4d:d9:88:66:d6:7f:56:4c:4b:37:d1:3d:a8:
+         d9:4a:6b:45:1d:4d:a7:12:9f:29:77:6a:55:c1:b5:1d:0e:a5:
+         b9:4f:38:16:3c:7d:85:ae:ff:23:34:c7:2c:f6:14:0f:55:ef:
+         b8:00:89:f1:b2:8a:75:15:41:81:72:d0:43:a6:86:d1:06:e6:
+         ce:81:7e:5f:33:e6:f4:19:d6:70:00:ba:48:6e:05:fd:4c:3c:
+         c3:51:1b:bd:43:1a:24:c5:79:ea:7a:f0:85:a5:40:10:85:e9:
+         23:09:09:80:38:9d:bc:81:5e:59:8c:5a:4d:58:56:b9:71:c2:
+         78:cd:f3:b0
+-----BEGIN CERTIFICATE-----
+MIIDuTCCAqGgAwIBAgICEAMwDQYJKoZIhvcNAQELBQAwZjELMAkGA1UEBhMCVVMx
+FzAVBgNVBAgMDk5vcnRoIENhcm9saW5hMQ8wDQYDVQQHDAZEdXJoYW0xEDAOBgNV
+BAoMB0Fuc2libGUxGzAZBgNVBAMMEmFuc2libGUuaHR0cC50ZXN0czAeFw0xODAz
+MjExODIyNDdaFw0yODAzMTgxODIyNDdaMFwxCzAJBgNVBAYTAlVTMRcwFQYDVQQI
+DA5Ob3J0aCBDYXJvbGluYTEQMA4GA1UECgwHQW5zaWJsZTEiMCAGA1UEAwwZY2xp
+ZW50LmFuc2libGUuaHR0cC50ZXN0czCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
+AQoCggEBANPKJXGuxLJn5CuIxPqwVgKpC2QuqUhZ9ulOD0Hp9lzv3e6gzMtQard5
++JlSGHEjvx1DeA0+CzWwrdaF4WmTWfggvyWXUxLW4kqTa9irNlWGOBy9Cl5X2Cbt
+u3Sd0/MaPmHh+Humug+e1LJ4G8wpWipNPu0SdqlErLdqdvBE+BO3sHmMS8mQ9rO4
+hz6igyxg4dZ++PihWx6/Q3ZKw/SjpwdyxG6MUyHP8lMnVJNDFpxpW7IqldpxfsS7
+UwmAa3+NT9Ao1ERxX0t2d442Cpj202/SH72zsN8dhebxTc86v5t1xbpjLjv3UoYJ
+pQ22I2pPWW+7/jJ3hZGRdsUWbmElRSkCAwEAAaN7MHkwCQYDVR0TBAIwADAsBglg
+hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0O
+BBYEFK/z5Srrz8d+pNZJkvkp7mobaKsPMB8GA1UdIwQYMBaAFBMuMPAE6kFftwi9
+NDHXEepWppnwMA0GCSqGSIb3DQEBCwUAA4IBAQApYjkleVjrpLMM6qodK5Z8bhDO
+Fge3cH8W2v0g5qLZtIjg+YSH+LANd4uuJ/Xu5k+GoS10B3zHXcK95HDnQuQU7rm3
+Y7iMbSFhVguW9hW6eq6AmKxXmXk9eqnYJpMwF1N8LQJLZEklZedpWgjPhJSOakKn
+0U+6OUt8EWcx9xsrzXnCKE3ZiGbWf1ZMSzfRPajZSmtFHU2nEp8pd2pVwbUdDqW5
+TzgWPH2Frv8jNMcs9hQPVe+4AInxsop1FUGBctBDpobRBubOgX5fM+b0GdZwALpI
+bgX9TDzDURu9QxokxXnqevCFpUAQhekjCQmAOJ28gV5ZjFpNWFa5ccJ4zfOw
+-----END CERTIFICATE-----

test/units/module_utils/urls/fixtures/client.txt

@@ -0,0 +1,3 @@
+client.pem and client.key were retrieved from httptester docker image:
+
+ansible/ansible@sha256:fa5def8c294fc50813af131c0b5737594d852abac9cbe7ba38e17bf1c8476f3f

test/units/module_utils/urls/fixtures/netrc

@@ -0,0 +1,3 @@
+machine ansible.com
+login user
+password passwd