diff --git a/README.md b/README.md index 9bc1f603..21aed3b4 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,7 @@ Name | Description [kubernetes.core.helm_template](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.helm_template_module.rst)|Render chart templates [kubernetes.core.k8s](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_module.rst)|Manage Kubernetes (K8s) objects [kubernetes.core.k8s_cluster_info](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_cluster_info_module.rst)|Describe Kubernetes (K8s) cluster, APIs available and their respective versions +[kubernetes.core.k8s_cp](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_cp_module.rst)|Copy files and directories to and from pod. [kubernetes.core.k8s_exec](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_exec_module.rst)|Execute command in Pod [kubernetes.core.k8s_info](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_info_module.rst)|Describe Kubernetes (K8s) objects [kubernetes.core.k8s_json_patch](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_json_patch_module.rst)|Apply JSON patch operations to existing objects diff --git a/docs/kubernetes.core.k8s_cp_module.rst b/docs/kubernetes.core.k8s_cp_module.rst new file mode 100644 index 00000000..adad5213 --- /dev/null +++ b/docs/kubernetes.core.k8s_cp_module.rst @@ -0,0 +1,549 @@ +.. _kubernetes.core.k8s_cp_module: + + +********************** +kubernetes.core.k8s_cp +********************** + +**Copy files and directories to and from pod.** + + +Version added: 2.1.0 + +.. contents:: + :local: + :depth: 1 + + +Synopsis +-------- +- Use the Kubernetes Python client to copy files and directories to and from containers inside a pod. + + + +Requirements +------------ +The below requirements are needed on the host that executes this module. + +- python >= 3.6 +- kubernetes >= 12.0.0 + + +Parameters +---------- + +.. raw:: html + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ParameterChoices/DefaultsComments
+
+ api_key + +
+ string +
+ 		+ +
Token used to authenticate with the API. Can also be specified via K8S_AUTH_API_KEY environment variable.
+
+
+ ca_cert + +
+ path +
+ 		+ +
Path to a CA certificate used to authenticate with the API. The full certificate chain must be provided to avoid certificate validation errors. Can also be specified via K8S_AUTH_SSL_CA_CERT environment variable.
+

aliases: ssl_ca_cert
+
+
+ client_cert + +
+ path +
+ 		+ +
Path to a certificate used to authenticate with the API. Can also be specified via K8S_AUTH_CERT_FILE environment variable.
+

aliases: cert_file
+
+
+ client_key + +
+ path +
+ 		+ +
Path to a key file used to authenticate with the API. Can also be specified via K8S_AUTH_KEY_FILE environment variable.
+

aliases: key_file
+
+
+ container + +
+ string +
+ 		+ +
The name of the container in the pod to copy files/directories from/to.
+
Defaults to the only container if there is only one container in the pod.
+
+
+ content + +
+ string +
+ 		+ +
When used instead of local_path, sets the contents of a local file directly to the specified value.
+
Works only when remote_path is a file. Creates the file if it does not exist.
+
For advanced formatting or if the content contains a variable, use the ansible.builtin.template module.
+
Mutually exclusive with local_path.
+
+
+ context + +
+ string +
+ 		+ +
The name of a context found in the config file. Can also be specified via K8S_AUTH_CONTEXT environment variable.
+
+
+ host + +
+ string +
+ 		+ +
Provide a URL for accessing the API. Can also be specified via K8S_AUTH_HOST environment variable.
+
+
+ kubeconfig + +
+ path +
+ 		+ +
Path to an existing Kubernetes config file. If not provided, and no other connection options are provided, the Kubernetes client will attempt to load the default configuration file from ~/.kube/config. Can also be specified via K8S_AUTH_KUBECONFIG environment variable.
+
+
+ local_path + +
+ path +
+ 		+ +
Path of the local file or directory.
+
Required when state is set to from_pod.
+
Mutually exclusive with content.
+
+
+ namespace + +
+ string + / required +
+ 		+ +
The pod namespace name.
+
+
+ no_preserve + +
+ string +
+ 		+ Default:
"no"
+ 		+
The copied file/directory's ownership and permissions will not be preserved in the container.
+
This option is ignored when content is set or when state is set to from_pod.
+
+
+ password + +
+ string +
+ 		+ +
Provide a password for authenticating with the API. Can also be specified via K8S_AUTH_PASSWORD environment variable.
+
Please read the description of the username option for a discussion of when this option is applicable.
+
+
+ persist_config + +
+ boolean +
+ 		+
    Choices: +
  • no
    • +
  • yes
    • +
+ 		+
Whether or not to save the kube config refresh tokens. Can also be specified via K8S_AUTH_PERSIST_CONFIG environment variable.
+
When the k8s context is using a user credentials with refresh tokens (like oidc or gke/gcloud auth), the token is refreshed by the k8s python client library but not saved by default. So the old refresh token can expire and the next auth might fail. Setting this flag to true will tell the k8s python client to save the new refresh token to the kube config file.
+
Default to false.
+
Please note that the current version of the k8s python client library does not support setting this flag to True yet.
+
The fix for this k8s python library is here: https://github.com/kubernetes-client/python-base/pull/169
+
+
+ pod + +
+ string + / required +
+ 		+ +
The pod name.
+
+
+ proxy + +
+ string +
+ 		+ +
The URL of an HTTP proxy to use for the connection. Can also be specified via K8S_AUTH_PROXY environment variable.
+
Please note that this module does not pick up typical proxy settings from the environment (e.g. HTTP_PROXY).
+
+
+ proxy_headers + +
+ dictionary +
+
added in 2.0.0
+ 		+ +
The Header used for the HTTP proxy.
+
Documentation can be found here https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html?highlight=proxy_headers#urllib3.util.make_headers.
+
+
+ basic_auth + +
+ string +
+ 		+ +
Colon-separated username:password for basic authentication header.
+
Can also be specified via K8S_AUTH_PROXY_HEADERS_BASIC_AUTH environment.
+
+
+ proxy_basic_auth + +
+ string +
+ 		+ +
Colon-separated username:password for proxy basic authentication header.
+
Can also be specified via K8S_AUTH_PROXY_HEADERS_PROXY_BASIC_AUTH environment.
+
+
+ user_agent + +
+ string +
+ 		+ +
String representing the user-agent you want, such as foo/1.0.
+
Can also be specified via K8S_AUTH_PROXY_HEADERS_USER_AGENT environment.
+
+
+ remote_path + +
+ path + / required +
+ 		+ +
Path of the file or directory to copy.
+
+
+ state + +
+ string +
+ 		+
    Choices: +
  • to_pod ←
    • +
  • from_pod
    • +
+ 		+
When set to to_pod, the local local_path file or directory will be copied to remote_path into the pod.
+
When set to from_pod, the remote file or directory remote_path from pod will be copied locally to local_path.
+
+
+ username + +
+ string +
+ 		+ +
Provide a username for authenticating with the API. Can also be specified via K8S_AUTH_USERNAME environment variable.
+
Please note that this only works with clusters configured to use HTTP Basic Auth. If your cluster has a different form of authentication (e.g. OAuth2 in OpenShift), this option will not work as expected and you should look into the community.okd.k8s_auth module, as that might do what you need.
+
+
+ validate_certs + +
+ boolean +
+ 		+
    Choices: +
  • no
    • +
  • yes
    • +
+ 		+
Whether or not to verify the API server's SSL certificates. Can also be specified via K8S_AUTH_VERIFY_SSL environment variable.
+

aliases: verify_ssl
+
+
+ + +Notes +----- + +.. note:: + - To avoid SSL certificate validation errors when ``validate_certs`` is *True*, the full certificate chain for the API server must be provided via ``ca_cert`` or in the kubeconfig file. + + + +Examples +-------- + +.. code-block:: yaml + + # kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar + - name: Copy /tmp/foo local file to /tmp/bar in a remote pod + kubernetes.core.k8s: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/bar + local_path: /tmp/foo + + # kubectl cp /tmp/foo_dir some-namespace/some-pod:/tmp/bar_dir + - name: Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod + kubernetes.core.k8s: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/bar_dir + local_path: /tmp/foo_dir + + # kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar -c some-container + - name: Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container + kubernetes.core.k8s: + namespace: some-namespace + pod: some-pod + container: some-container + remote_path: /tmp/bar + local_path: /tmp/foo + no_preserve: True + state: to_pod + + # kubectl cp some-namespace/some-pod:/tmp/foo /tmp/bar + - name: Copy /tmp/foo from a remote pod to /tmp/bar locally + kubernetes.core.k8s: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/foo + local_path: /tmp/bar + state: from_pod + + # copy content into a file in the remote pod + - name: Copy /tmp/foo from a remote pod to /tmp/bar locally + kubernetes.core.k8s: + state: to_pod + namespace: some-namespace + pod: some-pod + remote_path: /tmp/foo.txt + content: "This content will be copied into remote file" + + + +Return Values +------------- +Common return values are documented `here `_, the following are the fields unique to this module: + +.. raw:: html + + + + + + + + + + + + +
KeyReturnedDescription
+
+ result + +
+ string +
+ 		success +
message describing the copy operation successfully done.
+
+
+

+ + +Status +------ + + +Authors +~~~~~~~ + +- Aubin Bikouo (@abikouo) diff --git a/meta/runtime.yml b/meta/runtime.yml index b8a86784..ff4c1484 100644 --- a/meta/runtime.yml +++ b/meta/runtime.yml @@ -14,6 +14,7 @@ action_groups: - k8s_log - k8s_scale - k8s_service + - k8s_cp plugin_routing: action: @@ -43,6 +44,8 @@ plugin_routing: redirect: kubernetes.core.k8s_info k8s_service: redirect: kubernetes.core.k8s_info + k8s_cp: + redirect: kubernetes.core.k8s_info inventory: openshift: redirect: community.okd.openshift diff --git a/molecule/default/converge.yml b/molecule/default/converge.yml index 4f2d9601..bc5b38c8 100644 --- a/molecule/default/converge.yml +++ b/molecule/default/converge.yml @@ -177,6 +177,11 @@ tags: - helm + - role: k8scopy + tags: + - copy + - k8s + post_tasks: - name: Ensure namespace exists k8s: diff --git a/molecule/default/roles/k8scopy/defaults/main.yml b/molecule/default/roles/k8scopy/defaults/main.yml new file mode 100644 index 00000000..06080b06 --- /dev/null +++ b/molecule/default/roles/k8scopy/defaults/main.yml @@ -0,0 +1,15 @@ +--- +# defaults file for k8copy +copy_namespace: copy + +pod_with_one_container: + name: pod-copy-0 + container: container-00 + +pod_with_two_container: + name: pod-copy-1 + container: + - container-10 + - container-11 + +kubectl_path: /tmp/kubectl diff --git a/molecule/default/roles/k8scopy/files/archive.tar b/molecule/default/roles/k8scopy/files/archive.tar new file mode 100644 index 00000000..be47f0b2 Binary files /dev/null and b/molecule/default/roles/k8scopy/files/archive.tar differ diff --git a/molecule/default/roles/k8scopy/files/data/ansible/collection.txt b/molecule/default/roles/k8scopy/files/data/ansible/collection.txt new file mode 100644 index 00000000..2ac78be4 --- /dev/null +++ b/molecule/default/roles/k8scopy/files/data/ansible/collection.txt @@ -0,0 +1 @@ +kubernetes.core diff --git a/molecule/default/roles/k8scopy/files/data/ansible/module.txt b/molecule/default/roles/k8scopy/files/data/ansible/module.txt new file mode 100644 index 00000000..c9931cf8 --- /dev/null +++ b/molecule/default/roles/k8scopy/files/data/ansible/module.txt @@ -0,0 +1 @@ +k8s_cp diff --git a/molecule/default/roles/k8scopy/files/data/file.txt b/molecule/default/roles/k8scopy/files/data/file.txt new file mode 100644 index 00000000..38c57aac --- /dev/null +++ b/molecule/default/roles/k8scopy/files/data/file.txt @@ -0,0 +1 @@ +This is a simple file used to test k8s_cp module on ansible. diff --git a/molecule/default/roles/k8scopy/files/data/teams/ansible.txt b/molecule/default/roles/k8scopy/files/data/teams/ansible.txt new file mode 100644 index 00000000..0073319b --- /dev/null +++ b/molecule/default/roles/k8scopy/files/data/teams/ansible.txt @@ -0,0 +1,2 @@ +cloud team +content team diff --git a/molecule/default/roles/k8scopy/files/hello b/molecule/default/roles/k8scopy/files/hello new file mode 100755 index 00000000..6c29b326 Binary files /dev/null and b/molecule/default/roles/k8scopy/files/hello differ diff --git a/molecule/default/roles/k8scopy/files/simple_file.txt b/molecule/default/roles/k8scopy/files/simple_file.txt new file mode 100644 index 00000000..b26bf561 --- /dev/null +++ b/molecule/default/roles/k8scopy/files/simple_file.txt @@ -0,0 +1 @@ +This content will be copied into remote Pod. \ No newline at end of file diff --git a/molecule/default/roles/k8scopy/files/simple_zip_file.txt.gz b/molecule/default/roles/k8scopy/files/simple_zip_file.txt.gz new file mode 100644 index 00000000..7ecf120e Binary files /dev/null and b/molecule/default/roles/k8scopy/files/simple_zip_file.txt.gz differ diff --git a/molecule/default/roles/k8scopy/library/k8s_create_file.py b/molecule/default/roles/k8scopy/library/k8s_create_file.py new file mode 100644 index 00000000..5e464172 --- /dev/null +++ b/molecule/default/roles/k8scopy/library/k8s_create_file.py @@ -0,0 +1,91 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# Copyright: (c) 2021, Aubin Bikouo <@abikouo> +# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) + +from __future__ import absolute_import, division, print_function + +__metaclass__ = type + + +DOCUMENTATION = r''' + +module: k8s_diff + +short_description: Create large file with a defined size. + +author: + - Aubin Bikouo (@abikouo) + +description: + - This module is used to validate k8s_cp module. + +options: + path: + description: + - The destination path for the file to create. + type: path + required: yes + size: + description: + - The size of the output file in MB. + type: int + default: 400 + binary: + description: + - If this flag is set to yes, the generated file content binary data. + type: bool + default: False +''' + +EXAMPLES = r''' +- name: create 150MB file + k8s_diff: + path: large_file.txt + size: 150 +''' + + +RETURN = r''' +''' + +import os + +from ansible.module_utils.basic import AnsibleModule +from ansible.module_utils._text import to_native + + +def execute_module(module): + try: + size = module.params.get('size') * 1024 * 1024 + path = module.params.get('path') + write_mode = "w" + if module.params.get('binary'): + content = os.urandom(size) + write_mode = "wb" + else: + content = "" + count = 0 + while len(content) < size: + content += f"This file has been generated using ansible: {count}\n" + count += 1 + + with open(path, write_mode) as f: + f.write(content) + module.exit_json(changed=True, size=len(content)) + except Exception as e: + module.fail_json(msg="failed to create file due to: {0}".format(to_native(e))) + + +def main(): + argument_spec = {} + argument_spec['size'] = {'type': 'int', 'default': 400} + argument_spec['path'] = {'type': 'path', 'required': True} + argument_spec['binary'] = {'type': 'bool', 'default': False} + module = AnsibleModule(argument_spec=argument_spec) + + execute_module(module) + + +if __name__ == '__main__': + main() diff --git a/molecule/default/roles/k8scopy/library/kubectl_file_compare.py b/molecule/default/roles/k8scopy/library/kubectl_file_compare.py new file mode 100644 index 00000000..530d038e --- /dev/null +++ b/molecule/default/roles/k8scopy/library/kubectl_file_compare.py @@ -0,0 +1,215 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# Copyright: (c) 2021, Aubin Bikouo <@abikouo> +# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) + +from __future__ import absolute_import, division, print_function + +__metaclass__ = type + + +DOCUMENTATION = r''' + +module: kubectl_file_compare + +short_description: Compare file and directory using kubectl + +author: + - Aubin Bikouo (@abikouo) + +description: + - This module is used to validate k8s_cp module. + - Compare the local file/directory with the remote pod version + +notes: + - This module authenticates on kubernetes cluster using default kubeconfig only. + +options: + namespace: + description: + - The pod namespace name + type: str + required: yes + pod: + description: + - The pod name + type: str + required: yes + container: + description: + - The container to retrieve files from. + type: str + remote_path: + description: + - Path of the file or directory on Pod. + type: path + required: yes + local_path: + description: + - Path of the local file or directory. + type: path + content: + description: + - local content to compare with remote file from pod. + - mutually exclusive with option I(local_path). + type: path + required: yes + args: + description: + - The file is considered to be an executable. + - The tool will be run locally and on pod and compare result from output and stderr. + type: list + kubectl_path: + description: + - Path to the kubectl executable, if not specified it will be download. + type: path +''' + +EXAMPLES = r''' +- name: compare local /tmp/foo with /tmp/bar in a remote pod + kubectl_file_compare: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/bar + local_path: /tmp/foo + kubectl_path: /tmp/test/kubectl + +- name: Compare executable running help command + kubectl_file_compare: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/test/kubectl + local_path: kubectl + kubectl_path: /tmp/test/kubectl + args: + - "--help" +''' + + +RETURN = r''' +''' + +import os +import filecmp + +from tempfile import NamedTemporaryFile, TemporaryDirectory +from ansible.module_utils.basic import AnsibleModule + + +def kubectl_get_content(module, dest_dir): + kubectl_path = module.params.get('kubectl_path') + if kubectl_path is None: + kubectl_path = module.get_bin_path('kubectl', required=True) + + namespace = module.params.get('namespace') + pod = module.params.get('pod') + file = module.params.get('remote_path') + + cmd = [ + kubectl_path, + 'cp', + "{0}/{1}:{2}".format(namespace, pod, file) + ] + container = module.params.get('container') + if container: + cmd += ['-c', container] + local_file = os.path.join(dest_dir, os.path.basename(module.params.get('remote_path'))) + cmd.append(local_file) + rc, out, err = module.run_command(cmd) + return local_file, err, rc, out + + +def kubectl_run_from_pod(module): + kubectl_path = module.params.get('kubectl_path') + if kubectl_path is None: + kubectl_path = module.get_bin_path('kubectl', required=True) + + cmd = [ + kubectl_path, + 'exec', + module.params.get('pod'), + '-n', + module.params.get('namespace') + ] + container = module.params.get('container') + if container: + cmd += ['-c', container] + cmd += ['--', module.params.get('remote_path')] + cmd += module.params.get('args') + return module.run_command(cmd) + + +def compare_directories(dir1, dir2): + test = filecmp.dircmp(dir1, dir2) + if any([len(test.left_only) > 0, len(test.right_only) > 0, len(test.funny_files) > 0]): + return False + (t, mismatch, errors) = filecmp.cmpfiles(dir1, dir2, test.common_files, shallow=False) + if len(mismatch) > 0 or len(errors) > 0: + return False + for common_dir in test.common_dirs: + new_dir1 = os.path.join(dir1, common_dir) + new_dir2 = os.path.join(dir2, common_dir) + if not compare_directories(new_dir1, new_dir2): + return False + return True + + +def execute_module(module): + + args = module.params.get('args') + local_path = module.params.get('local_path') + namespace = module.params.get('namespace') + pod = module.params.get('pod') + file = module.params.get('remote_path') + content = module.params.get('content') + if args: + pod_rc, pod_out, pod_err = kubectl_run_from_pod(module) + rc, out, err = module.run_command([module.params.get('local_path')] + args) + if rc == pod_rc and out == pod_out: + module.exit_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.", rc=rc, stderr=err, stdout=out) + result = dict(local=dict(rc=rc, out=out, err=err), remote=dict(rc=pod_rc, out=pod_out, err=pod_err)) + module.fail_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.", **result) + else: + with TemporaryDirectory() as tmpdirname: + file_from_pod, err, rc, out = kubectl_get_content(module=module, dest_dir=tmpdirname) + if not os.path.exists(file_from_pod): + module.fail_json(msg="failed to copy content from pod", error=err, output=out) + + if content is not None: + with NamedTemporaryFile(mode="w") as tmp_file: + tmp_file.write(content) + tmp_file.flush() + if filecmp.cmp(file_from_pod, tmp_file.name): + module.exit_json(msg=f"defined content and {namespace}/{pod}:{file} are same.") + module.fail_json(msg=f"defined content and {namespace}/{pod}:{file} are same.") + + if os.path.isfile(local_path): + if filecmp.cmp(file_from_pod, local_path): + module.exit_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.") + module.fail_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.") + + if os.path.isdir(local_path): + if compare_directories(file_from_pod, local_path): + module.exit_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.") + module.fail_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.") + + +def main(): + argument_spec = {} + argument_spec['namespace'] = {'type': 'str', 'required': True} + argument_spec['pod'] = {'type': 'str', 'required': True} + argument_spec['container'] = {} + argument_spec['remote_path'] = {'type': 'path', 'required': True} + argument_spec['local_path'] = {'type': 'path'} + argument_spec['content'] = {'type': 'str'} + argument_spec['kubectl_path'] = {'type': 'path'} + argument_spec['args'] = {'type': 'list'} + module = AnsibleModule(argument_spec=argument_spec, + mutually_exclusive=[('local_path', 'content')], + required_one_of=[['local_path', 'content']]) + + execute_module(module) + + +if __name__ == '__main__': + main() diff --git a/molecule/default/roles/k8scopy/meta/main.yml b/molecule/default/roles/k8scopy/meta/main.yml new file mode 100644 index 00000000..d05af689 --- /dev/null +++ b/molecule/default/roles/k8scopy/meta/main.yml @@ -0,0 +1,3 @@ +--- +collections: + - kubernetes.core diff --git a/molecule/default/roles/k8scopy/tasks/main.yml b/molecule/default/roles/k8scopy/tasks/main.yml new file mode 100644 index 00000000..dd823747 --- /dev/null +++ b/molecule/default/roles/k8scopy/tasks/main.yml @@ -0,0 +1,46 @@ +--- +- block: + - name: Download kubeclt executable used to compare results + get_url: + url: https://dl.k8s.io/release/v1.21.3/bin/linux/amd64/kubectl + dest: "{{ kubectl_path }}" + + - name: make kubectl executable + ansible.builtin.file: + path: "{{ kubectl_path }}" + mode: "+x" + + # Ensure namespace and create pod to perform tests on + - name: Ensure namespace exists + k8s: + definition: + apiVersion: v1 + kind: Namespace + metadata: + name: "{{ copy_namespace }}" + + - name: Create Pods + k8s: + namespace: '{{ copy_namespace }}' + wait: yes + template: pods_definition.j2 + + - include_tasks: test_copy_errors.yml + - include_tasks: test_copy_file.yml + - include_tasks: test_multi_container_pod.yml + - include_tasks: test_copy_directory.yml + - include_tasks: test_copy_large_file.yml + + always: + - name: Remove kubectl executable + ansible.builtin.file: + path: "{{ kubectl_path }}" + state: absent + ignore_errors: true + + - name: Remove namespace + k8s: + kind: Namespace + name: "{{ copy_namespace }}" + state: absent + ignore_errors: true diff --git a/molecule/default/roles/k8scopy/tasks/test_copy_directory.yml b/molecule/default/roles/k8scopy/tasks/test_copy_directory.yml new file mode 100644 index 00000000..f91e7b59 --- /dev/null +++ b/molecule/default/roles/k8scopy/tasks/test_copy_directory.yml @@ -0,0 +1,85 @@ +--- +- block: + - name: copy directory into remote Pod (create new directory) + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /dest_data + local_path: files/data + state: to_pod + + - name: compare directories + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /dest_data + local_path: '{{ role_path }}/files/data' + kubectl_path: "{{ kubectl_path }}" + + - name: copy directory into remote Pod (existing directory) + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp + local_path: files/data + state: to_pod + + - name: compare directories + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/data + local_path: '{{ role_path }}/files/data' + kubectl_path: "{{ kubectl_path }}" + + - name: copy directory from Pod into local filesystem (new directory to create) + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/data + local_path: /tmp/test + state: from_pod + + - name: compare directories + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/data + local_path: /tmp/test + kubectl_path: "{{ kubectl_path }}" + + - name: copy directory from Pod into local filesystem (existing directory) + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/data + local_path: /tmp + state: from_pod + + - name: compare directories + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/data + local_path: /tmp/data + kubectl_path: "{{ kubectl_path }}" + + always: + - name: Remove directories created into remote Pod + k8s_exec: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + command: 'rm -rf {{ item }}' + ignore_errors: true + with_items: + - /dest_data + - /tmp/data + + - name: Remove local directories + file: + path: '{{ item }}' + state: absent + ignore_errors: true + with_items: + - /tmp/data + - /tmp/test diff --git a/molecule/default/roles/k8scopy/tasks/test_copy_errors.yml b/molecule/default/roles/k8scopy/tasks/test_copy_errors.yml new file mode 100644 index 00000000..b1e799bf --- /dev/null +++ b/molecule/default/roles/k8scopy/tasks/test_copy_errors.yml @@ -0,0 +1,69 @@ +--- +# copy non-existent local file should fail +- name: copy non-existent file into remote Pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp + local_path: this_file_does_not_exist + state: to_pod + ignore_errors: true + register: copy_non_existent + +- name: check that error message is as expected + assert: + that: + - copy_non_existent is failed + - copy_non_existent.msg == "this_file_does_not_exist does not exist in local filesystem" + +# copy non-existent pod file should fail +- name: copy of non-existent file from remote pod should fail + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /this_file_does_not_exist + local_path: /tmp + state: from_pod + ignore_errors: true + register: copy_non_existent + +- name: check that error message is as expected + assert: + that: + - copy_non_existent is failed + - copy_non_existent.msg == "/this_file_does_not_exist does not exist in remote pod filesystem" + +# copy file into multiple container pod without specifying the container should fail +- name: copy file into multiple container pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /tmp + local_path: files/simple_file.txt + state: to_pod + ignore_errors: true + register: copy_multi_container + +- name: check that error message is as expected + assert: + that: + - copy_multi_container is failed + - copy_multi_container.msg == "Pod contains more than 1 container, option 'container' should be set" + +# copy using non-existent container from pod should failed +- name: copy file into multiple container pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /tmp + local_path: files/simple_file.txt + state: to_pod + container: this_is_a_fake_container + ignore_errors: true + register: copy_fake_container + +- name: check that error message is as expected + assert: + that: + - copy_fake_container is failed + - copy_fake_container.msg == "Pod has no container this_is_a_fake_container" diff --git a/molecule/default/roles/k8scopy/tasks/test_copy_file.yml b/molecule/default/roles/k8scopy/tasks/test_copy_file.yml new file mode 100644 index 00000000..0d611804 --- /dev/null +++ b/molecule/default/roles/k8scopy/tasks/test_copy_file.yml @@ -0,0 +1,191 @@ +--- +- block: + # Text file + - name: copy text file into remote pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp + local_path: files/simple_file.txt + state: to_pod + + - name: Compare files + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/simple_file.txt + content: "{{ lookup('file', 'simple_file.txt')}}" + kubectl_path: "{{ kubectl_path }}" + + - name: Copy simple text file from Pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/simple_file.txt + local_path: /tmp/copy_from_pod.txt + state: from_pod + + - name: Compare files + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/simple_file.txt + local_path: /tmp/copy_from_pod.txt + kubectl_path: "{{ kubectl_path }}" + + # Binary file + - name: Generate random content + set_fact: + hello_arg: "{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=16') }}" + + - name: Copy executable into Pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/hello.exe + local_path: files/hello + state: to_pod + + - name: Compare executable + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/hello.exe + local_path: "{{ role_path }}/files/hello" + kubectl_path: "{{ kubectl_path }}" + args: + - "{{ hello_arg }}" + + - name: Copy executable from Pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/hello.exe + local_path: /tmp/hello + state: from_pod + + - name: update executable permission + file: + path: /tmp/hello + mode: '0755' + + - name: Compare executable + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/hello.exe + local_path: /tmp/hello + kubectl_path: "{{ kubectl_path }}" + args: + - "{{ hello_arg }}" + + # zip files + - name: copy zip file into remote pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp + local_path: files/simple_zip_file.txt.gz + state: to_pod + + - name: compare zip files + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/simple_zip_file.txt.gz + local_path: '{{ role_path }}/files/simple_zip_file.txt.gz' + kubectl_path: "{{ kubectl_path }}" + + - name: copy zip file from pod into local filesystem + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/simple_zip_file.txt.gz + local_path: /tmp/copied_from_pod.txt.gz + state: from_pod + + - name: compare zip files + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/simple_zip_file.txt.gz + local_path: /tmp/copied_from_pod.txt.gz + kubectl_path: "{{ kubectl_path }}" + + # tar files + - name: copy archive into remote pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp + local_path: files/archive.tar + state: to_pod + + - name: compare archive + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/archive.tar + local_path: '{{ role_path }}/files/archive.tar' + kubectl_path: "{{ kubectl_path }}" + + - name: copy archive from remote pod into local filesystem + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/archive.tar + local_path: /tmp/local_archive.tar + state: from_pod + + - name: compare archive + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /tmp/archive.tar + local_path: /tmp/local_archive.tar + kubectl_path: "{{ kubectl_path }}" + + # Copy into Pod using content option + - name: set content to be copied into Pod + set_fact: + pod_content: "{{ lookup('password', '/dev/null chars=ascii_lowercase,digits,punctuation length=128') }}" + + - name: copy archive into remote pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /this_content.txt + content: '{{ pod_content }}' + state: to_pod + + - name: Assert that content is as expected into Pod + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /this_content.txt + content: '{{ pod_content }}' + kubectl_path: "{{ kubectl_path }}" + + always: + - name: Delete file created on Pod + k8s_exec: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + command: 'rm {{ item }}' + ignore_errors: true + with_items: + - /tmp/simple_file.txt + - /tmp/hello.exe + - /tmp/simple_zip_file.txt.gz + - /tmp/archive.tar + - /this_content.txt + + - name: Delete file created locally + file: + path: '{{ item }}' + state: absent + with_items: + - /tmp/copy_from_pod.txt + - /tmp/hello + - /tmp/copied_from_pod.txt.gz + - /tmp/local_archive.tar diff --git a/molecule/default/roles/k8scopy/tasks/test_copy_large_file.yml b/molecule/default/roles/k8scopy/tasks/test_copy_large_file.yml new file mode 100644 index 00000000..4db51fa1 --- /dev/null +++ b/molecule/default/roles/k8scopy/tasks/test_copy_large_file.yml @@ -0,0 +1,103 @@ +--- +- name: test copy of large binary and text files + block: + - set_fact: + test_directory: "/tmp/test_k8scp_large_files" + no_log: true + + - name: create temporary directory for local files + ansible.builtin.file: + path: "{{ test_directory }}" + state: directory + + - name: create large text file + k8s_create_file: + path: "{{ test_directory }}/large_text_file.txt" + size: 150 + + - name: create large binary file + k8s_create_file: + path: "{{ test_directory }}/large_bin_file.bin" + size: 200 + binary: true + + # Copy large text file from/to local filesystem to Pod + - name: copy large file into remote Pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_text_file.txt + local_path: "{{ test_directory }}/large_text_file.txt" + state: to_pod + + - name: Compare files + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_text_file.txt + local_path: "{{ test_directory }}/large_text_file.txt" + kubectl_path: "{{ kubectl_path }}" + + - name: copy large file from Pod into local filesystem + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_text_file.txt + local_path: "{{ test_directory }}/large_text_file_from_pod.txt" + state: from_pod + + - name: Compare files + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_text_file.txt + local_path: "{{ test_directory }}/large_text_file_from_pod.txt" + kubectl_path: "{{ kubectl_path }}" + + # Copy large binary file from/to local filesystem to Pod + - name: copy large file into remote Pod + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_bin_file.bin + local_path: "{{ test_directory }}/large_bin_file.bin" + state: to_pod + + - name: Compare executable, local vs remote + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_bin_file.bin + local_path: "{{ test_directory }}/large_bin_file.bin" + + - name: copy executable from pod into local filesystem + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_bin_file.bin + local_path: "{{ test_directory }}/large_bin_file_from_pod.bin" + state: from_pod + + - name: Compare executable, local vs remote + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + remote_path: /large_bin_file.bin + local_path: "{{ test_directory }}/large_bin_file_from_pod.bin" + + always: + - name: Delete temporary directory created for the test + ansible.builtin.file: + path: "{{ test_directory }}" + state: absent + ignore_errors: true + + - name: Delete file created on Pod + k8s_exec: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_one_container.name }}' + command: 'rm {{ item }}' + ignore_errors: true + with_items: + - /large_text_file.txt + - /large_bin_file.bin diff --git a/molecule/default/roles/k8scopy/tasks/test_multi_container_pod.yml b/molecule/default/roles/k8scopy/tasks/test_multi_container_pod.yml new file mode 100644 index 00000000..6cb4d664 --- /dev/null +++ b/molecule/default/roles/k8scopy/tasks/test_multi_container_pod.yml @@ -0,0 +1,71 @@ +--- +- set_fact: + random_content: "{{ lookup('password', '/dev/null chars=ascii_lowercase,digits,punctuation length=128') }}" + +- name: Copy content into first pod's container + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /file_from_localhost.txt + content: '{{ random_content }}' + container: '{{ pod_with_two_container.container[0] }}' + state: to_pod + +- name: Assert that content has been copied into first container + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /file_from_localhost.txt + container: '{{ pod_with_two_container.container[0] }}' + content: '{{ random_content }}' + kubectl_path: "{{ kubectl_path }}" + +- name: Assert that content has not been copied into second container + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /file_from_localhost.txt + container: '{{ pod_with_two_container.container[1] }}' + content: '{{ random_content }}' + kubectl_path: "{{ kubectl_path }}" + register: diff + ignore_errors: true + +- name: check that diff failed + assert: + that: + - diff is failed + +- name: Copy content into second's pod container + k8s_cp: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /file_from_localhost_01.txt + content: '{{ random_content }}-secondpod' + container: '{{ pod_with_two_container.container[1] }}' + state: to_pod + +- name: Assert that content has not been copied into first container + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /file_from_localhost_01.txt + container: '{{ pod_with_two_container.container[0] }}' + content: '{{ random_content }}-secondpod' + kubectl_path: "{{ kubectl_path }}" + ignore_errors: true + register: diff_1 + +- name: check that diff failed + assert: + that: + - diff_1 is failed + +- name: Assert that content has been copied into second container + kubectl_file_compare: + namespace: '{{ copy_namespace }}' + pod: '{{ pod_with_two_container.name }}' + remote_path: /file_from_localhost_01.txt + container: '{{ pod_with_two_container.container[1] }}' + content: '{{ random_content }}-secondpod' + kubectl_path: "{{ kubectl_path }}" diff --git a/molecule/default/roles/k8scopy/templates/pods_definition.j2 b/molecule/default/roles/k8scopy/templates/pods_definition.j2 new file mode 100644 index 00000000..b5780543 --- /dev/null +++ b/molecule/default/roles/k8scopy/templates/pods_definition.j2 @@ -0,0 +1,33 @@ +--- +apiVersion: v1 +kind: Pod +metadata: + name: '{{ pod_with_one_container.name }}' +spec: + containers: + - name: '{{ pod_with_one_container.container }}' + image: busybox + command: + - /bin/sh + - -c + - while true;do date;sleep 5; done +--- +apiVersion: v1 +kind: Pod +metadata: + name: '{{ pod_with_two_container.name }}' +spec: + containers: + - name: '{{ pod_with_two_container.container[0] }}' + image: busybox:1.32.0 + command: + - /bin/sh + - -c + - while true;do date;sleep 5; done + - name: '{{ pod_with_two_container.container[1] }}' + image: busybox:1.33.0 + command: + - /bin/sh + - -c + - while true;do date;sleep 5; done + diff --git a/plugins/action/k8s_cp.py b/plugins/action/k8s_cp.py new file mode 120000 index 00000000..667980ed --- /dev/null +++ b/plugins/action/k8s_cp.py @@ -0,0 +1 @@ +k8s_info.py \ No newline at end of file diff --git a/plugins/action/k8s_info.py b/plugins/action/k8s_info.py index e214825c..532a6565 100644 --- a/plugins/action/k8s_info.py +++ b/plugins/action/k8s_info.py @@ -8,6 +8,7 @@ __metaclass__ = type import copy import traceback +import os from contextlib import contextmanager @@ -179,6 +180,20 @@ class ActionModule(ActionBase): new_module_args.pop('template') new_module_args['definition'] = result_template + def get_file_realpath(self, local_path): + # local_path is only supported by k8s_cp module. + if self._task.action not in ('k8s_cp', 'kubernetes.core.k8s_cp', 'community.kubernetes.k8s_cp'): + raise AnsibleActionFail("'local_path' is only supported parameter for 'k8s_cp' module.") + + if os.path.exists(local_path): + return local_path + + try: + # find in expected paths + return self._find_needle('files', local_path) + except AnsibleError: + raise AnsibleActionFail("%s does not exist in local filesystem" % local_path) + def run(self, tmp=None, task_vars=None): ''' handler for k8s options ''' if task_vars is None: @@ -238,6 +253,11 @@ class ActionModule(ActionBase): if template: self.load_template(template, new_module_args, task_vars) + local_path = self._task.args.get('local_path') + state = self._task.args.get('state', None) + if local_path and state == 'to_pod': + new_module_args['local_path'] = self.get_file_realpath(local_path) + # Execute the k8s_* module. module_return = self._execute_module(module_name=self._task.action, module_args=new_module_args, task_vars=task_vars) diff --git a/plugins/module_utils/common.py b/plugins/module_utils/common.py index 96b67acf..70c6fb19 100644 --- a/plugins/module_utils/common.py +++ b/plugins/module_utils/common.py @@ -208,13 +208,13 @@ get_api_client._pool = {} class K8sAnsibleMixin(object): - def __init__(self, module, *args, **kwargs): + def __init__(self, module, pyyaml_required=True, *args, **kwargs): if not HAS_K8S_MODULE_HELPER: module.fail_json(msg=missing_required_lib('kubernetes'), exception=K8S_IMP_ERR, error=to_native(k8s_import_exception)) self.kubernetes_version = kubernetes.__version__ - if not HAS_YAML: + if pyyaml_required and not HAS_YAML: module.fail_json(msg=missing_required_lib("PyYAML"), exception=YAML_IMP_ERR) def find_resource(self, kind, api_version, fail=False): diff --git a/plugins/modules/k8s_cp.py b/plugins/modules/k8s_cp.py new file mode 100644 index 00000000..463757b7 --- /dev/null +++ b/plugins/modules/k8s_cp.py @@ -0,0 +1,490 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# Copyright: (c) 2021, Aubin Bikouo <@abikouo> +# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) + +from __future__ import absolute_import, division, print_function +__metaclass__ = type + + +DOCUMENTATION = r''' + +module: k8s_cp + +short_description: Copy files and directories to and from pod. + +version_added: "2.2.0" + +author: + - Aubin Bikouo (@abikouo) + +description: + - Use the Kubernetes Python client to copy files and directories to and from containers inside a pod. + +extends_documentation_fragment: + - kubernetes.core.k8s_auth_options + +requirements: + - "python >= 3.6" + - "kubernetes >= 12.0.0" + +options: + namespace: + description: + - The pod namespace name. + type: str + required: yes + pod: + description: + - The pod name. + type: str + required: yes + container: + description: + - The name of the container in the pod to copy files/directories from/to. + - Defaults to the only container if there is only one container in the pod. + type: str + remote_path: + description: + - Path of the file or directory to copy. + type: path + required: yes + local_path: + description: + - Path of the local file or directory. + - Required when I(state) is set to C(from_pod). + - Mutually exclusive with I(content). + type: path + content: + description: + - When used instead of I(local_path), sets the contents of a local file directly to the specified value. + - Works only when I(remote_path) is a file. Creates the file if it does not exist. + - For advanced formatting or if the content contains a variable, use the M(ansible.builtin.template) module. + - Mutually exclusive with I(local_path). + type: str + state: + description: + - When set to C(to_pod), the local I(local_path) file or directory will be copied to I(remote_path) into the pod. + - When set to C(from_pod), the remote file or directory I(remote_path) from pod will be copied locally to I(local_path). + type: str + default: to_pod + choices: [ to_pod, from_pod ] + no_preserve: + description: + - The copied file/directory's ownership and permissions will not be preserved in the container. + - This option is ignored when I(content) is set or when I(state) is set to C(from_pod). + type: bool + default: False + +notes: + - the tar binary is required on the container when copying from local filesystem to pod. +''' + +EXAMPLES = r''' +# kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar +- name: Copy /tmp/foo local file to /tmp/bar in a remote pod + kubernetes.core.k8s_cp: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/bar + local_path: /tmp/foo + +# kubectl cp /tmp/foo_dir some-namespace/some-pod:/tmp/bar_dir +- name: Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod + kubernetes.core.k8s_cp: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/bar_dir + local_path: /tmp/foo_dir + +# kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar -c some-container +- name: Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container + kubernetes.core.k8s_cp: + namespace: some-namespace + pod: some-pod + container: some-container + remote_path: /tmp/bar + local_path: /tmp/foo + no_preserve: True + state: to_pod + +# kubectl cp some-namespace/some-pod:/tmp/foo /tmp/bar +- name: Copy /tmp/foo from a remote pod to /tmp/bar locally + kubernetes.core.k8s_cp: + namespace: some-namespace + pod: some-pod + remote_path: /tmp/foo + local_path: /tmp/bar + state: from_pod + +# copy content into a file in the remote pod +- name: Copy /tmp/foo from a remote pod to /tmp/bar locally + kubernetes.core.k8s_cp: + state: to_pod + namespace: some-namespace + pod: some-pod + remote_path: /tmp/foo.txt + content: "This content will be copied into remote file" +''' + + +RETURN = r''' +result: + description: + - message describing the copy operation successfully done. + returned: success + type: str +''' + +import copy +import os +from tempfile import TemporaryFile, NamedTemporaryFile +from select import select +from abc import ABCMeta, abstractmethod +import tarfile + +# from ansible_collections.kubernetes.core.plugins.module_utils.ansiblemodule import AnsibleModule +from ansible.module_utils.basic import AnsibleModule +from ansible.module_utils._text import to_native +from ansible_collections.kubernetes.core.plugins.module_utils.common import K8sAnsibleMixin, get_api_client +from ansible_collections.kubernetes.core.plugins.module_utils.args_common import AUTH_ARG_SPEC + +try: + from kubernetes.client.api import core_v1_api + from kubernetes.stream import stream + from kubernetes.stream.ws_client import STDOUT_CHANNEL, STDERR_CHANNEL, ERROR_CHANNEL, ABNF +except ImportError: + pass + +try: + import yaml +except ImportError: + # ImportError are managed by the common module already. + pass + + +class K8SCopy(metaclass=ABCMeta): + + def __init__(self, module, client): + self.client = client + self.module = module + self.api_instance = core_v1_api.CoreV1Api(client.client) + + self.local_path = module.params.get('local_path') + self.name = module.params.get('pod') + self.namespace = module.params.get('namespace') + self.remote_path = module.params.get('remote_path') + self.content = module.params.get('content') + + self.no_preserve = module.params.get('no_preserve') + self.container_arg = {} + if module.params.get('container'): + self.container_arg['container'] = module.params.get('container') + + @abstractmethod + def run(self): + pass + + +class K8SCopyFromPod(K8SCopy): + """ + Copy files/directory from Pod into local filesystem + """ + def __init__(self, module, client): + super(K8SCopyFromPod, self).__init__(module, client) + self.is_remote_path_dir = None + self.files_to_copy = list() + + def list_remote_files(self): + """ + This method will check if the remote path is a dir or file + if it is a directory the file list will be updated accordingly + """ + try: + find_cmd = ['find', self.remote_path, '-type', 'f', '-name', '*'] + response = stream(self.api_instance.connect_get_namespaced_pod_exec, + self.name, + self.namespace, + command=find_cmd, + stdout=True, stderr=True, + stdin=False, tty=False, + _preload_content=False, **self.container_arg) + except Exception as e: + self.module.fail_json(msg="Failed to execute on pod {0}/{1} due to : {2}".format(self.namespace, self.name, to_native(e))) + stderr = [] + while response.is_open(): + response.update(timeout=1) + if response.peek_stdout(): + self.files_to_copy.extend(response.read_stdout().rstrip('\n').split('\n')) + if response.peek_stderr(): + err = response.read_stderr() + if "No such file or directory" in err: + self.module.fail_json(msg="{0} does not exist in remote pod filesystem".format(self.remote_path)) + stderr.append(err) + error = response.read_channel(ERROR_CHANNEL) + response.close() + error = yaml.safe_load(error) + if error['status'] != 'Success': + self.module.fail_json(msg="Failed to execute on Pod due to: {0}".format(error)) + + def read(self): + self.stdout = None + self.stderr = None + + if self.response.is_open(): + if not self.response.sock.connected: + self.response._connected = False + else: + ret, out, err = select((self.response.sock.sock, ), (), (), 0) + if ret: + code, frame = self.response.sock.recv_data_frame(True) + if code == ABNF.OPCODE_CLOSE: + self.response._connected = False + elif code in (ABNF.OPCODE_BINARY, ABNF.OPCODE_TEXT) and len(frame.data) > 1: + channel = frame.data[0] + content = frame.data[1:] + if content: + if channel == STDOUT_CHANNEL: + self.stdout = content + elif channel == STDERR_CHANNEL: + self.stderr = content.decode("utf-8", "replace") + + def copy(self): + is_remote_path_dir = len(self.files_to_copy) > 1 or self.files_to_copy[0] != self.remote_path + relpath_start = self.remote_path + if is_remote_path_dir and os.path.isdir(self.local_path): + relpath_start = os.path.dirname(self.remote_path) + + for remote_file in self.files_to_copy: + dest_file = self.local_path + if is_remote_path_dir: + dest_file = os.path.join(self.local_path, os.path.relpath(remote_file, start=relpath_start)) + # create directory to copy file in + os.makedirs(os.path.dirname(dest_file), exist_ok=True) + + pod_command = ['cat', remote_file] + self.response = stream(self.api_instance.connect_get_namespaced_pod_exec, + self.name, + self.namespace, + command=pod_command, + stderr=True, stdin=True, + stdout=True, tty=False, + _preload_content=False, **self.container_arg) + errors = [] + with open(dest_file, 'wb') as fh: + while self.response._connected: + self.read() + if self.stdout: + fh.write(self.stdout) + if self.stderr: + errors.append(self.stderr) + if errors: + self.module.fail_json(msg="Failed to copy file from Pod: {0}".format(''.join(errors))) + self.module.exit_json(changed=True, result="{0} successfully copied locally into {1}".format(self.remote_path, self.local_path)) + + def run(self): + try: + self.list_remote_files() + if self.files_to_copy == []: + self.module.exit_json(changed=False, warning="No file found from directory '{0}' into remote Pod.".format(self.remote_path)) + self.copy() + except Exception as e: + self.module.fail_json(msg="Failed to copy file/directory from Pod due to: {0}".format(to_native(e))) + + +class K8SCopyToPod(K8SCopy): + """ + Copy files/directory from local filesystem into remote Pod + """ + def __init__(self, module, client): + super(K8SCopyToPod, self).__init__(module, client) + self.files_to_copy = list() + + def run_from_pod(self, command): + response = stream(self.api_instance.connect_get_namespaced_pod_exec, + self.name, + self.namespace, + command=command, + stderr=True, stdin=False, + stdout=True, tty=False, + _preload_content=False, **self.container_arg) + errors = [] + while response.is_open(): + response.update(timeout=1) + if response.peek_stderr(): + errors.append(response.read_stderr()) + response.close() + err = response.read_channel(ERROR_CHANNEL) + err = yaml.safe_load(err) + response.close() + if err['status'] != 'Success': + self.module.fail_json(msg="Failed to run {0} on Pod.".format(command), errors=errors) + + def is_remote_path_dir(self): + pod_command = ['test', '-d', self.remote_path] + response = stream(self.api_instance.connect_get_namespaced_pod_exec, + self.name, + self.namespace, + command=pod_command, + stdout=True, stderr=True, + stdin=False, tty=False, + _preload_content=False, **self.container_arg) + while response.is_open(): + response.update(timeout=1) + err = response.read_channel(ERROR_CHANNEL) + err = yaml.safe_load(err) + response.close() + if err['status'] == 'Success': + return True + return False + + def close_temp_file(self): + if self.named_temp_file: + self.named_temp_file.close() + + def run(self): + try: + # remove trailing slash from destination path + dest_file = self.remote_path.rstrip("/") + src_file = self.local_path + self.named_temp_file = None + if self.content: + self.named_temp_file = NamedTemporaryFile(mode="w") + self.named_temp_file.write(self.content) + self.named_temp_file.flush() + src_file = self.named_temp_file.name + else: + if not os.path.exists(self.local_path): + self.module.fail_json(msg="{0} does not exist in local filesystem".format(self.local_path)) + if not os.access(self.local_path, os.R_OK): + self.module.fail_json(msg="{0} not readable".format(self.local_path)) + + if self.is_remote_path_dir(): + if self.content: + self.module.fail_json(msg="When content is specified, remote path should not be an existing directory") + else: + dest_file = os.path.join(dest_file, os.path.basename(src_file)) + + if self.no_preserve: + tar_command = ['tar', '--no-same-permissions', '--no-same-owner', '-xmf', '-'] + else: + tar_command = ['tar', '-xmf', '-'] + + response = stream(self.api_instance.connect_get_namespaced_pod_exec, + self.name, + self.namespace, + command=tar_command, + stderr=True, stdin=True, + stdout=True, tty=False, + _preload_content=False, **self.container_arg) + with TemporaryFile() as tar_buffer: + with tarfile.open(fileobj=tar_buffer, mode='w') as tar: + tar.add(src_file, dest_file) + tar_buffer.seek(0) + commands = [] + # push command in chunk mode + size = 1024 * 1024 + while True: + data = tar_buffer.read(size) + if not data: + break + commands.append(data) + + stderr, stdout = [], [] + while response.is_open(): + if response.peek_stdout(): + stdout.append(response.read_stdout().rstrip("\n")) + if response.peek_stderr(): + stderr.append(response.read_stderr().rstrip("\n")) + if commands: + cmd = commands.pop(0) + response.write_stdin(cmd) + else: + break + response.close() + if stderr: + self.close_temp_file() + self.module.fail_json(command=tar_command, msg="Failed to copy local file/directory into Pod due to: {0}".format(''.join(stderr))) + self.close_temp_file() + if self.content: + self.module.exit_json(changed=True, result="Content successfully copied into {0} on remote Pod".format(self.remote_path)) + self.module.exit_json(changed=True, result="{0} successfully copied into remote Pod into {1}".format(self.local_path, self.remote_path)) + + except Exception as e: + self.module.fail_json(msg="Failed to copy local file/directory into Pod due to: {0}".format(to_native(e))) + + +def check_pod(k8s_ansible_mixin, module): + resource = k8s_ansible_mixin.find_resource("Pod", None, True) + namespace = module.params.get('namespace') + name = module.params.get('pod') + container = module.params.get('container') + + def _fail(exc): + arg = {} + if hasattr(exc, 'body'): + msg = "Namespace={0} Kind=Pod Name={1}: Failed requested object: {2}".format(namespace, name, exc.body) + else: + msg = to_native(exc) + for attr in ['status', 'reason']: + if hasattr(exc, attr): + arg[attr] = getattr(exc, attr) + module.fail_json(msg=msg, **arg) + + try: + result = resource.get(name=name, namespace=namespace) + containers = [c['name'] for c in result.to_dict()['status']['containerStatuses']] + if container and container not in containers: + module.fail_json(msg="Pod has no container {0}".format(container)) + return containers + except Exception as exc: + _fail(exc) + + +def execute_module(module): + + k8s_ansible_mixin = K8sAnsibleMixin(module, pyyaml_required=False) + k8s_ansible_mixin.check_library_version() + + k8s_ansible_mixin.module = module + k8s_ansible_mixin.argspec = module.argument_spec + k8s_ansible_mixin.params = k8s_ansible_mixin.module.params + k8s_ansible_mixin.fail_json = k8s_ansible_mixin.module.fail_json + k8s_ansible_mixin.fail = k8s_ansible_mixin.module.fail_json + + k8s_ansible_mixin.client = get_api_client(module=module) + containers = check_pod(k8s_ansible_mixin, module) + if len(containers) > 1 and module.params.get('container') is None: + module.fail_json(msg="Pod contains more than 1 container, option 'container' should be set") + + try: + load_class = {'to_pod': K8SCopyToPod, 'from_pod': K8SCopyFromPod} + state = module.params.get('state') + k8s_copy = load_class.get(state)(module, k8s_ansible_mixin.client) + k8s_copy.run() + except Exception as e: + module.fail_json("Failed to copy object due to: {0}".format(to_native(e))) + + +def main(): + argument_spec = copy.deepcopy(AUTH_ARG_SPEC) + argument_spec['namespace'] = {'type': 'str', 'required': True} + argument_spec['pod'] = {'type': 'str', 'required': True} + argument_spec['container'] = {} + argument_spec['remote_path'] = {'type': 'path', 'required': True} + argument_spec['local_path'] = {'type': 'path'} + argument_spec['content'] = {'type': 'str'} + argument_spec['state'] = {'type': 'str', 'default': 'to_pod', 'choices': ['to_pod', 'from_pod']} + argument_spec['no_preserve'] = {'type': 'bool', 'default': False} + + module = AnsibleModule(argument_spec=argument_spec, + mutually_exclusive=[('local_path', 'content')], + required_if=[('state', 'from_pod', ['local_path'])], + required_one_of=[['local_path', 'content']], + supports_check_mode=True) + + execute_module(module) + + +if __name__ == '__main__': + main() diff --git a/tests/sanity/ignore-2.10.txt b/tests/sanity/ignore-2.10.txt index bae688b8..aa5c00b2 100644 --- a/tests/sanity/ignore-2.10.txt +++ b/tests/sanity/ignore-2.10.txt @@ -232,3 +232,5 @@ plugins/modules/k8s_service.py validate-modules:return-syntax-error tests/sanity/refresh_ignore_files shebang!skip tests/unit/module_utils/test_discoverer.py future-import-boilerplate!skip tests/unit/module_utils/test_discoverer.py metaclass-boilerplate!skip +molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang +molecule/default/roles/k8scopy/library/k8s_create_file.py shebang diff --git a/tests/sanity/ignore-2.11.txt b/tests/sanity/ignore-2.11.txt index bae688b8..aa5c00b2 100644 --- a/tests/sanity/ignore-2.11.txt +++ b/tests/sanity/ignore-2.11.txt @@ -232,3 +232,5 @@ plugins/modules/k8s_service.py validate-modules:return-syntax-error tests/sanity/refresh_ignore_files shebang!skip tests/unit/module_utils/test_discoverer.py future-import-boilerplate!skip tests/unit/module_utils/test_discoverer.py metaclass-boilerplate!skip +molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang +molecule/default/roles/k8scopy/library/k8s_create_file.py shebang diff --git a/tests/sanity/ignore-2.12.txt b/tests/sanity/ignore-2.12.txt index 541a59d6..028d7e91 100644 --- a/tests/sanity/ignore-2.12.txt +++ b/tests/sanity/ignore-2.12.txt @@ -45,6 +45,8 @@ plugins/module_utils/client/discovery.py import-3.8!skip plugins/module_utils/client/discovery.py import-3.9!skip plugins/module_utils/client/discovery.py import-3.10!skip plugins/module_utils/client/discovery.py metaclass-boilerplate!skip +molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang +molecule/default/roles/k8scopy/library/k8s_create_file.py shebang plugins/module_utils/client/resource.py import-2.6!skip plugins/module_utils/client/resource.py import-2.7!skip plugins/module_utils/client/resource.py import-3.5!skip diff --git a/tests/sanity/ignore-2.9.txt b/tests/sanity/ignore-2.9.txt index 6481d14b..8def4ec0 100644 --- a/tests/sanity/ignore-2.9.txt +++ b/tests/sanity/ignore-2.9.txt @@ -226,3 +226,5 @@ plugins/modules/k8s_service.py validate-modules:parameter-type-not-in-doc tests/sanity/refresh_ignore_files shebang!skip tests/unit/module_utils/test_discoverer.py future-import-boilerplate!skip tests/unit/module_utils/test_discoverer.py metaclass-boilerplate!skip +molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang +molecule/default/roles/k8scopy/library/k8s_create_file.py shebang