internet/kubernetes.core
3
0
Fork 0
mirror of https://github.com/ansible-collections/kubernetes.core.git synced 2026-03-26 21:33:02 +00:00
Code Issues Projects Releases Wiki Activity

k8s_cp - a new module for copying files to/from a Pod (#127)

Browse Source 
* k8s_cp module

* add documentation for k8s_cp module

* add doc for the new module

* pods should be running

* support for binary, archive and zip file

* sanity

* Delete file.txt

* remove unused

* set back

* Update collection.txt

* Update test_copy_errors.yml

* Update plugins/modules/k8s_cp.py

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>

* Update k8s_cp.py

* Update k8s_cp.py

* tar binary requirements

* Update common.py

* Update k8s_cp.py

* Update k8s_cp.py

* replace kind with binary file

* Update test_copy_large_file.yml

* Update plugins/action/k8s_info.py

Co-authored-by: Mike Graves <mgraves@redhat.com>

* Update k8s_info.py

* Update k8s_info.py

* Update k8s_cp.py

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Mike Graves <mgraves@redhat.com>
This commit is contained in:
abikouo
2021-07-26 13:21:34 +02:00
committed by GitHub
parent 2f59c3db77
commit c330c7ec65
31 changed files with 2007 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@@ -58,6 +58,7 @@ Name | Description
[kubernetes.core.helm_template](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.helm_template_module.rst)|Render chart templates
[kubernetes.core.k8s](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_module.rst)|Manage Kubernetes (K8s) objects
[kubernetes.core.k8s_cluster_info](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_cluster_info_module.rst)|Describe Kubernetes (K8s) cluster, APIs available and their respective versions
[kubernetes.core.k8s_cp](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_cp_module.rst)|Copy files and directories to and from pod.
[kubernetes.core.k8s_exec](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_exec_module.rst)|Execute command in Pod
[kubernetes.core.k8s_info](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_info_module.rst)|Describe Kubernetes (K8s) objects
[kubernetes.core.k8s_json_patch](https://github.com/ansible-collections/kubernetes.core/blob/main/docs/kubernetes.core.k8s_json_patch_module.rst)|Apply JSON patch operations to existing objects

549
docs/kubernetes.core.k8s_cp_module.rst Normal file
View File

@@ -0,0 +1,549 @@
.. _kubernetes.core.k8s_cp_module:
**********************
kubernetes.core.k8s_cp
**********************
**Copy files and directories to and from pod.**
Version added: 2.1.0
.. contents::
:local:
:depth: 1
Synopsis
--------
- Use the Kubernetes Python client to copy files and directories to and from containers inside a pod.
Requirements
------------
The below requirements are needed on the host that executes this module.
- python >= 3.6
- kubernetes >= 12.0.0
Parameters
----------
.. raw:: html
<table border=0 cellpadding=0 class="documentation-table">
<tr>
<th colspan="2">Parameter</th>
<th>Choices/<font color="blue">Defaults</font></th>
<th width="100%">Comments</th>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>api_key</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>Token used to authenticate with the API. Can also be specified via K8S_AUTH_API_KEY environment variable.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>ca_cert</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">path</span>
</div>
</td>
<td>
</td>
<td>
<div>Path to a CA certificate used to authenticate with the API. The full certificate chain must be provided to avoid certificate validation errors. Can also be specified via K8S_AUTH_SSL_CA_CERT environment variable.</div>
<div style="font-size: small; color: darkgreen"><br/>aliases: ssl_ca_cert</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>client_cert</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">path</span>
</div>
</td>
<td>
</td>
<td>
<div>Path to a certificate used to authenticate with the API. Can also be specified via K8S_AUTH_CERT_FILE environment variable.</div>
<div style="font-size: small; color: darkgreen"><br/>aliases: cert_file</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>client_key</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">path</span>
</div>
</td>
<td>
</td>
<td>
<div>Path to a key file used to authenticate with the API. Can also be specified via K8S_AUTH_KEY_FILE environment variable.</div>
<div style="font-size: small; color: darkgreen"><br/>aliases: key_file</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>container</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>The name of the container in the pod to copy files/directories from/to.</div>
<div>Defaults to the only container if there is only one container in the pod.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>content</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>When used instead of <em>local_path</em>, sets the contents of a local file directly to the specified value.</div>
<div>Works only when <em>remote_path</em> is a file. Creates the file if it does not exist.</div>
<div>For advanced formatting or if the content contains a variable, use the <span class='module'>ansible.builtin.template</span> module.</div>
<div>Mutually exclusive with <em>local_path</em>.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>context</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>The name of a context found in the config file. Can also be specified via K8S_AUTH_CONTEXT environment variable.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>host</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>Provide a URL for accessing the API. Can also be specified via K8S_AUTH_HOST environment variable.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>kubeconfig</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">path</span>
</div>
</td>
<td>
</td>
<td>
<div>Path to an existing Kubernetes config file. If not provided, and no other connection options are provided, the Kubernetes client will attempt to load the default configuration file from <em>~/.kube/config</em>. Can also be specified via K8S_AUTH_KUBECONFIG environment variable.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>local_path</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">path</span>
</div>
</td>
<td>
</td>
<td>
<div>Path of the local file or directory.</div>
<div>Required when <em>state</em> is set to <code>from_pod</code>.</div>
<div>Mutually exclusive with <em>content</em>.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>namespace</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
/ <span style="color: red">required</span>
</div>
</td>
<td>
</td>
<td>
<div>The pod namespace name.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>no_preserve</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
<b>Default:</b><br/><div style="color: blue">"no"</div>
</td>
<td>
<div>The copied file/directory&#x27;s ownership and permissions will not be preserved in the container.</div>
<div>This option is ignored when <em>content</em> is set or when <em>state</em> is set to <code>from_pod</code>.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>password</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>Provide a password for authenticating with the API. Can also be specified via K8S_AUTH_PASSWORD environment variable.</div>
<div>Please read the description of the <code>username</code> option for a discussion of when this option is applicable.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>persist_config</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">boolean</span>
</div>
</td>
<td>
<ul style="margin: 0; padding: 0"><b>Choices:</b>
<li>no</li>
<li>yes</li>
</ul>
</td>
<td>
<div>Whether or not to save the kube config refresh tokens. Can also be specified via K8S_AUTH_PERSIST_CONFIG environment variable.</div>
<div>When the k8s context is using a user credentials with refresh tokens (like oidc or gke/gcloud auth), the token is refreshed by the k8s python client library but not saved by default. So the old refresh token can expire and the next auth might fail. Setting this flag to true will tell the k8s python client to save the new refresh token to the kube config file.</div>
<div>Default to false.</div>
<div>Please note that the current version of the k8s python client library does not support setting this flag to True yet.</div>
<div>The fix for this k8s python library is here: https://github.com/kubernetes-client/python-base/pull/169</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>pod</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
/ <span style="color: red">required</span>
</div>
</td>
<td>
</td>
<td>
<div>The pod name.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>proxy</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>The URL of an HTTP proxy to use for the connection. Can also be specified via K8S_AUTH_PROXY environment variable.</div>
<div>Please note that this module does not pick up typical proxy settings from the environment (e.g. HTTP_PROXY).</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>proxy_headers</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">dictionary</span>
</div>
<div style="font-style: italic; font-size: small; color: darkgreen">added in 2.0.0</div>
</td>
<td>
</td>
<td>
<div>The Header used for the HTTP proxy.</div>
<div>Documentation can be found here <a href='https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html?highlight=proxy_headers#urllib3.util.make_headers'>https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html?highlight=proxy_headers#urllib3.util.make_headers</a>.</div>
</td>
</tr>
<tr>
<td class="elbow-placeholder"></td>
<td colspan="1">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>basic_auth</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>Colon-separated username:password for basic authentication header.</div>
<div>Can also be specified via K8S_AUTH_PROXY_HEADERS_BASIC_AUTH environment.</div>
</td>
</tr>
<tr>
<td class="elbow-placeholder"></td>
<td colspan="1">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>proxy_basic_auth</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>Colon-separated username:password for proxy basic authentication header.</div>
<div>Can also be specified via K8S_AUTH_PROXY_HEADERS_PROXY_BASIC_AUTH environment.</div>
</td>
</tr>
<tr>
<td class="elbow-placeholder"></td>
<td colspan="1">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>user_agent</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>String representing the user-agent you want, such as foo/1.0.</div>
<div>Can also be specified via K8S_AUTH_PROXY_HEADERS_USER_AGENT environment.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>remote_path</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">path</span>
/ <span style="color: red">required</span>
</div>
</td>
<td>
</td>
<td>
<div>Path of the file or directory to copy.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>state</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
<ul style="margin: 0; padding: 0"><b>Choices:</b>
<li><div style="color: blue"><b>to_pod</b>&nbsp;&larr;</div></li>
<li>from_pod</li>
</ul>
</td>
<td>
<div>When set to <code>to_pod</code>, the local <em>local_path</em> file or directory will be copied to <em>remote_path</em> into the pod.</div>
<div>When set to <code>from_pod</code>, the remote file or directory <em>remote_path</em> from pod will be copied locally to <em>local_path</em>.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>username</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>
</td>
<td>
<div>Provide a username for authenticating with the API. Can also be specified via K8S_AUTH_USERNAME environment variable.</div>
<div>Please note that this only works with clusters configured to use HTTP Basic Auth. If your cluster has a different form of authentication (e.g. OAuth2 in OpenShift), this option will not work as expected and you should look into the <span class='module'>community.okd.k8s_auth</span> module, as that might do what you need.</div>
</td>
</tr>
<tr>
<td colspan="2">
<div class="ansibleOptionAnchor" id="parameter-"></div>
<b>validate_certs</b>
<a class="ansibleOptionLink" href="#parameter-" title="Permalink to this option"></a>
<div style="font-size: small">
<span style="color: purple">boolean</span>
</div>
</td>
<td>
<ul style="margin: 0; padding: 0"><b>Choices:</b>
<li>no</li>
<li>yes</li>
</ul>
</td>
<td>
<div>Whether or not to verify the API server&#x27;s SSL certificates. Can also be specified via K8S_AUTH_VERIFY_SSL environment variable.</div>
<div style="font-size: small; color: darkgreen"><br/>aliases: verify_ssl</div>
</td>
</tr>
</table>
<br/>
Notes
-----
.. note::
- To avoid SSL certificate validation errors when ``validate_certs`` is *True*, the full certificate chain for the API server must be provided via ``ca_cert`` or in the kubeconfig file.
Examples
--------
.. code-block:: yaml
# kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar
- name: Copy /tmp/foo local file to /tmp/bar in a remote pod
kubernetes.core.k8s:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/bar
local_path: /tmp/foo
# kubectl cp /tmp/foo_dir some-namespace/some-pod:/tmp/bar_dir
- name: Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod
kubernetes.core.k8s:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/bar_dir
local_path: /tmp/foo_dir
# kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar -c some-container
- name: Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container
kubernetes.core.k8s:
namespace: some-namespace
pod: some-pod
container: some-container
remote_path: /tmp/bar
local_path: /tmp/foo
no_preserve: True
state: to_pod
# kubectl cp some-namespace/some-pod:/tmp/foo /tmp/bar
- name: Copy /tmp/foo from a remote pod to /tmp/bar locally
kubernetes.core.k8s:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/foo
local_path: /tmp/bar
state: from_pod
# copy content into a file in the remote pod
- name: Copy /tmp/foo from a remote pod to /tmp/bar locally
kubernetes.core.k8s:
state: to_pod
namespace: some-namespace
pod: some-pod
remote_path: /tmp/foo.txt
content: "This content will be copied into remote file"
Return Values
-------------
Common return values are documented `here <https://docs.ansible.com/ansible/latest/reference_appendices/common_return_values.html#common-return-values>`_, the following are the fields unique to this module:
.. raw:: html
<table border=0 cellpadding=0 class="documentation-table">
<tr>
<th colspan="1">Key</th>
<th>Returned</th>
<th width="100%">Description</th>
</tr>
<tr>
<td colspan="1">
<div class="ansibleOptionAnchor" id="return-"></div>
<b>result</b>
<a class="ansibleOptionLink" href="#return-" title="Permalink to this return value"></a>
<div style="font-size: small">
<span style="color: purple">string</span>
</div>
</td>
<td>success</td>
<td>
<div>message describing the copy operation successfully done.</div>
<br/>
</td>
</tr>
</table>
<br/><br/>
Status
------
Authors
~~~~~~~
- Aubin Bikouo (@abikouo)

3
meta/runtime.yml
View File

@@ -14,6 +14,7 @@ action_groups:
- k8s_log
- k8s_scale
- k8s_service
- k8s_cp
plugin_routing:
action:
@@ -43,6 +44,8 @@ plugin_routing:
redirect: kubernetes.core.k8s_info
k8s_service:
redirect: kubernetes.core.k8s_info
k8s_cp:
redirect: kubernetes.core.k8s_info
inventory:
openshift:
redirect: community.okd.openshift

5
molecule/default/converge.yml
View File

@@ -177,6 +177,11 @@
tags:
- helm
- role: k8scopy
tags:
- copy
- k8s
post_tasks:
- name: Ensure namespace exists
k8s:

15
molecule/default/roles/k8scopy/defaults/main.yml Normal file
View File

@@ -0,0 +1,15 @@
---
# defaults file for k8copy
copy_namespace: copy
pod_with_one_container:
name: pod-copy-0
container: container-00
pod_with_two_container:
name: pod-copy-1
container:
- container-10
- container-11
kubectl_path: /tmp/kubectl

BIN
molecule/default/roles/k8scopy/files/archive.tar Normal file
View File

Binary file not shown.

1
molecule/default/roles/k8scopy/files/data/ansible/collection.txt Normal file
View File

@@ -0,0 +1 @@
kubernetes.core

1
molecule/default/roles/k8scopy/files/data/ansible/module.txt Normal file
View File

@@ -0,0 +1 @@
k8s_cp

1
molecule/default/roles/k8scopy/files/data/file.txt Normal file
View File

@@ -0,0 +1 @@
This is a simple file used to test k8s_cp module on ansible.

2
molecule/default/roles/k8scopy/files/data/teams/ansible.txt Normal file
View File

@@ -0,0 +1,2 @@
cloud team
content team

BIN
molecule/default/roles/k8scopy/files/hello Executable file
View File

Binary file not shown.

1
molecule/default/roles/k8scopy/files/simple_file.txt Normal file
View File

@@ -0,0 +1 @@
This content will be copied into remote Pod.

BIN
molecule/default/roles/k8scopy/files/simple_zip_file.txt.gz Normal file
View File

Binary file not shown.

91
molecule/default/roles/k8scopy/library/k8s_create_file.py Normal file
View File

@@ -0,0 +1,91 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright: (c) 2021, Aubin Bikouo <@abikouo>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = r'''
module: k8s_diff
short_description: Create large file with a defined size.
author:
- Aubin Bikouo (@abikouo)
description:
- This module is used to validate k8s_cp module.
options:
path:
description:
- The destination path for the file to create.
type: path
required: yes
size:
description:
- The size of the output file in MB.
type: int
default: 400
binary:
description:
- If this flag is set to yes, the generated file content binary data.
type: bool
default: False
'''
EXAMPLES = r'''
- name: create 150MB file
k8s_diff:
path: large_file.txt
size: 150
'''
RETURN = r'''
'''
import os
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils._text import to_native
def execute_module(module):
try:
size = module.params.get('size') * 1024 * 1024
path = module.params.get('path')
write_mode = "w"
if module.params.get('binary'):
content = os.urandom(size)
write_mode = "wb"
else:
content = ""
count = 0
while len(content) < size:
content += f"This file has been generated using ansible: {count}\n"
count += 1
with open(path, write_mode) as f:
f.write(content)
module.exit_json(changed=True, size=len(content))
except Exception as e:
module.fail_json(msg="failed to create file due to: {0}".format(to_native(e)))
def main():
argument_spec = {}
argument_spec['size'] = {'type': 'int', 'default': 400}
argument_spec['path'] = {'type': 'path', 'required': True}
argument_spec['binary'] = {'type': 'bool', 'default': False}
module = AnsibleModule(argument_spec=argument_spec)
execute_module(module)
if __name__ == '__main__':
main()

215
molecule/default/roles/k8scopy/library/kubectl_file_compare.py Normal file
View File

@@ -0,0 +1,215 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright: (c) 2021, Aubin Bikouo <@abikouo>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = r'''
module: kubectl_file_compare
short_description: Compare file and directory using kubectl
author:
- Aubin Bikouo (@abikouo)
description:
- This module is used to validate k8s_cp module.
- Compare the local file/directory with the remote pod version
notes:
- This module authenticates on kubernetes cluster using default kubeconfig only.
options:
namespace:
description:
- The pod namespace name
type: str
required: yes
pod:
description:
- The pod name
type: str
required: yes
container:
description:
- The container to retrieve files from.
type: str
remote_path:
description:
- Path of the file or directory on Pod.
type: path
required: yes
local_path:
description:
- Path of the local file or directory.
type: path
content:
description:
- local content to compare with remote file from pod.
- mutually exclusive with option I(local_path).
type: path
required: yes
args:
description:
- The file is considered to be an executable.
- The tool will be run locally and on pod and compare result from output and stderr.
type: list
kubectl_path:
description:
- Path to the kubectl executable, if not specified it will be download.
type: path
'''
EXAMPLES = r'''
- name: compare local /tmp/foo with /tmp/bar in a remote pod
kubectl_file_compare:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/bar
local_path: /tmp/foo
kubectl_path: /tmp/test/kubectl
- name: Compare executable running help command
kubectl_file_compare:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/test/kubectl
local_path: kubectl
kubectl_path: /tmp/test/kubectl
args:
- "--help"
'''
RETURN = r'''
'''
import os
import filecmp
from tempfile import NamedTemporaryFile, TemporaryDirectory
from ansible.module_utils.basic import AnsibleModule
def kubectl_get_content(module, dest_dir):
kubectl_path = module.params.get('kubectl_path')
if kubectl_path is None:
kubectl_path = module.get_bin_path('kubectl', required=True)
namespace = module.params.get('namespace')
pod = module.params.get('pod')
file = module.params.get('remote_path')
cmd = [
kubectl_path,
'cp',
"{0}/{1}:{2}".format(namespace, pod, file)
]
container = module.params.get('container')
if container:
cmd += ['-c', container]
local_file = os.path.join(dest_dir, os.path.basename(module.params.get('remote_path')))
cmd.append(local_file)
rc, out, err = module.run_command(cmd)
return local_file, err, rc, out
def kubectl_run_from_pod(module):
kubectl_path = module.params.get('kubectl_path')
if kubectl_path is None:
kubectl_path = module.get_bin_path('kubectl', required=True)
cmd = [
kubectl_path,
'exec',
module.params.get('pod'),
'-n',
module.params.get('namespace')
]
container = module.params.get('container')
if container:
cmd += ['-c', container]
cmd += ['--', module.params.get('remote_path')]
cmd += module.params.get('args')
return module.run_command(cmd)
def compare_directories(dir1, dir2):
test = filecmp.dircmp(dir1, dir2)
if any([len(test.left_only) > 0, len(test.right_only) > 0, len(test.funny_files) > 0]):
return False
(t, mismatch, errors) = filecmp.cmpfiles(dir1, dir2, test.common_files, shallow=False)
if len(mismatch) > 0 or len(errors) > 0:
return False
for common_dir in test.common_dirs:
new_dir1 = os.path.join(dir1, common_dir)
new_dir2 = os.path.join(dir2, common_dir)
if not compare_directories(new_dir1, new_dir2):
return False
return True
def execute_module(module):
args = module.params.get('args')
local_path = module.params.get('local_path')
namespace = module.params.get('namespace')
pod = module.params.get('pod')
file = module.params.get('remote_path')
content = module.params.get('content')
if args:
pod_rc, pod_out, pod_err = kubectl_run_from_pod(module)
rc, out, err = module.run_command([module.params.get('local_path')] + args)
if rc == pod_rc and out == pod_out:
module.exit_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.", rc=rc, stderr=err, stdout=out)
result = dict(local=dict(rc=rc, out=out, err=err), remote=dict(rc=pod_rc, out=pod_out, err=pod_err))
module.fail_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.", **result)
else:
with TemporaryDirectory() as tmpdirname:
file_from_pod, err, rc, out = kubectl_get_content(module=module, dest_dir=tmpdirname)
if not os.path.exists(file_from_pod):
module.fail_json(msg="failed to copy content from pod", error=err, output=out)
if content is not None:
with NamedTemporaryFile(mode="w") as tmp_file:
tmp_file.write(content)
tmp_file.flush()
if filecmp.cmp(file_from_pod, tmp_file.name):
module.exit_json(msg=f"defined content and {namespace}/{pod}:{file} are same.")
module.fail_json(msg=f"defined content and {namespace}/{pod}:{file} are same.")
if os.path.isfile(local_path):
if filecmp.cmp(file_from_pod, local_path):
module.exit_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.")
module.fail_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.")
if os.path.isdir(local_path):
if compare_directories(file_from_pod, local_path):
module.exit_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.")
module.fail_json(msg=f"{local_path} and {namespace}/{pod}:{file} are same.")
def main():
argument_spec = {}
argument_spec['namespace'] = {'type': 'str', 'required': True}
argument_spec['pod'] = {'type': 'str', 'required': True}
argument_spec['container'] = {}
argument_spec['remote_path'] = {'type': 'path', 'required': True}
argument_spec['local_path'] = {'type': 'path'}
argument_spec['content'] = {'type': 'str'}
argument_spec['kubectl_path'] = {'type': 'path'}
argument_spec['args'] = {'type': 'list'}
module = AnsibleModule(argument_spec=argument_spec,
mutually_exclusive=[('local_path', 'content')],
required_one_of=[['local_path', 'content']])
execute_module(module)
if __name__ == '__main__':
main()

3
molecule/default/roles/k8scopy/meta/main.yml Normal file
View File

@@ -0,0 +1,3 @@
---
collections:
- kubernetes.core

46
molecule/default/roles/k8scopy/tasks/main.yml Normal file
View File

@@ -0,0 +1,46 @@
---
- block:
- name: Download kubeclt executable used to compare results
get_url:
url: https://dl.k8s.io/release/v1.21.3/bin/linux/amd64/kubectl
dest: "{{ kubectl_path }}"
- name: make kubectl executable
ansible.builtin.file:
path: "{{ kubectl_path }}"
mode: "+x"
# Ensure namespace and create pod to perform tests on
- name: Ensure namespace exists
k8s:
definition:
apiVersion: v1
kind: Namespace
metadata:
name: "{{ copy_namespace }}"
- name: Create Pods
k8s:
namespace: '{{ copy_namespace }}'
wait: yes
template: pods_definition.j2
- include_tasks: test_copy_errors.yml
- include_tasks: test_copy_file.yml
- include_tasks: test_multi_container_pod.yml
- include_tasks: test_copy_directory.yml
- include_tasks: test_copy_large_file.yml
always:
- name: Remove kubectl executable
ansible.builtin.file:
path: "{{ kubectl_path }}"
state: absent
ignore_errors: true
- name: Remove namespace
k8s:
kind: Namespace
name: "{{ copy_namespace }}"
state: absent
ignore_errors: true

85
molecule/default/roles/k8scopy/tasks/test_copy_directory.yml Normal file
View File

@@ -0,0 +1,85 @@
---
- block:
- name: copy directory into remote Pod (create new directory)
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /dest_data
local_path: files/data
state: to_pod
- name: compare directories
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /dest_data
local_path: '{{ role_path }}/files/data'
kubectl_path: "{{ kubectl_path }}"
- name: copy directory into remote Pod (existing directory)
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp
local_path: files/data
state: to_pod
- name: compare directories
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/data
local_path: '{{ role_path }}/files/data'
kubectl_path: "{{ kubectl_path }}"
- name: copy directory from Pod into local filesystem (new directory to create)
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/data
local_path: /tmp/test
state: from_pod
- name: compare directories
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/data
local_path: /tmp/test
kubectl_path: "{{ kubectl_path }}"
- name: copy directory from Pod into local filesystem (existing directory)
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/data
local_path: /tmp
state: from_pod
- name: compare directories
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/data
local_path: /tmp/data
kubectl_path: "{{ kubectl_path }}"
always:
- name: Remove directories created into remote Pod
k8s_exec:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
command: 'rm -rf {{ item }}'
ignore_errors: true
with_items:
- /dest_data
- /tmp/data
- name: Remove local directories
file:
path: '{{ item }}'
state: absent
ignore_errors: true
with_items:
- /tmp/data
- /tmp/test

69
molecule/default/roles/k8scopy/tasks/test_copy_errors.yml Normal file
View File

@@ -0,0 +1,69 @@
---
# copy non-existent local file should fail
- name: copy non-existent file into remote Pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp
local_path: this_file_does_not_exist
state: to_pod
ignore_errors: true
register: copy_non_existent
- name: check that error message is as expected
assert:
that:
- copy_non_existent is failed
- copy_non_existent.msg == "this_file_does_not_exist does not exist in local filesystem"
# copy non-existent pod file should fail
- name: copy of non-existent file from remote pod should fail
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /this_file_does_not_exist
local_path: /tmp
state: from_pod
ignore_errors: true
register: copy_non_existent
- name: check that error message is as expected
assert:
that:
- copy_non_existent is failed
- copy_non_existent.msg == "/this_file_does_not_exist does not exist in remote pod filesystem"
# copy file into multiple container pod without specifying the container should fail
- name: copy file into multiple container pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /tmp
local_path: files/simple_file.txt
state: to_pod
ignore_errors: true
register: copy_multi_container
- name: check that error message is as expected
assert:
that:
- copy_multi_container is failed
- copy_multi_container.msg == "Pod contains more than 1 container, option 'container' should be set"
# copy using non-existent container from pod should failed
- name: copy file into multiple container pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /tmp
local_path: files/simple_file.txt
state: to_pod
container: this_is_a_fake_container
ignore_errors: true
register: copy_fake_container
- name: check that error message is as expected
assert:
that:
- copy_fake_container is failed
- copy_fake_container.msg == "Pod has no container this_is_a_fake_container"

191
molecule/default/roles/k8scopy/tasks/test_copy_file.yml Normal file
View File

@@ -0,0 +1,191 @@
---
- block:
# Text file
- name: copy text file into remote pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp
local_path: files/simple_file.txt
state: to_pod
- name: Compare files
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/simple_file.txt
content: "{{ lookup('file', 'simple_file.txt')}}"
kubectl_path: "{{ kubectl_path }}"
- name: Copy simple text file from Pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/simple_file.txt
local_path: /tmp/copy_from_pod.txt
state: from_pod
- name: Compare files
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/simple_file.txt
local_path: /tmp/copy_from_pod.txt
kubectl_path: "{{ kubectl_path }}"
# Binary file
- name: Generate random content
set_fact:
hello_arg: "{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=16') }}"
- name: Copy executable into Pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/hello.exe
local_path: files/hello
state: to_pod
- name: Compare executable
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/hello.exe
local_path: "{{ role_path }}/files/hello"
kubectl_path: "{{ kubectl_path }}"
args:
- "{{ hello_arg }}"
- name: Copy executable from Pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/hello.exe
local_path: /tmp/hello
state: from_pod
- name: update executable permission
file:
path: /tmp/hello
mode: '0755'
- name: Compare executable
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/hello.exe
local_path: /tmp/hello
kubectl_path: "{{ kubectl_path }}"
args:
- "{{ hello_arg }}"
# zip files
- name: copy zip file into remote pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp
local_path: files/simple_zip_file.txt.gz
state: to_pod
- name: compare zip files
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/simple_zip_file.txt.gz
local_path: '{{ role_path }}/files/simple_zip_file.txt.gz'
kubectl_path: "{{ kubectl_path }}"
- name: copy zip file from pod into local filesystem
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/simple_zip_file.txt.gz
local_path: /tmp/copied_from_pod.txt.gz
state: from_pod
- name: compare zip files
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/simple_zip_file.txt.gz
local_path: /tmp/copied_from_pod.txt.gz
kubectl_path: "{{ kubectl_path }}"
# tar files
- name: copy archive into remote pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp
local_path: files/archive.tar
state: to_pod
- name: compare archive
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/archive.tar
local_path: '{{ role_path }}/files/archive.tar'
kubectl_path: "{{ kubectl_path }}"
- name: copy archive from remote pod into local filesystem
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/archive.tar
local_path: /tmp/local_archive.tar
state: from_pod
- name: compare archive
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /tmp/archive.tar
local_path: /tmp/local_archive.tar
kubectl_path: "{{ kubectl_path }}"
# Copy into Pod using content option
- name: set content to be copied into Pod
set_fact:
pod_content: "{{ lookup('password', '/dev/null chars=ascii_lowercase,digits,punctuation length=128') }}"
- name: copy archive into remote pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /this_content.txt
content: '{{ pod_content }}'
state: to_pod
- name: Assert that content is as expected into Pod
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /this_content.txt
content: '{{ pod_content }}'
kubectl_path: "{{ kubectl_path }}"
always:
- name: Delete file created on Pod
k8s_exec:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
command: 'rm {{ item }}'
ignore_errors: true
with_items:
- /tmp/simple_file.txt
- /tmp/hello.exe
- /tmp/simple_zip_file.txt.gz
- /tmp/archive.tar
- /this_content.txt
- name: Delete file created locally
file:
path: '{{ item }}'
state: absent
with_items:
- /tmp/copy_from_pod.txt
- /tmp/hello
- /tmp/copied_from_pod.txt.gz
- /tmp/local_archive.tar

103
molecule/default/roles/k8scopy/tasks/test_copy_large_file.yml Normal file
View File

@@ -0,0 +1,103 @@
---
- name: test copy of large binary and text files
block:
- set_fact:
test_directory: "/tmp/test_k8scp_large_files"
no_log: true
- name: create temporary directory for local files
ansible.builtin.file:
path: "{{ test_directory }}"
state: directory
- name: create large text file
k8s_create_file:
path: "{{ test_directory }}/large_text_file.txt"
size: 150
- name: create large binary file
k8s_create_file:
path: "{{ test_directory }}/large_bin_file.bin"
size: 200
binary: true
# Copy large text file from/to local filesystem to Pod
- name: copy large file into remote Pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_text_file.txt
local_path: "{{ test_directory }}/large_text_file.txt"
state: to_pod
- name: Compare files
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_text_file.txt
local_path: "{{ test_directory }}/large_text_file.txt"
kubectl_path: "{{ kubectl_path }}"
- name: copy large file from Pod into local filesystem
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_text_file.txt
local_path: "{{ test_directory }}/large_text_file_from_pod.txt"
state: from_pod
- name: Compare files
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_text_file.txt
local_path: "{{ test_directory }}/large_text_file_from_pod.txt"
kubectl_path: "{{ kubectl_path }}"
# Copy large binary file from/to local filesystem to Pod
- name: copy large file into remote Pod
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_bin_file.bin
local_path: "{{ test_directory }}/large_bin_file.bin"
state: to_pod
- name: Compare executable, local vs remote
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_bin_file.bin
local_path: "{{ test_directory }}/large_bin_file.bin"
- name: copy executable from pod into local filesystem
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_bin_file.bin
local_path: "{{ test_directory }}/large_bin_file_from_pod.bin"
state: from_pod
- name: Compare executable, local vs remote
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /large_bin_file.bin
local_path: "{{ test_directory }}/large_bin_file_from_pod.bin"
always:
- name: Delete temporary directory created for the test
ansible.builtin.file:
path: "{{ test_directory }}"
state: absent
ignore_errors: true
- name: Delete file created on Pod
k8s_exec:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
command: 'rm {{ item }}'
ignore_errors: true
with_items:
- /large_text_file.txt
- /large_bin_file.bin

71
molecule/default/roles/k8scopy/tasks/test_multi_container_pod.yml Normal file
View File

@@ -0,0 +1,71 @@
---
- set_fact:
random_content: "{{ lookup('password', '/dev/null chars=ascii_lowercase,digits,punctuation length=128') }}"
- name: Copy content into first pod's container
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /file_from_localhost.txt
content: '{{ random_content }}'
container: '{{ pod_with_two_container.container[0] }}'
state: to_pod
- name: Assert that content has been copied into first container
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /file_from_localhost.txt
container: '{{ pod_with_two_container.container[0] }}'
content: '{{ random_content }}'
kubectl_path: "{{ kubectl_path }}"
- name: Assert that content has not been copied into second container
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /file_from_localhost.txt
container: '{{ pod_with_two_container.container[1] }}'
content: '{{ random_content }}'
kubectl_path: "{{ kubectl_path }}"
register: diff
ignore_errors: true
- name: check that diff failed
assert:
that:
- diff is failed
- name: Copy content into second's pod container
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /file_from_localhost_01.txt
content: '{{ random_content }}-secondpod'
container: '{{ pod_with_two_container.container[1] }}'
state: to_pod
- name: Assert that content has not been copied into first container
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /file_from_localhost_01.txt
container: '{{ pod_with_two_container.container[0] }}'
content: '{{ random_content }}-secondpod'
kubectl_path: "{{ kubectl_path }}"
ignore_errors: true
register: diff_1
- name: check that diff failed
assert:
that:
- diff_1 is failed
- name: Assert that content has been copied into second container
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_two_container.name }}'
remote_path: /file_from_localhost_01.txt
container: '{{ pod_with_two_container.container[1] }}'
content: '{{ random_content }}-secondpod'
kubectl_path: "{{ kubectl_path }}"

33
molecule/default/roles/k8scopy/templates/pods_definition.j2 Normal file
View File

@@ -0,0 +1,33 @@
---
apiVersion: v1
kind: Pod
metadata:
name: '{{ pod_with_one_container.name }}'
spec:
containers:
- name: '{{ pod_with_one_container.container }}'
image: busybox
command:
- /bin/sh
- -c
- while true;do date;sleep 5; done
---
apiVersion: v1
kind: Pod
metadata:
name: '{{ pod_with_two_container.name }}'
spec:
containers:
- name: '{{ pod_with_two_container.container[0] }}'
image: busybox:1.32.0
command:
- /bin/sh
- -c
- while true;do date;sleep 5; done
- name: '{{ pod_with_two_container.container[1] }}'
image: busybox:1.33.0
command:
- /bin/sh
- -c
- while true;do date;sleep 5; done

1
plugins/action/k8s_cp.py Symbolic link
View File

@@ -0,0 +1 @@
k8s_info.py

20
plugins/action/k8s_info.py
View File

@@ -8,6 +8,7 @@ __metaclass__ = type
import copy
import traceback
import os
from contextlib import contextmanager
@@ -179,6 +180,20 @@ class ActionModule(ActionBase):
new_module_args.pop('template')
new_module_args['definition'] = result_template
def get_file_realpath(self, local_path):
# local_path is only supported by k8s_cp module.
if self._task.action not in ('k8s_cp', 'kubernetes.core.k8s_cp', 'community.kubernetes.k8s_cp'):
raise AnsibleActionFail("'local_path' is only supported parameter for 'k8s_cp' module.")
if os.path.exists(local_path):
return local_path
try:
# find in expected paths
return self._find_needle('files', local_path)
except AnsibleError:
raise AnsibleActionFail("%s does not exist in local filesystem" % local_path)
def run(self, tmp=None, task_vars=None):
''' handler for k8s options '''
if task_vars is None:
@@ -238,6 +253,11 @@ class ActionModule(ActionBase):
if template:
self.load_template(template, new_module_args, task_vars)
local_path = self._task.args.get('local_path')
state = self._task.args.get('state', None)
if local_path and state == 'to_pod':
new_module_args['local_path'] = self.get_file_realpath(local_path)
# Execute the k8s_* module.
module_return = self._execute_module(module_name=self._task.action, module_args=new_module_args, task_vars=task_vars)

4
plugins/module_utils/common.py
View File

@@ -208,13 +208,13 @@ get_api_client._pool = {}
class K8sAnsibleMixin(object):
def __init__(self, module, *args, **kwargs):
def __init__(self, module, pyyaml_required=True, *args, **kwargs):
if not HAS_K8S_MODULE_HELPER:
module.fail_json(msg=missing_required_lib('kubernetes'), exception=K8S_IMP_ERR,
error=to_native(k8s_import_exception))
self.kubernetes_version = kubernetes.__version__
if not HAS_YAML:
if pyyaml_required and not HAS_YAML:
module.fail_json(msg=missing_required_lib("PyYAML"), exception=YAML_IMP_ERR)
def find_resource(self, kind, api_version, fail=False):

490
plugins/modules/k8s_cp.py Normal file
View File

@@ -0,0 +1,490 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright: (c) 2021, Aubin Bikouo <@abikouo>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = r'''
module: k8s_cp
short_description: Copy files and directories to and from pod.
version_added: "2.2.0"
author:
- Aubin Bikouo (@abikouo)
description:
- Use the Kubernetes Python client to copy files and directories to and from containers inside a pod.
extends_documentation_fragment:
- kubernetes.core.k8s_auth_options
requirements:
- "python >= 3.6"
- "kubernetes >= 12.0.0"
options:
namespace:
description:
- The pod namespace name.
type: str
required: yes
pod:
description:
- The pod name.
type: str
required: yes
container:
description:
- The name of the container in the pod to copy files/directories from/to.
- Defaults to the only container if there is only one container in the pod.
type: str
remote_path:
description:
- Path of the file or directory to copy.
type: path
required: yes
local_path:
description:
- Path of the local file or directory.
- Required when I(state) is set to C(from_pod).
- Mutually exclusive with I(content).
type: path
content:
description:
- When used instead of I(local_path), sets the contents of a local file directly to the specified value.
- Works only when I(remote_path) is a file. Creates the file if it does not exist.
- For advanced formatting or if the content contains a variable, use the M(ansible.builtin.template) module.
- Mutually exclusive with I(local_path).
type: str
state:
description:
- When set to C(to_pod), the local I(local_path) file or directory will be copied to I(remote_path) into the pod.
- When set to C(from_pod), the remote file or directory I(remote_path) from pod will be copied locally to I(local_path).
type: str
default: to_pod
choices: [ to_pod, from_pod ]
no_preserve:
description:
- The copied file/directory's ownership and permissions will not be preserved in the container.
- This option is ignored when I(content) is set or when I(state) is set to C(from_pod).
type: bool
default: False
notes:
- the tar binary is required on the container when copying from local filesystem to pod.
'''
EXAMPLES = r'''
# kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar
- name: Copy /tmp/foo local file to /tmp/bar in a remote pod
kubernetes.core.k8s_cp:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/bar
local_path: /tmp/foo
# kubectl cp /tmp/foo_dir some-namespace/some-pod:/tmp/bar_dir
- name: Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod
kubernetes.core.k8s_cp:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/bar_dir
local_path: /tmp/foo_dir
# kubectl cp /tmp/foo some-namespace/some-pod:/tmp/bar -c some-container
- name: Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container
kubernetes.core.k8s_cp:
namespace: some-namespace
pod: some-pod
container: some-container
remote_path: /tmp/bar
local_path: /tmp/foo
no_preserve: True
state: to_pod
# kubectl cp some-namespace/some-pod:/tmp/foo /tmp/bar
- name: Copy /tmp/foo from a remote pod to /tmp/bar locally
kubernetes.core.k8s_cp:
namespace: some-namespace
pod: some-pod
remote_path: /tmp/foo
local_path: /tmp/bar
state: from_pod
# copy content into a file in the remote pod
- name: Copy /tmp/foo from a remote pod to /tmp/bar locally
kubernetes.core.k8s_cp:
state: to_pod
namespace: some-namespace
pod: some-pod
remote_path: /tmp/foo.txt
content: "This content will be copied into remote file"
'''
RETURN = r'''
result:
description:
- message describing the copy operation successfully done.
returned: success
type: str
'''
import copy
import os
from tempfile import TemporaryFile, NamedTemporaryFile
from select import select
from abc import ABCMeta, abstractmethod
import tarfile
# from ansible_collections.kubernetes.core.plugins.module_utils.ansiblemodule import AnsibleModule
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils._text import to_native
from ansible_collections.kubernetes.core.plugins.module_utils.common import K8sAnsibleMixin, get_api_client
from ansible_collections.kubernetes.core.plugins.module_utils.args_common import AUTH_ARG_SPEC
try:
from kubernetes.client.api import core_v1_api
from kubernetes.stream import stream
from kubernetes.stream.ws_client import STDOUT_CHANNEL, STDERR_CHANNEL, ERROR_CHANNEL, ABNF
except ImportError:
pass
try:
import yaml
except ImportError:
# ImportError are managed by the common module already.
pass
class K8SCopy(metaclass=ABCMeta):
def __init__(self, module, client):
self.client = client
self.module = module
self.api_instance = core_v1_api.CoreV1Api(client.client)
self.local_path = module.params.get('local_path')
self.name = module.params.get('pod')
self.namespace = module.params.get('namespace')
self.remote_path = module.params.get('remote_path')
self.content = module.params.get('content')
self.no_preserve = module.params.get('no_preserve')
self.container_arg = {}
if module.params.get('container'):
self.container_arg['container'] = module.params.get('container')
@abstractmethod
def run(self):
pass
class K8SCopyFromPod(K8SCopy):
"""
Copy files/directory from Pod into local filesystem
"""
def __init__(self, module, client):
super(K8SCopyFromPod, self).__init__(module, client)
self.is_remote_path_dir = None
self.files_to_copy = list()
def list_remote_files(self):
"""
This method will check if the remote path is a dir or file
if it is a directory the file list will be updated accordingly
"""
try:
find_cmd = ['find', self.remote_path, '-type', 'f', '-name', '*']
response = stream(self.api_instance.connect_get_namespaced_pod_exec,
self.name,
self.namespace,
command=find_cmd,
stdout=True, stderr=True,
stdin=False, tty=False,
_preload_content=False, **self.container_arg)
except Exception as e:
self.module.fail_json(msg="Failed to execute on pod {0}/{1} due to : {2}".format(self.namespace, self.name, to_native(e)))
stderr = []
while response.is_open():
response.update(timeout=1)
if response.peek_stdout():
self.files_to_copy.extend(response.read_stdout().rstrip('\n').split('\n'))
if response.peek_stderr():
err = response.read_stderr()
if "No such file or directory" in err:
self.module.fail_json(msg="{0} does not exist in remote pod filesystem".format(self.remote_path))
stderr.append(err)
error = response.read_channel(ERROR_CHANNEL)
response.close()
error = yaml.safe_load(error)
if error['status'] != 'Success':
self.module.fail_json(msg="Failed to execute on Pod due to: {0}".format(error))
def read(self):
self.stdout = None
self.stderr = None
if self.response.is_open():
if not self.response.sock.connected:
self.response._connected = False
else:
ret, out, err = select((self.response.sock.sock, ), (), (), 0)
if ret:
code, frame = self.response.sock.recv_data_frame(True)
if code == ABNF.OPCODE_CLOSE:
self.response._connected = False
elif code in (ABNF.OPCODE_BINARY, ABNF.OPCODE_TEXT) and len(frame.data) > 1:
channel = frame.data[0]
content = frame.data[1:]
if content:
if channel == STDOUT_CHANNEL:
self.stdout = content
elif channel == STDERR_CHANNEL:
self.stderr = content.decode("utf-8", "replace")
def copy(self):
is_remote_path_dir = len(self.files_to_copy) > 1 or self.files_to_copy[0] != self.remote_path
relpath_start = self.remote_path
if is_remote_path_dir and os.path.isdir(self.local_path):
relpath_start = os.path.dirname(self.remote_path)
for remote_file in self.files_to_copy:
dest_file = self.local_path
if is_remote_path_dir:
dest_file = os.path.join(self.local_path, os.path.relpath(remote_file, start=relpath_start))
# create directory to copy file in
os.makedirs(os.path.dirname(dest_file), exist_ok=True)
pod_command = ['cat', remote_file]
self.response = stream(self.api_instance.connect_get_namespaced_pod_exec,
self.name,
self.namespace,
command=pod_command,
stderr=True, stdin=True,
stdout=True, tty=False,
_preload_content=False, **self.container_arg)
errors = []
with open(dest_file, 'wb') as fh:
while self.response._connected:
self.read()
if self.stdout:
fh.write(self.stdout)
if self.stderr:
errors.append(self.stderr)
if errors:
self.module.fail_json(msg="Failed to copy file from Pod: {0}".format(''.join(errors)))
self.module.exit_json(changed=True, result="{0} successfully copied locally into {1}".format(self.remote_path, self.local_path))
def run(self):
try:
self.list_remote_files()
if self.files_to_copy == []:
self.module.exit_json(changed=False, warning="No file found from directory '{0}' into remote Pod.".format(self.remote_path))
self.copy()
except Exception as e:
self.module.fail_json(msg="Failed to copy file/directory from Pod due to: {0}".format(to_native(e)))
class K8SCopyToPod(K8SCopy):
"""
Copy files/directory from local filesystem into remote Pod
"""
def __init__(self, module, client):
super(K8SCopyToPod, self).__init__(module, client)
self.files_to_copy = list()
def run_from_pod(self, command):
response = stream(self.api_instance.connect_get_namespaced_pod_exec,
self.name,
self.namespace,
command=command,
stderr=True, stdin=False,
stdout=True, tty=False,
_preload_content=False, **self.container_arg)
errors = []
while response.is_open():
response.update(timeout=1)
if response.peek_stderr():
errors.append(response.read_stderr())
response.close()
err = response.read_channel(ERROR_CHANNEL)
err = yaml.safe_load(err)
response.close()
if err['status'] != 'Success':
self.module.fail_json(msg="Failed to run {0} on Pod.".format(command), errors=errors)
def is_remote_path_dir(self):
pod_command = ['test', '-d', self.remote_path]
response = stream(self.api_instance.connect_get_namespaced_pod_exec,
self.name,
self.namespace,
command=pod_command,
stdout=True, stderr=True,
stdin=False, tty=False,
_preload_content=False, **self.container_arg)
while response.is_open():
response.update(timeout=1)
err = response.read_channel(ERROR_CHANNEL)
err = yaml.safe_load(err)
response.close()
if err['status'] == 'Success':
return True
return False
def close_temp_file(self):
if self.named_temp_file:
self.named_temp_file.close()
def run(self):
try:
# remove trailing slash from destination path
dest_file = self.remote_path.rstrip("/")
src_file = self.local_path
self.named_temp_file = None
if self.content:
self.named_temp_file = NamedTemporaryFile(mode="w")
self.named_temp_file.write(self.content)
self.named_temp_file.flush()
src_file = self.named_temp_file.name
else:
if not os.path.exists(self.local_path):
self.module.fail_json(msg="{0} does not exist in local filesystem".format(self.local_path))
if not os.access(self.local_path, os.R_OK):
self.module.fail_json(msg="{0} not readable".format(self.local_path))
if self.is_remote_path_dir():
if self.content:
self.module.fail_json(msg="When content is specified, remote path should not be an existing directory")
else:
dest_file = os.path.join(dest_file, os.path.basename(src_file))
if self.no_preserve:
tar_command = ['tar', '--no-same-permissions', '--no-same-owner', '-xmf', '-']
else:
tar_command = ['tar', '-xmf', '-']
response = stream(self.api_instance.connect_get_namespaced_pod_exec,
self.name,
self.namespace,
command=tar_command,
stderr=True, stdin=True,
stdout=True, tty=False,
_preload_content=False, **self.container_arg)
with TemporaryFile() as tar_buffer:
with tarfile.open(fileobj=tar_buffer, mode='w') as tar:
tar.add(src_file, dest_file)
tar_buffer.seek(0)
commands = []
# push command in chunk mode
size = 1024 * 1024
while True:
data = tar_buffer.read(size)
if not data:
break
commands.append(data)
stderr, stdout = [], []
while response.is_open():
if response.peek_stdout():
stdout.append(response.read_stdout().rstrip("\n"))
if response.peek_stderr():
stderr.append(response.read_stderr().rstrip("\n"))
if commands:
cmd = commands.pop(0)
response.write_stdin(cmd)
else:
break
response.close()
if stderr:
self.close_temp_file()
self.module.fail_json(command=tar_command, msg="Failed to copy local file/directory into Pod due to: {0}".format(''.join(stderr)))
self.close_temp_file()
if self.content:
self.module.exit_json(changed=True, result="Content successfully copied into {0} on remote Pod".format(self.remote_path))
self.module.exit_json(changed=True, result="{0} successfully copied into remote Pod into {1}".format(self.local_path, self.remote_path))
except Exception as e:
self.module.fail_json(msg="Failed to copy local file/directory into Pod due to: {0}".format(to_native(e)))
def check_pod(k8s_ansible_mixin, module):
resource = k8s_ansible_mixin.find_resource("Pod", None, True)
namespace = module.params.get('namespace')
name = module.params.get('pod')
container = module.params.get('container')
def _fail(exc):
arg = {}
if hasattr(exc, 'body'):
msg = "Namespace={0} Kind=Pod Name={1}: Failed requested object: {2}".format(namespace, name, exc.body)
else:
msg = to_native(exc)
for attr in ['status', 'reason']:
if hasattr(exc, attr):
arg[attr] = getattr(exc, attr)
module.fail_json(msg=msg, **arg)
try:
result = resource.get(name=name, namespace=namespace)
containers = [c['name'] for c in result.to_dict()['status']['containerStatuses']]
if container and container not in containers:
module.fail_json(msg="Pod has no container {0}".format(container))
return containers
except Exception as exc:
_fail(exc)
def execute_module(module):
k8s_ansible_mixin = K8sAnsibleMixin(module, pyyaml_required=False)
k8s_ansible_mixin.check_library_version()
k8s_ansible_mixin.module = module
k8s_ansible_mixin.argspec = module.argument_spec
k8s_ansible_mixin.params = k8s_ansible_mixin.module.params
k8s_ansible_mixin.fail_json = k8s_ansible_mixin.module.fail_json
k8s_ansible_mixin.fail = k8s_ansible_mixin.module.fail_json
k8s_ansible_mixin.client = get_api_client(module=module)
containers = check_pod(k8s_ansible_mixin, module)
if len(containers) > 1 and module.params.get('container') is None:
module.fail_json(msg="Pod contains more than 1 container, option 'container' should be set")
try:
load_class = {'to_pod': K8SCopyToPod, 'from_pod': K8SCopyFromPod}
state = module.params.get('state')
k8s_copy = load_class.get(state)(module, k8s_ansible_mixin.client)
k8s_copy.run()
except Exception as e:
module.fail_json("Failed to copy object due to: {0}".format(to_native(e)))
def main():
argument_spec = copy.deepcopy(AUTH_ARG_SPEC)
argument_spec['namespace'] = {'type': 'str', 'required': True}
argument_spec['pod'] = {'type': 'str', 'required': True}
argument_spec['container'] = {}
argument_spec['remote_path'] = {'type': 'path', 'required': True}
argument_spec['local_path'] = {'type': 'path'}
argument_spec['content'] = {'type': 'str'}
argument_spec['state'] = {'type': 'str', 'default': 'to_pod', 'choices': ['to_pod', 'from_pod']}
argument_spec['no_preserve'] = {'type': 'bool', 'default': False}
module = AnsibleModule(argument_spec=argument_spec,
mutually_exclusive=[('local_path', 'content')],
required_if=[('state', 'from_pod', ['local_path'])],
required_one_of=[['local_path', 'content']],
supports_check_mode=True)
execute_module(module)
if __name__ == '__main__':
main()

2
tests/sanity/ignore-2.10.txt
View File

@@ -232,3 +232,5 @@ plugins/modules/k8s_service.py validate-modules:return-syntax-error
tests/sanity/refresh_ignore_files shebang!skip
tests/unit/module_utils/test_discoverer.py future-import-boilerplate!skip
tests/unit/module_utils/test_discoverer.py metaclass-boilerplate!skip
molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang
molecule/default/roles/k8scopy/library/k8s_create_file.py shebang

2
tests/sanity/ignore-2.11.txt
View File

@@ -232,3 +232,5 @@ plugins/modules/k8s_service.py validate-modules:return-syntax-error
tests/sanity/refresh_ignore_files shebang!skip
tests/unit/module_utils/test_discoverer.py future-import-boilerplate!skip
tests/unit/module_utils/test_discoverer.py metaclass-boilerplate!skip
molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang
molecule/default/roles/k8scopy/library/k8s_create_file.py shebang

2
tests/sanity/ignore-2.12.txt
View File

@@ -45,6 +45,8 @@ plugins/module_utils/client/discovery.py import-3.8!skip
plugins/module_utils/client/discovery.py import-3.9!skip
plugins/module_utils/client/discovery.py import-3.10!skip
plugins/module_utils/client/discovery.py metaclass-boilerplate!skip
molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang
molecule/default/roles/k8scopy/library/k8s_create_file.py shebang
plugins/module_utils/client/resource.py import-2.6!skip
plugins/module_utils/client/resource.py import-2.7!skip
plugins/module_utils/client/resource.py import-3.5!skip

2
tests/sanity/ignore-2.9.txt
View File

@@ -226,3 +226,5 @@ plugins/modules/k8s_service.py validate-modules:parameter-type-not-in-doc
tests/sanity/refresh_ignore_files shebang!skip
tests/unit/module_utils/test_discoverer.py future-import-boilerplate!skip
tests/unit/module_utils/test_discoverer.py metaclass-boilerplate!skip
molecule/default/roles/k8scopy/library/kubectl_file_compare.py shebang
molecule/default/roles/k8scopy/library/k8s_create_file.py shebang