k8s_cp - wait for remote tar to finish before closing the stream (#1217)

* fix(k8s_cp): wait for remote tar to finish before closing the stream

The module closed the exec connection right after writing the last chunk,
killing tar mid-extract, and never checked its exit status, so a partial
or failed copy was reported as success (#776).

The v4 exec protocol cannot half-close stdin, and busybox tar blocks for
EOF rather than stopping at the end-of-archive marker, so bound the stream
with `head -c N | tar ...`. tar then gets a clean EOF, exits on its own,
and its status is read from the error channel. Containers without /bin/sh
and head keep the previous behaviour and get a warning.

Also add copy_timeout (default 300s, #621) and stream the archive from the
temp file instead of buffering every chunk in memory.

* test(k8s_cp): split combined assert into separate statements

Addresses review feedback on #1217.

* chore(changelog): link the PR and use single-line fragment entries

* fix(linter): reformat with black

* fix(linter): sort imports in test_copy.py

isort --profile black puts ansible_collections in the same third-party
section as pytest, so the blank line between them is a failure.

* fix(k8s_cp): report what tar said when the copy fails early

An early close or a stall is the symptom; tar has usually already
written the cause (a read-only mount, a missing directory) to stderr,
and drain() has it. Discarding it left only the generic truncation or
timeout message.

* refactor(k8s_cp): move the stderr suffix into _fail

The nested fail() helper added a branch inside a closure, which counts
double for nesting and took _stream_tar_to_pod over Sonar's cognitive
complexity limit (14 -> 16). _fail already had the response; give it
the stderr too.

* test(k8s_cp): tempfile for the repeated-copy test directory

Co-authored-by: Bikouo Aubin <79859644+abikouo@users.noreply.github.com>

---------

Co-authored-by: Bikouo Aubin <79859644+abikouo@users.noreply.github.com>
This commit is contained in:
Yuriy Novostavskiy
2026-09-15 09:20:10 +01:00
committed by GitHub
co-authored by Bikouo Aubin
parent 031bc475fa
commit 8a9dd54fe7
8 changed files with 568 additions and 56 deletions
@@ -143,9 +143,7 @@ def compare_directories(dir1, dir2):
[len(test.left_only) > 0, len(test.right_only) > 0, len(test.funny_files) > 0]
):
return False
(t, mismatch, errors) = filecmp.cmpfiles(
dir1, dir2, test.common_files, shallow=False
)
t, mismatch, errors = filecmp.cmpfiles(dir1, dir2, test.common_files, shallow=False)
if len(mismatch) > 0 or len(errors) > 0:
return False
for common_dir in test.common_dirs:
@@ -41,6 +41,7 @@
- include_tasks: test_multi_container_pod.yml
- include_tasks: test_copy_directory.yml
- include_tasks: test_copy_large_file.yml
- include_tasks: test_copy_repeated.yml
- include_tasks: test_copy_item_with_space_in_its_name.yml
- include_tasks: test_init_container_pod.yml
@@ -0,0 +1,46 @@
---
# Regression coverage for
# https://github.com/ansible-collections/kubernetes.core/issues/776
#
# The module used to close the exec connection as soon as the last chunk had
# been written, which killed the remote tar mid-extract and truncated the file.
# The failure was timing dependent, so a single copy rarely caught it: repeat
# the copy and compare every time.
- name: create temporary directory for local files
ansible.builtin.tempfile:
state: directory
suffix: k8scp_repeated
register: test_dir
- name: test repeated copies are never truncated
block:
- set_fact:
test_directory: "{{ test_dir.path }}"
copy_iterations: 10
no_log: true
# Large enough to span several 1MiB write chunks, so the last chunk is not
# also the first and the drain actually has something to wait for.
- name: create a multi-chunk file
ansible.builtin.command:
cmd: dd if=/dev/urandom of={{ test_directory }}/repeated.bin bs=1M count=12
- name: copy the same file repeatedly and verify each time
ansible.builtin.include_tasks: verify_one_copy.yml
loop: "{{ range(0, copy_iterations) | list }}"
loop_control:
loop_var: iteration
always:
- name: delete temporary directory created for the test
ansible.builtin.file:
path: "{{ test_dir.path }}"
state: absent
ignore_errors: true
- name: delete file created on Pod
k8s_exec:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
command: 'rm -f /repeated.bin'
ignore_errors: true
@@ -0,0 +1,23 @@
---
# One iteration of the repeated-copy regression check, see test_copy_repeated.yml
- name: remove any file left by the previous iteration
k8s_exec:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
command: 'rm -f /repeated.bin'
ignore_errors: true
- name: "copy file into remote Pod (iteration {{ iteration }})"
k8s_cp:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /repeated.bin
local_path: "{{ test_directory }}/repeated.bin"
state: to_pod
- name: "compare local and remote file (iteration {{ iteration }})"
kubectl_file_compare:
namespace: '{{ copy_namespace }}'
pod: '{{ pod_with_one_container.name }}'
remote_path: /repeated.bin
local_path: "{{ test_directory }}/repeated.bin"