ResumableUploadAbortException on gsutil

Question

I keep seeing the following error when using gsutil

ResumableUploadAbortException: Upload complete with 6275 additional bytes left in stream

The command is very simple, something like

gsutil cp -r <source_path> gs://<target-bucket>/<target_path>

with about 80 files inside <source_path>. There are nested folders inside <source_path>, as well. Changing gsutil cp to gsutil -m cp doesn't make a difference. And this error is reproducible when I run it inside a python script together with lots of a other code. However, when I run the command separately in bash, it doesn't seem to have any problem. So I wonder what could possibly be the reason to ResumableUploadAbortException, please?

Tail of debug output with gsutil -D -m cp

total_bytes_transferred: 794750002
Total bytes copied=794750002, total elapsed time=7.932 secs (95.55 MiBps)
DEBUG: Exception stack trace:
    Traceback (most recent call last):
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 565, in _RunNamedCommandAndHandleExceptions
        parallel_operations, perf_trace_token=perf_trace_token)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 280, in RunNamedCommand
        return_code = command_inst.RunCommand()
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 998, in RunCommand
        self.op_failure_count, plural_str, plural_str))
    CommandException: CommandException: 1 file/object could not be transferred.

Answer 1

Make sure you are transferring a file that gets changed during the transfer process. In my case, the problem is that I redirect the output to a log file, while the log file is also being transferred to the cloud, which causes the above problem.