gsutil copy to storage failing

Question

i'm working in a instance at us-central1-a zone and I can't copy a ~200GB file.

i've tried :

• gsutil -m cp -L my.log my.file gs://my-bucket/
• gsutil -m cp -L my.second.log my.file gs://my-bucket2/

And after several "catch ups" I get the following error:

CommandException: Some temporary components were not uploaded successfully. Please retry this upload. CommandException: X files/objects could not be transferred.

Any clues?

Thanks

Answer 1

This is a message you'll see if gsutil's parallel composite uploads feature fails to upload at least one of the pieces of the file.

A couple of questions...

1. Have you already tried performing this upload again, after you saw this message?

2. If this error persists, could you please provide the stack trace from gsutil -d cp...

If you're consistently seeing this error and need an immediate fix (if this is a bug with parallel uploads), you can set parallel_composite_upload_threshold=0 in the GSUtil section your boto config to disable parallel uploads.