Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS S3 CLI CP file and add metadata

Tags:

Trying to copy a local file named test.txt to my s3 bucket and add metadata to the file.

But it always prints error:

argument --metadata-directive: Invalid choice, valid choices are: COPY | REPLACE

Is it possible to do this with the cp command, as I understand the docs it should be possible. AWS CLI CP DOCS

This is the commands I've tried:

aws s3 cp test.txt to s3://a-bucket/test.txt --metadata x-amz-meta-cms-id:34533452  aws s3 cp test.txt to s3://a-bucket/test.txt --metadata-directive COPY --metadata x-amz-meta-cms-id:34533452  aws s3 cp test.txt to s3://a-bucket/test.txt --metadata-directive COPY --metadata '{"x-amz-meta-cms-id":"34533452"}'  aws s3 cp test.txt to s3://a-bucket/test.txt --metadata '{"x-amz-meta-cms-id":"34533452"}' 

aws --version: aws-cli/1.9.7 Python/2.7.10 Darwin/16.1.0 botocore/1.3.7

OS: macOS Sierra version 10.12.1

Edit

Worth mentioning is that uploading a file without the --metadata flag works fine.

Hmm, I've checked the help for my version of cli with aws s3 cp help Turns out it does not list --metadata as an option, as the docs at the given link above does.

If runnig older version of aws cli

Use aws s3api put-object 

How to upload a file to a bucket and add metadata:

aws s3api put-object --bucket a-bucket --key test.txt --body test.txt --metadata '{"x-amz-meta-cms-id":"34533452"}' 

Docs: AWS S3API DOCS

like image 625
Jonathan Andersson Avatar asked Nov 16 '16 08:11

Jonathan Andersson


People also ask

How do I update metadata for an existing Amazon S3 file?

You can set object metadata at the time you upload it. After you upload the object, you cannot modify object metadata. The only way to modify object metadata is to make a copy of the object and set the metadata.

How do I update aws metadata?

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . Navigate to your Amazon S3 bucket or folder, and select the check box to the left of the names of the objects with metadata you want to edit. On the Actions menu, choose Edit actions, and choose Edit metadata.

Can we append data to S3 file?

Unfortunately, you can't. S3 doesn't have an "append" operation. Once an object has been uploaded, there is no way to modify it in place; your only option is to upload a new object to replace it, which doesn't meet your requirements.


Video Answer


1 Answers

Indeed the support for metadata option has been added since 1.9.10

aws s3 Added support for custom metadata in cp, mv, and sync.

so upgrading your aws cli to this version (or even better to latest) - and the metadata value needs to be a map so

aws s3 cp test.txt s3://a-bucket/test.txt --metadata '{"x-amz-meta-cms-id":"34533452"}' 
like image 158
Frederic Henri Avatar answered Jan 23 '23 11:01

Frederic Henri