Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

boto3 how to create object with metadata?

In the example below I want to set a timestamp metadata attribute when created an S3 object. How do I do that? The documentation is not clear.

import uuuid
import json
import boto3
import botocore
import time

from boto3.session import Session
session = Session(aws_access_key_id='XXX',
                  aws_secret_access_key='XXX')

s3 = session.resource('s3')

bucket = s3.Bucket('blah')

for filename in glob.glob('json/*.json'):
    with open(filename, 'rb') as f:
        data = f.read().decode('utf-8')
        timestamp = str(round(time.time(),10))
        my_s3obj = s3.Object('blah', str(uuid.uuid4())).put(Body=json.dumps(data))
like image 573
Duke Dougal Avatar asked Nov 18 '15 03:11

Duke Dougal


People also ask

What is metadata of an object?

Metadata identifies properties of the object, as well as specifies how the object should be handled when it's accessed. Metadata exists as key:value pairs. For example, the storage class of an object is represented by the metadata entry storageClass:STANDARD .

What is the maximum size of S3 object metadata?

The total volume of data and number of objects you can store are unlimited. Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB. The largest object that can be uploaded in a single PUT is 5 GB.

What is Boto3 resource (' S3 ')?

Python, Boto3, and AWS S3: Demystified At its core, all that Boto3 does is call AWS APIs on your behalf. For the majority of the AWS services, Boto3 offers two distinct ways of accessing these abstracted APIs: Client: low-level service access. Resource: higher-level object-oriented service access.


1 Answers

As for boto3, You have the upload_file possibility detailed in the boto3 website here .

import boto3
#Create the S3 client
s3ressource = client(
    service_name='s3', 
    endpoint_url= param_3,
    aws_access_key_id= param_1,
    aws_secret_access_key=param_2,
    use_ssl=True,
    )

uploading a file, you have to specify the key ( which is basically your robject/file name), and adding metadata when creating the key would be done using the "ExtraArgs" option :

s3ressource.upload_file(Filename, bucketname, key, ExtraArgs={"Metadata": {"metadata1":"ImageName","metadata2":"ImagePROPERTIES" ,"metadata3":"ImageCREATIONDATE"}})
like image 187
MouIdri Avatar answered Sep 28 '22 03:09

MouIdri