Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make 10,000 files in S3 public

I have a folder in a bucket with 10,000 files. There seems to be no way to upload them and make them public straight away. So I uploaded them all, they're private, and I need to make them all public.

I've tried the aws console, it just gives an error (works fine with folders with less files).

I've tried using S3 organizing in Firefox, same thing.

Is there some software or some script I can run to make all these public?

like image 657
PeterV Avatar asked Jun 29 '10 15:06

PeterV


People also ask

Is there a limit to the number of files in S3 bucket?

S3 provides unlimited scalability, and there is no official limit on the amount of data and number of objects you can store in an S3 bucket. The size limit for objects stored in a bucket is 5 TB.

Is there a limit to S3 storage?

Q: How much data can I store in Amazon S3? The total volume of data and number of objects you can store are unlimited. Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB. The largest object that can be uploaded in a single PUT is 5 GB.

How many files can you upload to S3 at once?

There are no limits to the number of files/folders you can place in a bucket. You are however limited to 100 buckets per account. 100 buckets per account.


1 Answers

You can generate a bucket policy (see example below) which gives access to all the files in the bucket. The bucket policy can be added to a bucket through AWS console.

{     "Id": "...",     "Statement": [ {         "Sid": "...",         "Action": [             "s3:GetObject"         ],         "Effect": "Allow",         "Resource": "arn:aws:s3:::bucket/*",         "Principal": {             "AWS": [ "*" ]         }     } ] } 

Also look at following policy generator tool provided by Amazon.

http://awspolicygen.s3.amazonaws.com/policygen.html

like image 113
Rajiv Avatar answered Nov 23 '22 04:11

Rajiv