Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

throttling amazon s3 to limit budget

I am creating a small site and would like to use Amazon's S3 to host and deliver user-uploaded images. However, I'm slightly concerned about bandwidth usage(and the bill) as I've read some horror stories about malicious users sending bills sky-high. Obvious thing to do is preventing hotlinking. But that only turns off people who just want to include the image. There is still the possibility of someone deliberately making large amount of requests. Apart from running your own front-end in EC2(that kinda defeats the purpose of S3?!) and running a monitoring script that shuts down service is there anything I can do?

like image 304
edofic Avatar asked Nov 12 '12 09:11

edofic


People also ask

Does S3 have rate limiting?

Resolution. Amazon S3 supports a request rate of 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per prefix in a bucket. The resources for this request rate aren't automatically assigned when a prefix is created.

How can you maximize the S3 bucket performance?

You can achieve the best performance by issuing multiple concurrent requests to Amazon S3. Spread these requests over separate connections to maximize the accessible bandwidth from Amazon S3. Amazon S3 doesn't have any limits for the number of connections made to your bucket.


1 Answers

One option is to setup CloudWatch Billing Alerts that will send you an email when your usage hits a certain threshold. You will receive an email when your threshold is reached, and then you can investigate the cause and take appropriate action.

While I haven't tried this, you should be able to set up a more automated approach. Those CloudWatch Alerts are posted to an SNS topic called "CloudWatchNotifications". You could add a subscription to this topic that delivers notifications to an HTTP endpoint. This endpoint could automatically restrict access to your S3 bucket until you determine which action to take.

Regarding your concern for a specific malicious user, see this post for a general idea of how IP address restrictions could be put in place once you identify a malicious user.

like image 100
David Marchelya Avatar answered Sep 22 '22 18:09

David Marchelya