How to solve throughput error for dynamodb?

Question

Traceback (the last output from console):

File "batchpy.py", line 61, in <module> obj.batch_w1() File "batchpy.py", line 49, in batch_w1 batch.put_item(data=item) File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py",   line 1641, in __exit__ self.resend_unprocessed() File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 1718, in resend_unprocessed resp = self.table.connection.batch_write_item(batch_data) File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 420, in batch_write_item body=json.dumps(params)) File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 2842, in make_request retry_handler=self._retry_handler) File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 954, in _mexe status = retry_handler(response, i, next_sleep) File "/usr/local/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 2876, in _retry_handler response.status, response.reason, data) boto.dynamodb2.exceptions.ProvisionedThroughputExceededException:    ProvisionedThroughputExceededException: 400 Bad Request {u'message': u'The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API', u'__type': u'com.amazonaws.dynamodb.v20120810#ProvisionedThroughputExceededException'} 

Answer 1

DynamoDB uses a provisioned throughput model for both reads and writes. That means your application will receive errors if it tries to perform more reads or writes than you have allocated to the table.

AWS has done a number of things to help out with this:

• The AWS SDK clients will automatically retry the request several times before you see a ProvisionedThroughputExceededException
• Up to five minutes of unused capacity can be consumed in bursts to accommodate spikes in requests
• You can increase the provisioned read/writes on a table an unlimited number of times each day (to perform more reads/writes per second)
• You can decrease the provisioned throughput on a table 9 times a day (4 times in the first 4 hours, and 1 each in the rest 4 hours window) (to save money)
• You can use Dynamic DynamoDB, a 3rd party app, to automatically manage scaling up and down the provisioned throughput

Depending on the type of app you are creating there are several things you can do to deal with these errors:

• Batch or long running applications can back-off and retry these requests if they occur to limit the table usage
• When bulk loading or bulk reading from a table you can manually scale up the throughput and scale down after you are done
• For a transactional application you can ensure you have your provisioned throughput above the level required to run the application
• Use Dynamic DynamoDB or script changes to table throughput yourself