Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to calculate Read Capacity Unit and Write Capacity Unit for DynamoDB

How to calculate RCU and WCU with the data given as: reading throughput of 32 GB/s and writing throughput of 16 GB/s.

like image 261
pavikirthi Avatar asked Feb 27 '17 20:02

pavikirthi


People also ask

How does DynamoDB determine read and write capacity?

If you need to write an item that is larger than 1 KB, DynamoDB must consume additional write capacity units. Transactional write requests require 2 write capacity units to perform one write per second for items up to 1 KB. The total number of write capacity units required depends on the item size.

What is a capacity unit DynamoDB?

A write capacity unit represents one write per second, for an item up to 1 KB in size. For example, suppose that you create a table with 10 write capacity units. This allows you to perform 10 writes per second, for items up to 1 KB in size per second. Item sizes for writes are rounded up to the next 1 KB multiple.

Which read/write capacity modes are available for DynamoDB?

There are two selectable capacity modes in DynamoDB: Provisioned and On-Demand. While On-demand is perfect for unpredictable workloads with sudden spikes of traffic, the provisioned mode is cheaper and better suited for workloads with predictable traffic.

What is read request units in DynamoDB?

Read request unit: API calls to read data from your table are billed in read request units. DynamoDB read requests can be either strongly consistent, eventually consistent, or transactional. A strongly consistent read request of up to 4 KB requires one read request unit.


1 Answers

DynamoDB Provisioned Throughput is based upon a certain size of units, and the number of items being written:

In DynamoDB, you specify provisioned throughput requirements in terms of capacity units. Use the following guidelines to determine your provisioned throughput:

  • One read capacity unit represents one strongly consistent read per second, or two eventually consistent reads per second, for items up to 4 KB in size. If you need to read an item that is larger than 4 KB, DynamoDB will need to consume additional read capacity units. The total number of read capacity units required depends on the item size, and whether you want an eventually consistent or strongly consistent read.
  • One write capacity unit represents one write per second for items up to 1 KB in size. If you need to write an item that is larger than 1 KB, DynamoDB will need to consume additional write capacity units. The total number of write capacity units required depends on the item size.

Therefore, when determining your desired capacity, you need to know how many items you wish to read and write per second, and the size of those items.

Rather than seeking a particular GB/s, you should be seeking a given number of items that you wish to read/write per second. That is the functionality that your application would require to meet operational performance.

There are also some DynamoDB limits that would apply, but these can be changed upon request:

  • US East (N. Virginia) Region:
  • Per table – 40,000 read capacity units and 40,000 write capacity units
  • Per account – 80,000 read capacity units and 80,000 write capacity units
  • All Other Regions:
  • Per table – 10,000 read capacity units and 10,000 write capacity units
  • Per account – 20,000 read capacity units and 20,000 write capacity units

At 40,000 read capacity units x 4KB x 2 (eventually consistent) = 320MB/s

If my calculations are correct, your requirements are 100x this amount, so it would appear that DynamoDB is not an appropriate solution for such high throughputs.

Are your speeds correct?

Then comes the question of how you are generating so much data per second. A full-duplex 10GFC fiber runs at 2550MB/s, so you would need multiple fiber connections to transmit such data if it is going into/out of the AWS cloud.

Even 10Gb Ethernet only provides 10Gbit/s, so transferring 32GB would require 28 seconds -- and that's to transmit one second of data!

Bottom line: Your data requirements are super high. Are you sure they are realistic?

like image 103
John Rotenstein Avatar answered Sep 23 '22 09:09

John Rotenstein