How can I get the total number of items in a DynamoDB table?

Question

I want to know how many items are in my dynamodb table. From the API guide, one way to do it is using a scan as follows:

<?php $dynamodb = new AmazonDynamoDB();  $scan_response = $dynamodb->scan(array(     'TableName' => 'ProductCatalog'  ));  echo "Total number of items: ".count($scan_response->body->Items)."\n"; 

However, this has to fetch all items and store them in an array in memory which isn't feasible in most cases I would presume. Is there a way to get the total item count more efficiently?

This data is not available in the AWS Dynamo web-console, I have already checked. (at first it looks like it is shown alongside the pagination buttons, but it turns out the figure grows as you go to the next page of items).

Answer 1

I can think of three options to get the total number of items in a DynamoDB table.

1. The first option is using the scan, but the scan function is inefficient and is in general a bad practice, especially for tables with heavy reads or production tables.

2. The second option is what was mention by Atharva:

   A better solution that comes to my mind is to maintain the total number of item counts for such tables in a separate table, where each item will have Table name as it's hash key and total number of items in that table as it's non-key attribute. You can then keep this Table possibly named "TotalNumberOfItemsPerTable" updated by making atomic update operations to increment/decrement the total item count for a particular table.

   The only problem this is that increment operations are not idempotent. So if a write fails or you write more than once this will be reflected in the count. If you need pin-point accuracy, use a conditional update instead.

3. The simplest solution is the DescribeTable which returns ItemCount. The only issue is that the count isn't up to date. The count is updated every 6 hours.

http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html

Answer 2

The Count option is definitely what you want, but you also have to take into account that there may be one or more "page" of results in your Scan result. The Scan operation only scans 1MB of data in your table at a time, so the value of Count in the result is only going to reflect the count of the first 1MB of the table. You will need to make subsequent requests using the value of LastEvaluatedKey in the result (if it is there). Here is some sample code for doing something like that:

<?php  $dynamo_db = new AmazonDynamoDB();  $total = 0; $start_key = null; $params = array(     'TableName' => 'my-table',     'Count'     => true );  do {     if ($start_key) {         $params['ExclusiveStartKey'] = $start_key->getArrayCopy();     }      $response = $dynamo_db->scan($params);      if ($response->isOK()) {         $total += (string) $response->body->Count;          if ($response->body->LastEvaluatedKey) {             $start_key = $response->body->LastEvaluatedKey->to_array();         } else {             $start_key = null;         }     } } while ($start_key);  echo "Count: {$total}";