Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get item count from DynamoDB?

I want to know item count with DynamoDB querying.

I can querying for DynamoDB, but I only want to know 'total count of item'.

For example, 'SELECT COUNT(*) FROM ... WHERE ...' in MySQL

$result = $aws->query(array(  'TableName' => 'game_table',  'IndexName' => 'week-point-index',  'KeyConditions' => array(     'week' => array(         'ComparisonOperator' => 'EQ',         'AttributeValueList' => array(             array(Type::STRING => $week)         )     ),     'point' => array(         'ComparisonOperator' => 'GE',         'AttributeValueList' => array(             array(Type::NUMBER => $my_point)         )     )  ), )); echo Count($result['Items']); 

this code gets the all users data higher than my point.

If count of $result is 100,000, $result is too much big. And it would exceed the limits of the query size.

I need help.

like image 751
sam lee Avatar asked Dec 05 '14 13:12

sam lee


People also ask

How many items can a DynamoDB query return?

In this case, DynamoDB reads up to six items, and then returns only those that match the filter expression. The final Query result contains six items or fewer, even if more items would have matched the filter expression if DynamoDB had kept reading more items.

How many items can DynamoDB have?

DynamoDB transactions DynamoDB transactional API operations have the following constraints: A transaction cannot contain more than 25 unique items. A transaction cannot contain more than 4 MB of data. No two actions in a transaction can work against the same item in the same table.

What does DynamoDB query return?

DynamoDB calculates the number of read capacity units consumed based on item size, not on the amount of data that is returned to an application. The number of capacity units consumed will be the same whether you request all of the attributes (the default behavior) or just some of them (using a projection expression).

How many attributes can a DynamoDB item have?

There is no limit to the number of attributes but the total item size is limited to 400kb. The maximum item size in DynamoDB is 400 KB, which includes both attribute name binary length (UTF-8 length) and attribute value lengths (again binary length). The attribute name counts towards the size limit.

How do I read an item from DynamoDB?

To read an item from a DynamoDB table, use the GetItem operation. You must provide the name of the table, along with the primary key of the item you want. The following AWS CLI example shows how to read an item from the ProductCatalog table. With GetItem, you must specify the entire primary key, not just part of it.

How do I get a row count in DynamoDB?

There are two ways you can get a row count in DynamoDB. The first is performing a full table scan and counting the rows as you go. For a table of any reasonable size this is generally a horrible idea as it will consume all of your provisioned read throughput.

What is live count in DynamoDB?

This means that dynamodb has to read every single item in the table to calculate the total count. It's best to perform the Live count action at times where your production-critical table is used the least. Performing a scan on a large table might exhaust its read capacity units and throttle user requests.

How do I create a DynamoDB table in AWS?

1) Log into AWS console and go to the DynamoDB console. 2) Click Create table and enter details as below. Leave all other settings as default and click Create. 3) Once table is created you will be brought to Overview tab for your new table. Click Manage DynamoDB stream button on this tab.


2 Answers

With the aws dynamodb cli you can get it via scan as follows:

aws dynamodb scan --table-name <TABLE_NAME> --select "COUNT" 

The response will look similar to this:

{     "Count": 123,     "ScannedCount": 123,     "ConsumedCapacity": null } 

notice that this information is in real time in contrast to the describe-table api

like image 74
Daniel Bubenheim Avatar answered Oct 21 '22 09:10

Daniel Bubenheim


You can use the Select parameter and use COUNT in the request. It "returns the number of matching items, rather than the matching items themselves". Important, as brought up by Saumitra R. Bhave in a comment, "If the size of the Query result set is larger than 1 MB, then ScannedCount and Count will represent only a partial count of the total items. You will need to perform multiple Query operations in order to retrieve all of the results".

I'm Not familiar with PHP but here is how you could use it with Java. And then instead of using Count (which I am guessing is a function in PHP) on the 'Items' you can use the Count value from the response - $result['Count']:

final String week = "whatever"; final Integer myPoint = 1337; Condition weekCondition = new Condition()         .withComparisonOperator(ComparisonOperator.EQ)         .withAttributeValueList(new AttributeValue().withS(week)); Condition myPointCondition = new Condition()         .withComparisonOperator(ComparisonOperator.GE)         .withAttributeValueList(new AttributeValue().withN(myPoint.toString()))  Map<String, Condition> keyConditions = new HashMap<>(); keyConditions.put("week", weekCondition); keyConditions.put("point", myPointCondition);  QueryRequest request = new QueryRequest("game_table"); request.setIndexName("week-point-index"); request.setSelect(Select.COUNT); request.setKeyConditions(keyConditions);  QueryResult result = dynamoDBClient.query(request); Integer count = result.getCount(); 

If you don't need to emulate the WHERE clause, you can use a DescribeTable request and use the resulting item count to get an estimate.

The number of items in the specified table. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value.

Also, an important note from the documentation as noted by Saumitra R. Bhave in the comments on this answer:

If the size of the Query result set is larger than 1 MB, ScannedCount and Count represent only a partial count of the total items. You need to perform multiple Query operations to retrieve all the results (see Paginating Table Query Results).

like image 38
mkobit Avatar answered Oct 21 '22 08:10

mkobit