Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

DynamoDB Scan with FilterExpression in nodejs

Tags:

I'm trying to retrieve all items from a DynamoDB table that match a FilterExpression, and although all of the items are scanned and half do match, the expected items aren't returned.

I have the following in an AWS Lambda function running on Node.js 6.10:

var AWS = require("aws-sdk"),     documentClient = new AWS.DynamoDB.DocumentClient(); function fetchQuotes(category) {     let params = {         "TableName": "quotient-quotes",         "FilterExpression": "category = :cat",         "ExpressionAttributeValues": {":cat": {"S": category}}     };     console.log(`params=${JSON.stringify(params)}`);     documentClient.scan(params, function(err, data) {         if (err) {             console.error(JSON.stringify(err));         } else {             console.log(JSON.stringify(data));         }     }); } 

There are 10 items in the table, one of which is:

{   "category": "ChuckNorris",   "quote": "Chuck Norris does not sleep. He waits.",   "uuid": "844a0af7-71e9-41b0-9ca7-d090bb71fdb8" } 

When testing with category "ChuckNorris", the log shows:

params={"TableName":"quotient-quotes","FilterExpression":"category = :cat","ExpressionAttributeValues":{":cat":{"S":"ChuckNorris"}}} {"Items":[],"Count":0,"ScannedCount":10} 

The scan call returns all 10 items when I only specify TableName:

params={"TableName":"quotient-quotes"} {"Items":[<snip>,{"category":"ChuckNorris","uuid":"844a0af7-71e9-41b0-9ca7-d090bb71fdb8","CamelCase":"thevalue","quote":"Chuck Norris does not sleep. He waits."},<snip>],"Count":10,"ScannedCount":10} 
like image 356
Mike Partridge Avatar asked Mar 15 '18 03:03

Mike Partridge


People also ask

What is Filterexpression in DynamoDB?

A filter expression determines which items within the Query results should be returned to you. All of the other results are discarded. A filter expression is applied after a Query finishes, but before the results are returned.

How can I make DynamoDB Scan faster?

Parallel Scan in DynamoDB Scans are generally speaking slow. To make that process faster, you can use a feature called "Parallel Scans" which divide the whole DynamoDB Table into Segments. A separate thread/worker then processes each Segment so N workers can work simultaneously to go through the whole keyspace faster.

How do I get data from DynamoDB to node js?

You can run this script to load the data into DynamoDB: // Load the AWS SDK for JS var AWS = require("aws-sdk"); var fs = require("fs"); AWS. config. update({region: "us-east-1"}); // ----------------------------------------- // Create the document client interface for DynamoDB var documentClient = new AWS.

Which is faster Scan or query in DynamoDB?

For faster response times, design your tables and indexes so that your applications can use Query instead of Scan . (For tables, you can also consider using the GetItem and BatchGetItem APIs.)


1 Answers

You do not need to specify the type ("S") in your ExpressionAttributeValues because you are using the DynamoDB DocumentClient. Per the documentation:

The document client simplifies working with items in Amazon DynamoDB by abstracting away the notion of attribute values. This abstraction annotates native JavaScript types supplied as input parameters, as well as converts annotated response data to native JavaScript types.

It's only when you're using the raw DynamoDB object via new AWS.DynamoDB() that you need to specify the attribute types (i.e., the simple objects keyed on "S", "N", and so on).

With DocumentClient, you should be able to use params like this:

const params = {     TableName: 'quotient-quotes',     FilterExpression: '#cat = :cat',     ExpressionAttributeNames: {         '#cat': 'category',     },     ExpressionAttributeValues: {         ':cat': category,     }, }; 

Note that I also moved the field name into an ExpressionAttributeNames value just for consistency and safety. It's a good practice because certain field names may break your requests if you do not.

like image 145
Joe Lafiosca Avatar answered Oct 03 '22 02:10

Joe Lafiosca