I'm trying to retrieve all items from a DynamoDB table that match a FilterExpression
, and although all of the items are scanned and half do match, the expected items aren't returned.
I have the following in an AWS Lambda function running on Node.js 6.10:
var AWS = require("aws-sdk"), documentClient = new AWS.DynamoDB.DocumentClient(); function fetchQuotes(category) { let params = { "TableName": "quotient-quotes", "FilterExpression": "category = :cat", "ExpressionAttributeValues": {":cat": {"S": category}} }; console.log(`params=${JSON.stringify(params)}`); documentClient.scan(params, function(err, data) { if (err) { console.error(JSON.stringify(err)); } else { console.log(JSON.stringify(data)); } }); }
There are 10 items in the table, one of which is:
{ "category": "ChuckNorris", "quote": "Chuck Norris does not sleep. He waits.", "uuid": "844a0af7-71e9-41b0-9ca7-d090bb71fdb8" }
When testing with category "ChuckNorris", the log shows:
params={"TableName":"quotient-quotes","FilterExpression":"category = :cat","ExpressionAttributeValues":{":cat":{"S":"ChuckNorris"}}} {"Items":[],"Count":0,"ScannedCount":10}
The scan
call returns all 10 items when I only specify TableName
:
params={"TableName":"quotient-quotes"} {"Items":[<snip>,{"category":"ChuckNorris","uuid":"844a0af7-71e9-41b0-9ca7-d090bb71fdb8","CamelCase":"thevalue","quote":"Chuck Norris does not sleep. He waits."},<snip>],"Count":10,"ScannedCount":10}
A filter expression determines which items within the Query results should be returned to you. All of the other results are discarded. A filter expression is applied after a Query finishes, but before the results are returned.
Parallel Scan in DynamoDB Scans are generally speaking slow. To make that process faster, you can use a feature called "Parallel Scans" which divide the whole DynamoDB Table into Segments. A separate thread/worker then processes each Segment so N workers can work simultaneously to go through the whole keyspace faster.
You can run this script to load the data into DynamoDB: // Load the AWS SDK for JS var AWS = require("aws-sdk"); var fs = require("fs"); AWS. config. update({region: "us-east-1"}); // ----------------------------------------- // Create the document client interface for DynamoDB var documentClient = new AWS.
For faster response times, design your tables and indexes so that your applications can use Query instead of Scan . (For tables, you can also consider using the GetItem and BatchGetItem APIs.)
You do not need to specify the type ("S"
) in your ExpressionAttributeValues
because you are using the DynamoDB DocumentClient. Per the documentation:
The document client simplifies working with items in Amazon DynamoDB by abstracting away the notion of attribute values. This abstraction annotates native JavaScript types supplied as input parameters, as well as converts annotated response data to native JavaScript types.
It's only when you're using the raw DynamoDB object via new AWS.DynamoDB()
that you need to specify the attribute types (i.e., the simple objects keyed on "S"
, "N"
, and so on).
With DocumentClient
, you should be able to use params like this:
const params = { TableName: 'quotient-quotes', FilterExpression: '#cat = :cat', ExpressionAttributeNames: { '#cat': 'category', }, ExpressionAttributeValues: { ':cat': category, }, };
Note that I also moved the field name into an ExpressionAttributeNames
value just for consistency and safety. It's a good practice because certain field names may break your requests if you do not.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With