How to scan between date range using Lambda and DynamoDB?

Question

I'm attempting to scan between a date range using a Node Lambda function. I have the data being scanned correctly, but I can't seem to get the date expression to work correctly.

var AWS = require('aws-sdk');
var dynamodb = new AWS.DynamoDB({apiVersion: '2012-08-10'});

exports.handler = function(event, context) {
    var tableName = "MyDDBTable";
    dynamodb.scan({
        TableName : tableName,
        FilterExpression: "start_date < :start_date",
        ExpressionAttributeValues: {
            ":start_date": {
                "S": "2016-12-01"
            }
        }
    }, function(err, data) {
        context.succeed(data);
    });
};

This currently doesn't try to return between a range, it's just looking at a single date right now. I didn't want to add an and to the expression until I knew this was working.

A sample document in my DynamoDB is structured like so:

{
    "end_date": {
        "S": "2016-12-02"
    },
    "name": {
        "S": "Name of document"
    },
    "start_date": {
        "S": "2016-10-10"
    },
    "document_id": {
        "N": "7"
    }
}

The document_id is my primary key. I'm pretty new to this whole Lamdba / DynamoDB combination, so I may have this completely set up wrong, but this is what I've managed to complete through my research.

What I'm ultimately trying to achieve is given a start date and an end date, return all DynamoDB documents that have a date range within that. Any help would be greatly appreciated.

Answer 1

Firstly, the scan operation is correct. The dynamodb.scan should be executed in a loop until LastEvaluatedKey is not available. Please refer this blog.

The lambda is not returning the result because it would have not found the data in the first scan. If you can extend the scan until LastEvaluatedKey is not available, the lambda is likely to return the result.

For Query and Scan operations, DynamoDB calculates the amount of consumed provisioned throughput based on item size, not on the amount of data that is returned to an application.

If you query or scan for specific attributes that match values that amount to more than 1 MB of data, you'll need to perform another Query or Scan request for the next 1 MB of data. To do this, take the LastEvaluatedKey value from the previous request, and use that value as the ExclusiveStartKey in the next request. This approach will let you progressively query or scan for new data in 1 MB increments.

BETWEEN Operator sample:-

FilterExpression: "start_date BETWEEN :date1 and :date2"