AWS Lambda: How to Add Numbers to a NS Set in Dynamodb

Question

The Issue

I have tried several approaches, but haven't been able to find out how to add numbers to a NS set. This is all running inside a lambda function.

What I'm trying to accomplish

I am creating a dynamodb table where different colors in hex align to a set of ids. I am optimizing the table for fast reads and to avoid duplicates which is why I would like to maintain a set of ids for each hex.

How I'm adding items to the table:

let doc = require('dynamodb-doc');
let dynamo = new doc.DynamoDB();

var object = {
    'TableName': 'Hex',
    'Item': {
        'hex': '#FEFEFE',
        'ids': {
            'NS': [2,3,4]
        }
    }
}

dynamo.putItem(object, callback);

Which results in

Then I try to add more ids to the set

Using the Dynamodb Update Item Documentation standards

var params = {
    "TableName" : "Hex",
    "Key": {
      "hex": "#FEFEFE"
    },
    "UpdateExpression" : "ADD #oldIds :newIds",
    "ExpressionAttributeNames" : {
      "#oldIds" : "ids"
    },
    "ExpressionAttributeValues": { 
      ":newIds" : {"NS": ["5", "6"]}
   },
};

dynamo.updateItem(params, callback);

This returns the following error, so dynamo thinks :newIds is a map type instead of a set(?)

"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"

I have also tried these alternative approaches

Try 2:

var setTest = new Set([5, 6]);

var params = {
    "TableName" : "Hex",
    "Key": {
      "hex": "#FEFEFE"
    },
    "UpdateExpression" : "ADD #oldIds :newIds",
    "ExpressionAttributeNames" : {
      "#oldIds" : "ids"
    },
    "ExpressionAttributeValues": { 
      ":newIds" : setTest
   },
};

dynamo.updateItem(params, callback);

Error 2 (same error):

"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"

Try 3:

var params = {
    "TableName" : "Hex",
    "Key": {
      "hex": "#FEFEFE"
    },
    "UpdateExpression" : "ADD #oldIds :newIds",
    "ExpressionAttributeNames" : {
      "#oldIds" : "ids"
    },
    "ExpressionAttributeValues": { 
      ":newIds" : { "NS" : { "L" : [ { "N" : "5" }, { "N" : "6" } ] }}
   },
};

dynamo.updateItem(params, callback);

Error 3 (same error):

"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"

Try 4:

var params = {
    "TableName" : "Hex",
    "Key": {
      "hex": "#FEFEFE"
    },
    "UpdateExpression" : "ADD #oldIds :newIds",
    "ExpressionAttributeNames" : {
      "#oldIds" : "ids"
    },
    "ExpressionAttributeValues": { 
      ":newIds" : [5,6]
   },
};

dynamo.updateItem(params, callback);

Error 4 (similar error, but dynamo thinks I'm adding a list this time)

"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: LIST"

Stack Overflow/Github Questions I've Tried

https://stackoverflow.com/a/37585600/4975772 (I'm adding to a set, not a list)

https://stackoverflow.com/a/37143879/4975772 (I'm using javascript, not python, but I basically need this same thing just different syntax)

https://github.com/awslabs/dynamodb-document-js-sdk/issues/40#issuecomment-123003444 (I need to do this exact thing, but I'm not using the dynamodb-document-js-sdk, I'm using AWS Lambda

Answer 1

How to Create a Set and Add Items to a Set

let AWS = require('aws-sdk');
let docClient = new AWS.DynamoDB.DocumentClient();

...

var params = {
    TableName : 'Hex',
    Key: {'hex': '#FEFEFE'},
    UpdateExpression : 'ADD #oldIds :newIds',
    ExpressionAttributeNames : {
      '#oldIds' : 'ids'
    },
    ExpressionAttributeValues : {
      ':newIds' : docClient.createSet([1,2])
    }
};

docClient.update(params, callback);

Which results in this dynamodb table:

If the set doesn't exist, then that code will create it for you. You can also run that code with a different set to update the set's elements. Super convenient.

Create a Set and Add Items to a Set (OLD API)

let doc = require('dynamodb-doc');
let dynamo = new doc.DynamoDB();

var params = {
    TableName : 'Hex',
    Key: {'hex': '#555555'},
    UpdateExpression : 'ADD #oldIds :newIds',
    ExpressionAttributeNames : {
      '#oldIds' : 'ids'
    },
    ExpressionAttributeValues : {
      ':newIds' : dynamo.Set([2,3], 'N')
    }
};

dynamo.updateItem(params, callback);

(Don't use this code for future development, I only include it to help anyone using the existing DynamoDB Document SDK)

Why the Original Was Failing

Notice how when I asked the question, the resulting set looked like a literal json map object (when viewed in the dynamodb screenshot), which would explain this error message

"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"

So I was using the wrong syntax. The solution is found in the (now depricated) AWS Labs dynamodb-document-js-sdk docs

The full documentation for the newer Document Client can be viewed here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html

Answer 2

I've been struggling with this too. I discovered that there are actually 2 api's for DynamoDB.

AWS.DynamoDB

AWS.DynamoDBDocumentClient

Like you, I was not able to make the ADD function work. I tried to import and use the DynamoDB api instead of the DynamoDBDocumentClient api. It solved the problem for me. Below is my code snippet that works with the DynamoDB api but not with the DynamoDBDocumentClient api. I use a String Set instead of a Number Set, but that won’t make a difference I guess.

var AWS = require('aws-sdk');
var dynamo = new AWS.DynamoDB();
// var dynamo = new AWS.DynamoDB.DocumentClient();

...

var params = {};
params.TableName = “MyTable”;
params.ExpressionAttributeValues = { ':newIds': { "SS": ["someId"] } };
// :newIds represents a new dynamodb set with 1 element
params.UpdateExpression = "ADD someAttribute :newIds";
dynamoClient.updateItem(params, function(err, data) { ….}

Answer 3

This answer might be helpful to those who are using npm dynamoDB module

We had the same issue . AS we were using npm dynamodb module for our queries rather than exporting from AWS.DynamoDB module, given solution of AWS.DynamoDBDocumentClint was implementable untill and unless we could shift from npm dynamoDb module to AWS.DynamoDB queries. So instead of shifting/transforming queries.

We tried to downgrade dynamoDB npm module to version ~1.1.0 and it worked. Previous version was 1.2.X.