Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

DynamoDB: How to store a list of items

I want to store a list of items beloging to some parent object.

Parent Object would be like following:

user_id - hash key
timestamp - range key
attributeA - String
attributeB - Number
listC - List of objects

listC is a list of objects (like JSON), where each object can have a few fields:

attrX - number
attrY - string
attrZ - string

The size of the list can be varied, from few elements to hundreads.

How should I store them?

Due to DynamoDB's limits, I'm afraid that I cannot keep this list as an attribute of parent object. I think to move these lists to another table. However, I'm not sure if should I:

  1. keep each list's item as a separate record,
  2. split list's items to many separate records, where each database record has few items.

Approach (1):

-------------------------------------
| parent_id | attrX | attrY | attrZ |
-------------------------------------
|  178      |  2    | "abc" | "xyz" |
-------------------------------------
|  178      | 2.4   | "klm" | "qwe" |
------------------------------------- 

Approach (2):

------------------------------------------------------------------------  
| parent_id | Chunk |  ListC                                            |
------------------------------------------------------------------------
|  178      |  1    | [{ X: "2", Y: "abc" }, { X: "2.4", Y: "klm" } ]   |
-------------------------------------------------------------------------
|  178      |  2    | [{ X: "2.8", Y: "nop" }, { X: "3.2", Y: "qrs" } ] |
------------------------------------------------------------------------ 

What would you recommend me?

like image 225
nicq Avatar asked Feb 28 '17 09:02

nicq


People also ask

Can I store list in DynamoDB?

DynamoDBMapper can store List<CustomObject> but not Set<CustomObject> #331.

How many items can DynamoDB store?

DynamoDB transactions DynamoDB transactional API operations have the following constraints: A transaction cannot contain more than 25 unique items. A transaction cannot contain more than 4 MB of data.

How do I import a CSV to AWS DynamoDB?

Choose Import CSV file. Select your CSV file and click Open. The data in the CSV file will be appended to your table. If your CSV file contains one or more rows that have the same keys as items already in your table, you will have the option of overwriting the existing items or appending them to the end of the table.


1 Answers

Actually, the approach depends on the Query Access Pattern (QAP).

Approach 1:-

  1. Typical, normalized approach similar to RDBMS design. However, we need to think this on NoSQL perspective. There is no join in DynamoDB. Potentially, you may need to read two tables to get the required data. Please note that cost is calculated based on Read Capacity Units. So, the two different reads will cost you.

  2. This approach may be acceptable if the item size exceeds the DynamoDB item size 400 KB

  3. You can write query expression to filter the data by attributes attrX, attrY and attrZ as they are stored as normal scalar data type attributes

Approach 2:-

  1. Preferred NoSQL approach to keep all the required data in one table. Join or additional read is not required

  2. Need to consider whether the item size can exceed 400 KB

  3. Whether you need to write query to filter the data by attributes attrX, attrY and attrZ. Please note that in this approach the ListC data is stored as List of Map DynamoDB data type. In most scenarios, DynamoDB doesn't have the flexibility to query the complex data structures like this (i.e. Map inside List data type)

List of Objects - means List of Map on DynamoDB database

{ X: "2.8", Y: "nop" } - is the object. This translates to MAP data type on DynamoDB database.

The outside square bracket translates to LIST data type on DynamoDB

like image 95
notionquest Avatar answered Oct 07 '22 11:10

notionquest