Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

DynamoDb - How to do a batch update?

Coming from a relational background, I'm used to being able to write something like:

UPDATE Table Set X = 1 Where Y = 2

However such an operation seems very difficult to accomplish in a db like Dynamodb. Let's say I have already done a query for the items where Y = 2.

The way I see it, with the API provided there are two options:

  • Do lots and lots of individual update requests, OR
  • Do a batch write and write ALL of the data back in, with the update applied.

Both of these methods seem terrible, performance-wise.

Am I missing something obvious here? Or are non relational databases not designed to handle 'updates' on this scale - and if so, can I achieve something similar without drastic performance costs?

like image 235
Daniel Avatar asked Jan 10 '17 04:01

Daniel


1 Answers

No, you are not missing anything obvious. Unfortunately those are the only options you have with DynamoDB, with one caveat being that a BatchWrite is only capable of batch submitting 25 update operations at a time so you'll still have to potentially issue multiple BatchWrite requests.

A BatchWrite is more of a convenience that the DynamoDB API offers to help you save on network traffic, reducing the overhead of 25 requests to the overhead of one, but otherwise it's not much savings.

The BatchWrite API is also a bit less flexible than the individual Update and Put item APIs so in some situations it's better to handle the concurrency yourself and just use the underlying operations.

Of course, the best scenario is if you can architect your solution such that you don't need to perform massive updates to a DynamoDB table. Writes are expensive and if you find yourself frequently having to update a large portion of a table, chances are that there is an alternative design that could be employed.

like image 124
Mike Dinescu Avatar answered Nov 08 '22 15:11

Mike Dinescu