AWS BotoCore Error - An AttributeValue may not contain an empty string

Question

I'm trying to fill a DynamoDB database with data from an old event store made out of a PostgreSQL database. After it ran through a good portion of the db entries, this error was thrown when attempting to call the put_item function.

botocore.exceptions.ClientError:-

An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: An AttributeValue may not contain an empty string

I decided to rerun the code and see what was happening by dumping out all of the table attributes right before it was inserted.

I can see the only "empty string" is in the answer_string attribute of a dictionary, called details, see below:-

Importing event type 5 completed by user: 1933
1933 5 {'answer': {'difficulty': 1, 'answer_string': ''}, 'card_id': n 
'13448', 'review_id': '153339', 'time_spent': 2431}
62 153339
2017-01-18 00:46:48.373009+00:00 2017-01-18 00:46:48.364217+00:00

I'm pretty certain this is what's causing the error to be thrown, as none of the other table attributes are incorrect.

My problem is the details dictionary can come from dozens of different locations and each details dictionary can have different attributes - the one with the answer_string attribute is just one of many possible dictionary configurations. I can't possibly check for all possible configurations of a dictionary and verify that they all don't have empty strings.

Is there a way I can do a one time overall check on a dictionary and see if any one part of it is empty?

Answer 1

Or, if you want replace all empty strings with None values:

def removeEmptyString(dic):
    for e in dic:
        if isinstance(dic[e], dict):
            dic[e] = removeEmptyString(dic[e])
        if (isinstance(dic[e], str) and dic[e] == ""):
            dic[e] = None
        if isinstance(dic[e], list):
            for entry in dic[e]:
                removeEmptyString(entry)
    return dic

dictionaryWithEmptyStringsReplacedWithNone = removeEmptyString(dicrionaryWithEmptyStrings)

It is far from perfect but it works.

Answer 2

If you want to get a dictionary just containing all keys with empty values, you can simply apply a dictionary comprehension to the details-dict to get all key-value pairs with empty values. E.g.:

empty_values = {key: value for key, value in details.items() if not value}

If you instead want to filter out the key-value pairs with empty values, so you're left with a dictionary where all keys have values, simply use the same comprehension without the not:

details = {key: value for key, value in details.items() if value}