Is it possible to export data from DynamoDB table in some format?
The concrete use case is that I want to export data from my production dynamodb database and import that data into my local dynamodb instance so my application can work with local copy of data instead of production data.
I use link as a local instance of DynamoDB.
To export a DynamoDB table, you use the AWS Data Pipeline console to create a new pipeline. The pipeline launches an Amazon EMR cluster to perform the actual export. Amazon EMR reads the data from DynamoDB, and writes the data to an export file in an Amazon S3 bucket.
This will export all items as jsons documents
aws dynamodb scan --table-name TABLE_NAME > export.json
This script will read from remote dynamodb table and import into the local the full table.
TABLE=YOURTABLE maxItems=25 index=0 DATA=$(aws dynamodb scan --table-name $TABLE --max-items $maxItems) ((index+=1)) echo $DATA | jq ".Items | {\"$TABLE\": [{\"PutRequest\": { \"Item\": .[]}}]}" > inserts.jsons aws dynamodb batch-write-item --request-items file://inserts.jsons --endpoint-url http://localhost:8000 nextToken=$(echo $DATA | jq '.NextToken') while [[ "${nextToken}" != "" ]] do DATA=$(aws dynamodb scan --table-name $TABLE --max-items $maxItems --starting-token $nextToken) ((index+=1)) echo $DATA | jq ".Items | {\"$TABLE\": [{\"PutRequest\": { \"Item\": .[]}}]}" > inserts.jsons aws dynamodb batch-write-item --request-items file://inserts.jsons --endpoint-url http://localhost:8000 nextToken=$(echo $DATA | jq '.NextToken') done
Here are a version of the script using files to keep the exported data on disk.
TABLE=YOURTABLE maxItems=25 index=0 DATA=$(aws dynamodb scan --table-name $TABLE --max-items $maxItems) ((index+=1)) echo $DATA | cat > "$TABLE-$index.json" nextToken=$(echo $DATA | jq '.NextToken') while [[ "${nextToken}" != "" ]] do DATA=$(aws dynamodb scan --table-name $TABLE --max-items $maxItems --starting-token $nextToken) ((index+=1)) echo $DATA | cat > "$TABLE-$index.json" nextToken=$(echo $DATA | jq '.NextToken') done for x in `ls *$TABLE*.json`; do cat $x | jq ".Items | {\"$TABLE\": [{\"PutRequest\": { \"Item\": .[]}}]}" > inserts.jsons aws dynamodb batch-write-item --request-items file://inserts.jsons --endpoint-url http://localhost:8000 done
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With