Menu
My docker compose file has two containers and looks like this
version: '3'
services:
dynamodb:
image: amazon/dynamodb-local
ports:
- '8000:8000'
networks:
- testnetwork
audit-server:
image: audit-dynamo
environment:
DYNAMO_URL: 'http://0.0.0.0:8000'
command: node app.js
ports:
- '3000:3000'
depends_on:
- dynamodb
# restart: always
networks:
- testnetwork
networks:
testnetwork:
My goal is to mount local data to some volume. currently losing data on docker-compose down
364
Whichever you choose, once you have set up a volume to the folder where the data is stored in the container, if you do a docker-compose down , and then a docker-compose up , your data will not be erased and it will become persistent.
Docker has an option to allow specific folders in a container to be mapped to the normal filesystem on the host. This allows us to have data in the container without making the data part of the Docker image, and without being bound to AUFS.
Volumes are the best way to persist data in Docker. Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.
Once you stop the local dynamodb docker and restart it, you can still see the table created persists. Alternatively, we can use docker-compose to run the container instead of using the docker build and run command with the existing Dockerfile. Once we have this file, we can run the docker-compose up command.
To persist data, the best option is to mount a volume to this. We can do this by using Dockerfile to create a local data folder in the container and map it to the volume on the local machine. There is already an example available for both Dockerfile.
The cluster is large when the data is large. This makes perfect sense when you’re playing to Spark’s strengths by operating on the data. However, when writing to DynamoDB we only need a few items at a time to batch writes efficiently. Reading a whole Dataset into large-scale distributed memory, for example, is wasteful.
First rule of thumb when trying to write lots of rows into DynamoDB — make sure the data is modeled so that you can batch insert, anything else is painfully slow. Even with Batch write item, latency matters. With my laptop in Singapore, writing to the AWS Singapore region, I sensed that latency was causing issues.
So that image uses by default in-memory dynamodb (what you can find by running docker inspect on that image)
"CMD [\"-jar\" \"DynamoDBLocal.jar\" \"-inMemory\"]"
So if you want to keep your data, you need to do something like this in your docker-compose:
version: '3'
volumes:
dynamodb_data:
services:
dynamodb:
image: amazon/dynamodb-local
command: -jar DynamoDBLocal.jar -sharedDb -dbPath /home/dynamodblocal/data/
volumes:
- dynamodb_data:/home/dynamodblocal/data
ports:
- "8000:8000"
To preserve data across docker installations create volume using docker.
docker volume create --driver local --opt type=none \
--opt device=/var/opt/dynamodb_data --opt o=bind dynamodb_data
use external option:
version: "3"
volumes:
dynamodb_data:
external: true
services:
dynamodb-local:
image: amazon/dynamodb-local
command: ["-jar", "DynamoDBLocal.jar", "-sharedDb", "-dbPath", "/home/dynamodblocal/data"]
volumes:
- dynamodb_data:/home/dynamodblocal/data
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
