How to recover a stopped docker container which ran out of space?

Question

I've got a docker container that reached the default 10GB limit, and now I cannot start it anymore. This is the error I receive:

Error response from daemon: Cannot start container dfc: setup mount namespace can't create pivot_root dir , error mkdir /var/lib/docker/devicemapper/mnt/dfc2df0ba006762d44e13a3fe12b113a970ca7d91a530a8af1e82178d22f5608/rootfs/.pivot_root753727542: no space left on device 2015/01/12 15:32:58 Error: failed to start one or more containers

There's a blog post explaining how to increase the size of the container past the 10GB limit, but it seens to assume the container is running. In my case, It's stopped and I can't find the correct device in /dev/mapper.

Is there any way to increase the container size, so I can recover access to my container?

Answer 1

It is not trivial but possible. I did it once on RedHat7. It will probably not work with older versions of Centos/Redhat.

I explained it in this post.

Basically, you perform these steps:

• Save the container as an image, using docker commit
• Save the new image as a tar file on disk, using docker save
• Reconfigure the default space per container, as explained in this post
• In the previous step will loose all local images and containers, but you have saved your image as a tar file on disk.
• Load the image you previousley saved, using docker load
• Run a new container from the just loaded image, using docker run