docker container wont start with Mysql docker image

Question

I use docker container with mysql offical images to create more than 11 database container, (container1 to container11). after setting up, all container running fine until container9. At container10, it only starts up about 1 mins and stop again. using docker logs to check container but I do not see anything. stop container9, and restart container10. It runs fine again. The situation seems only happen when I have 9 mysql container and trying to raise up 10th. If I stop one of them, and raise it up again. Then there is no problem. Is it bugs? or I miss some setting for docker bridge?

root@ec8dcb82f64d:/dev/shm# docker restart f4801b57c4cc
f4801b57c4cc
root@ec8dcb82f64d:/dev/shm# docker ps -a | grep f4801b57c4cc
f4801b57c4cc        mysql/mysql-server:5.7             "/entrypoint.sh my..."   2 weeks ago         Exited (1) 3 seconds ago                                                             db
root@ec8dcb82f64d:/dev/shm# docker logs f4801b57c4cc
Initializing database
Database initialized
MySQL init process in progress...
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.

/entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*


MySQL init process done. Ready for start up.

root@ec8dcb82f64d:/dev/shm#

Answer 1

I think I hit the solution after one week even though I do not really understand what happens. the following is what I tried and so far, I can bring up mysql container up to 20 with no problems.

1: Try to create a dummpy mysql container for testing

$ docker run -e MYSQL_ROOT_PASSWORD=password mysql
Unable to find image 'mysql:latest' locally
latest: Pulling from library/mysql
...
Initializing database
2017-08-09T17:58:30.034595Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2017-08-09T17:58:30.039274Z 0 [Warning] InnoDB: io_setup() failed with EAGAIN. Will make 5 attempts before giving up.
2017-08-09T17:58:30.039294Z 0 [Warning] InnoDB: io_setup() attempt 1.
2017-08-09T17:58:30.539495Z 0 [Warning] InnoDB: io_setup() attempt 2.
2017-08-09T17:58:31.039701Z 0 [Warning] InnoDB: io_setup() attempt 3.
2017-08-09T17:58:31.539902Z 0 [Warning] InnoDB: io_setup() attempt 4.
2017-08-09T17:58:32.040115Z 0 [Warning] InnoDB: io_setup() attempt 5.
2017-08-09T17:58:32.540330Z 0 [ERROR] InnoDB: io_setup() failed with EAGAIN after 5 attempts.
2017-08-09T17:58:32.540378Z 0 [ERROR] InnoDB: Cannot initialize AIO sub-system
2017-08-09T17:58:32.540390Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2017-08-09T17:58:32.540401Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2017-08-09T17:58:32.540408Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2017-08-09T17:58:32.540412Z 0 [ERROR] Failed to initialize plugins.
2017-08-09T17:58:32.540415Z 0 [ERROR] Aborting

and hit the error code for io_setup() failed with EAGAIN

2: Examines the current value of aio-max-nr

$ sysctl fs.aio-max-nr
fs.aio-max-nr = 65536

3: Increase the value of aio-max-nr to 2097152

$ sudo sysctl -w fs.aio-max-nr=2097152

3: Start mysql service

4: Try to create more mysql-containers and bring up original one with no problems