Single dockerfiles GitHub repository + Docker Hub automated builds

Question

I would like to create a dockerfiles repository on GitHub to centralize all custom Dockerfiles I've created so far. I also would like to automatize builds of each image on Docker Hub, independently.

There is many repositories like this on GitHub :

• https://github.com/tianon/dockerfiles
• https://github.com/vimagick/dockerfiles

(I would not talk here about the most starred repository : https://github.com/jessfraz/dockerfiles, because as you can see on Docker Hub, these are not automated builds)

Wanted structure of dockerfiles repository

I'd like my dockerfiles GitHub repository have the following structure (branch master), based on GitHub repositories I mentioned above :

dockerfiles (GitHub repo)
    |- docker-image-1
        |- Dockerfile
    |- docker-image-2
        |- Dockerfile
    |- ...

Docker Hub builds automation

Configuration

On Docker Hub, I could create 2 public automated builds : docker-image-1 and docker-image-2, both referring to my dockerfiles GitHub repository, but with different Build Settings (in fact, only Dockerfile Location differs) :

• for docker-image-1 :
  • Type : Branch
  • Name : master
  • Dockerfile Location : /docker-image-1/
  • Docker Tag Name : latest
• for docker-image-2 :
  • Type : Branch
  • Name : master
  • Dockerfile Location : /docker-image-2/
  • Docker Tag Name : latest

Drawbacks

With this configuration, all images will be rebuilt if a push on master branch is made (even if there is only 1 Dockerfile concerned by the push and Dockerfile Locations are different). I've checked, it seems to be configured like this for vimagick's repository. But I don't think it's the right way (if I'm wrong, tell me).

Of course, in Build Settings of each automated build, I could disable option "When active, builds will happen automatically on pushes" to avoid this, but I'll have to trigger each build manually depending on what Dockerfile I've updated (not automatic).

Improve automation?

Solutions I see to improve automation are :

1. separate each Dockerfile in its own GitHub repository. No more dockerfiles repository, so each one have its own workflow
2. use a single GitHub repository dockerfiles, but with different branches (one per Docker image). Each build could then be configured to be triggered only on the appropriate branch

With these solutions, problem is that I lost a global overview of all my Dockerfiles. I prefer the idea of a single repository with a single branch, but I don't want all images to be rebuilt on each push on this single branch.

If I'm choosing solution 1, I don't know if git can help me to keep a single repository dockerfiles referencing all my repositories, like :

 dockerfiles (GitHub repo)
    |- docker-image-1 -> link to https://github.com/norbjd/docker-image-1
        |- Dockerfile
    |- docker-image-2 -> link to https://github.com/norbjd/docker-image-2
        |- Dockerfile

It would then be easier to clone the whole project and do Pull Requests (since some images are related : updating docker-image-1/Dockerfile could lead to update docker-image-2/Dockerfile). I've read about Git submodules and subtrees, but I don't know if these are appropriate here.

Maybe it's possible with a single repository, and it's just about finding the right Docker Hub Build Settings.

What are the best practices?

Answer 1

First of all, there are no best practices but things that work for you. As I see, both decisions 1.) and 2.) about the common repository dockerfiles are quite good in order to apply the Docker Hub automation, but I would recommend to complement them as you mention with git submodules or git subtrees. I would not enter into details about the Docker Hub automation, but on how to keep the common repository dockerfiles:

1.) You can separate the Dockerfile images in its own GitHub repository. Then in your dockerfiles I would recommend to use git submodule:

git submodule add https://github.com/norbjd/docker-image-1
git commit -m "Add docker-image-1 as submodule"

Unfortunately you cannot AFAIK select only one file using submodule, instead you have the link to the origin repo. You can have a look at Linking a single file from another git repository and also git: symlink/reference to a file in an external repository. But be careful with submodules because there are well-known drawbacks, if someone besides you wants to use it. I also put an example here to show you how it looks like.

2.) You can use only one GitHub dockerfiles repository, but then I would use git subtree. You can split your repo using subtree in different branches, keeping several modules organized with a quite clean tree.

git subtree split --prefix=docker-image-1 -b docker-image-1-subtree

Check this post for further information regarding subtrees. I did not have time to play around with git subtree to give you an example, but I guess you can get the point.

So, being purist I would not recommend the use of git submodule and I read a lot of people saying git subtree is awesome, so I would go for option 2.). But I am not a purist at all and separating the repos would ease the Docker Hub automation and also it is a CI principle, so go for the option that best adapts to your needs.