Storing submodules for micro services, but still using forks

Question

I am stumped here. A lot of this is already in place, its just the wrapping that I cannot figure out.

We have a micro-service architecture, with many separate repositories. We are using Docker, and Docker Compose for building and running the development environment, which works beautifully.

The question I have, is how to package up the main collection of repositories. So if I have a folder structure like:

\
    service1
        .git
        Dockerfile
    service2 
        .git
        Dockerfile
    service3
        .git
        Dockerfile
    docker-compose.yml
    README.md

...Where service1, service2, service3 are each their own git repository.

My first thought was to use git submodules, which would work, however we enforce policies to require developers to fork repositories instead of working off the main repository due to continuous integration constraints and code reviews. I was not overly excited about using git submodules at all, even before I thought of this caveat, so an alternative solution would be much preferred.

At the moment i can only think to write scripts to store a list of repositories; run a query for each to see if the logged-in developer has a fork of each, creating one if not, then pulling into the master folder; and then booting docker-compose. This seems like a horrible solution though, enough so that I may just have to write docs to just tell devs how to manually do this process...

Thoughts??

Thanks for your time :)

Answer 1

I've had similar problem with workflow (with exception that I didn't use forks). Ultimately my requirements came down to 1 thing:

• As a developer, I want to run 1 command project bootstrap to bootstrap my environment

---

I would suggest to do few things to optimize the workflow, which worked incredibly well for me:

• store list of services in JSON file, where each service has "url", "fork_url", "name" and other common attributes required for docker-compose to understand how to handle this service. It seems like every team member on your team will have either fork url or base repo url

• build a command-line app (there are a number of options available - depends on your language of choice, for Ruby it's gem Thor, for Go it's package Cobra, etc). Usually it takes just a few hours to build a simple command-line app structure and first couple commands, but this type of automation will save you hours on a day-by-day basis and you will have a foundation to extend your command-line app as your needs increase and the concept is proven to be viable and useful. You also will have a unified interface to provision your environment across all of team members, which then becomes responsibility of a team to maintain it.

• build a project bootstrap command to provision your environment. For example, it will:
  • go over list of services from your JSON file:
    • do a fork if not exists
    • clone your repo
    • add another remote to your repo, which will represent fork (apart from origin)
    • will generate docker-compose.yml from template in a specified directory