SOA Webservices - How to manage individual service development on Git

Question

As part of a larger SOA program, we develop many web services on Oracle Service Bus. Currently these are managed on SVN. SVN allows us to manage individual service's version controlling as it allows us to create tags and branches at individual folder level. The following is the typical structure we follow,

+- Services
   |
   +- Service1
   |   |
   |   +- tags
   |   |
   |   +- branch1
   |   |
   |   +- trunk
   |       |
   |       +- artefacts
   |
   +- Service2
   |   |
   |   +- tags
   |   |
   |   +- branch1
   |   |
   |   +- trunk
   |       |
   |       +- artefacts
   |
   +- service N

Git definitely doesn't allow version controlling at the folder level. We have 150+ webservice, each one of them can have it's own development cycle and can be built into a deployable. A direct mapping of our current strategy to Git would be to have each service as a git repository and also follows git's principle, but that makes us have more than 150 git repositories.

Is this approach right? Is there any better approach? Anyone has similar usecase on development of large number of webservices?

We have also considered creating some kind of a logical grouping and create git repositories for the group, but I guess it is a wrong strategy as that would make my versioning and tagging of individual services go haywire. Is there any other way in git where we can manage this kind of grouping?

Answer 1

Submodules and Repo Management

You can group them with submodules - recursively, if need be. The ProGit book explains this very well. You can manage a very large number of repos very easily with gitolite - right down to what repo a user has access to and what branches within that repo.

Branch per Feature

As for a good alternative for the standard SVN branching strategy, google "Branch per Feature" and you should see my article on this exact subject. The strategy we came up with was in part due to dealing with a high number of repositories.

Because you are using Oracle, I should point out the obvious for everybody's sake: Don't integrate by database. If you are, just stop reading right here. I pray for your soul.

---

Architectural Glue

As per SOA, I would designate one repository that would hold your contracts or versioned messages - versioned meaning you support multiple versions of the same massage (usually 3: current, deprecated and specific exception - all previous should raise unhandled exceptions) so you can roll out new versions of scaled out services and support old and new clients during a 0-down-time deploy when dealing with client-server and publish-subscribe relationships. This repository is a submodule of all repositories that have an end-point for integration.

Hooks

Add a hook to this repository to allow forward only changes that embody OCP (Open Close Principle) on an architectural level. You would simply not allow any updates to published classes. Once it's published, you have to support it. Gitolite makes it very easy to administer hooks remotely. It itself uses them to add it's functionality transparently.

Contract Migration

To add, message version migration - among other things - is made easier with Event Sourcing state. This is a subset of CQRS (Command Query Responsibility Segregation). Even Microsoft is doing it and I was lucky to be involved with that. Related are Context Maps and Anti-Corruption Layers from DDD (Domain Driven Design) when it comes to managing messages that exist outside of a service vs. ones used within one. Further guidelines for this can be found under Ubiquitous Language (also from DDD) and Domain Specific Language (see Martin Fowler's article here).

Implementation

Consider 0MQ (ZeroMQ) for delivery to avoid a potentially large tech footprint of other "Message Bus" ideas. This does put responsibility of persistent queueing on your shoulders - driving out a more robust strategy overall. See previous paragraph for clues how that's accomplished. Consider Service Stack if faced with WCF in Windows environments. The less magic/auto-generated code there is in technology, the fewer issues you will have with difficult conflict resolution scenarios, deployments, testing and more.