How to incorporate version control (Git) in a large Lotus Notes project

Question

We are maintaining a large website based on Lotus Notes, running on Domino Server 8.5.3. Recently we have been fed up with the lack of source control in our project, so we thought we would try to Git things up a little. But how to do this right?

For various reasons that I will not get into here, we cannot run our applications locally on the development PCs. We can only run them on the development, staging and production servers. (Local Domino server running on the development PCs is not an option, unfortunately.)

This figure illustrates the current situation:

In production (and staging) there are multiple web sites that all have the same functionality. They are basically the same site with the same design, but with different content. Each site consists of a number of Notes databases (News, Archive, Discussion etc). The design of these databases is the same for all sites, so they all inherit their design from the same set of master templates (NewsTemplate, ArchiveTemplate, DiscussionTemplate etc).

A set of replicas of the master templates resides on the Development server. When we deploy new functionality, the required code changes are added to the templates on the Dev. server (more details on this below). Then these templates are replicated to the Staging server for testing, and finally to the production server. This part works quite well.

The problems may be found in the lower part of the figure. We are currently two developers, and we both work in the same set of dev. databases (News, Archive, Discussion etc) on the dev. server. Naturally we keep tripping over each others' work, which is quite frustrating. When it is time to deploy, the code that is to be deployed is selectively copied/pasted from the set of dev. databases to the dev. templates. This is done manually, and thus the risk of mistakes is high. We cannot simply replace the design of the templates with that of the dev. databases, because the dev. databases at all times contain code that is under development. So we must take great care to only deploy code that is "finished". Also, we have no history of changes. This is a horrible, horrible, horrible (add as many horribles as you see fit) way to do software development, and we know it.

Now, using Git, local replicas, database copies or whatnot, what would be the best way to achieve a more streamlined development and deployment scheme?

Answer 1

This figure illustrates a possible solution to the problem:

As explained here, each developer should have his own COPY (not replica) of the database/template (or in our case set of templates) to develop in. So on each dev. PC there is a set of templates, and each template is synchronized with an on-disk-project. The set of on-disk projects will be under source control with Git.

The developer will not be able to run the application locally, so he needs a set of test databases on the dev. server. There will be one such set for each developer. The test databases will inherit their design from the templates on the dev. PC. Thus, in order to test his code changes, the developer must refresh the design of his designated set of test databases with that of his templates (which reside on his PC).

The workflow for each developer will be as follows:

1. Make code changes in the set of templates on developer's own PC. Test continuously by refreshing the design of the test databases on the dev. server. Sync with on-disk-projects, commit and branch "at will".

2. When developer is ready to push to remote Git repo:

   a) Pull changes from remote Git repo.

   c) Push changes to remote Git repo. Sync databases with on-disk-projects.

   d) If currently on develop branch: Replace design of the set of templates on the dev. server with the design from the set of templates on the dev. PC. This will ensure that the develop branch of the remote Git repo is always consistent with the templates on the dev. server, even though there is no direct connection between them.

3. From here changes can be deployed to staging and production by replicating the templates from the dev. server. Before deploying to production, the develop branch should be merged into the master branch, so that master always reflects the current production version.

If anyone has any comments, additions or objections, I would love to hear them.

Edit: To shorten the feedback loop in point 1 above, development can be done directly in the testing databases on the dev. server. This removes the need to refresh the design of the testing databases every time you need to test your changes, though you must be careful to copy all changes back to the dev. templates at regular intervals to keep your code safe and keep the Git repo updated. This can be done pretty quickly by doing an element-by-element compare of the database and the template. This also serves as an opportunity to review your changes before making a commit, which is a very good habit.

Another option for copying the changes back to the template is to make the testing database on the dev. server a master template and temporarily set the template on the dev. PC to inherit the design from that master template. Then the changes can be copied with "Refresh design" on the dev. PC's template, after which you can remove the inheritance again.