SVN Web Development Workflow

Question

I've read through many of the questions here on SO relating to this issue, but I can't really find any good advice that fits our situation. I inherited this work flow, and I'm trying to make it better.

Our setup

• PHP code base (Kohana in particular)
• code base powers ~60 sites, each with unique templates (i.e. 60 QA [Quality Assurance] domains too)
• each site has A-records for different assets and resources (i.e. 8 A-records for each domain)
• 3 developers
• 3 designers
• developers have VMware image of production server for local development
• designers do not
• shared physical development server used for QA, a post-commit update keeps this server at the current HEAD of trunk at all times
• production server, updated to a mix and match of different revisions depending on what features are live

Our Work flow

• Developers work locally until a given feature is complete and then commit the whole feature to trunk for internal QA.
• Designers make small changes to CSS/images/templates only. They commit directly to trunk, QA their own changes, and update the corresponding files on the production server, generally speaking, immediately after their QA.
• When features are ready to go live, the production server is manually updated to the proper revision numbers for each file related to the feature. Sometimes this is simple, other times it's quite hairy (lots of svn log calls to look for upstream dependencies).

Our problems

• With three different developers working on different features that need different amounts of QA, we find ourselves running into upstream dependency issues on a regular basis.
• At any given time, we don't have a way to programatically determine which features are on the production server and which are not. svn status -u will show us which files are not up to date, but that's generally not a clear picture of features.

What I know

• Some of our problems could be alleviated by having a production branch. We could at least monitor which features were being added into production and when, although this wouldn't solve upstream dependency issues.
• Feature branches are an option, and we have tried that in the past. Due to the fact that our software requires 60 QA domains per branch, we run into process management issues there. For instance, the creation of 480 (60 domains x 8 records per domain) A-records for each feature branch.
• Developer branches are also an option, but our feature QA times vary. I can't definitely say that a previous feature will be out of QA before I need to commit something else.

Upstream Dependency Example

• Developer A adds a new slideshow feature in both the admin and front-end.
• Developer B adds a new feedback feature in both the admin and front-end.
• Both of these features are intertwined with the Location model/controller logic, so changes are made to those associated files to account for both of the new features.
• The slideshow feature enters QA, but is held up by some developmental oversight or scope creep.
• The feedback feature enters QA and passes without issue.
• We want to follow the timeline and push the feedback feature to production, but we cannot do it directly because both features required changes to the Location model/controller. That is, we can't just do an svn update file1 file2 file3.
• Note: This is a simple example, and it can be worked around by doing some reverse merging. Oftentimes our problems are complex than this.

Multi-site Structure Information

• We have a number of pre-defined structural themes, consisting of views, images, CSS files, JS files, etc.
• Each site is assigned a theme.
• For branding and expandability reasons, each site can override a theme view with a custom view or include additional site-specific CSS/JS files.

I'm sure there are some other people out there that have struggled with similar issues, and I'm hoping to get some insight from you smart people on the Internet. Please feel free to ask questions if something I said seems clear as mud!

Answer 1

Your workflow can be improved, here is a list of my advice:

• Start using SVN branches and release builds/deployments on a regular cycle.
• Figure out how long your deployment cycle should be, leaving time in for QA. So for instance if you are going to release a build every month have 3 weeks of development and 1 week of QA. Freeze development for that build after 3 weeks except for bug fixes.
• Decide on a numbering scheme for your builds that allows room to grow
• Utilize a ticketing system (ActiveCollab, JIRA, Mantis ... etc) and enforce the rule that any commit has to be associated with a ticket. Pick one where you can make your commits show up automatically in the ticket.
• If you are going to start to document your requirements before starting development, don't gather your requirements in the ticketing system (for gods sake)
• Doing this will help you determine what features are in a given build. So make some sort of release notes with a list of the ticket numbers or a general description of the tickets for each build.
• Embed your build numbers in your application so that you can track deployments on any given system.
• Start working with automated testing tools PHPUnit for unit tests and Selenium for Functional tests to speed up QA and increase the quality/stability of the web application by decreasing the likelihood that day to day changes introduce new bugs.
• Hudson and Phing can be lifesavers for automating your builds and automating your Testing.

Answer 2

OK, a couple of thoughts:

For this kind of workflow, you would do MUCH better with an SCM with a stream model, like Accurev or ClearCase. In a stream model, the work flows from each developer's streams to integration streams, then to QA streams, then to release streams (or whatever works best). Only the work ready to advance to the next stage gets moved up, and integration can be done with those bundles of work alone.

As deceze said, you need to break things up more modularly, but you also need to combine it with a localization system where client-specific functionality can override specific parts of the code. One way I've done that in PHP is by implementing a PHP file import wrapper that first looks for a client-specific version of a PHP file, and if it doesn't exist, load the generic one instead.

function importModule($module, $clientId){    if(is_file("${clientRootDir}/${clientId}/${module}.php")) {       import("${clientRootDir}/${clientId}/${module}.php");    } else {       import("${defaultRootDir}/${module}.php");    } } 

Using that technique, you can gracefully overwrite parts of the way the website works for just that client, and the person working on that functionality for that client is working in a completely different file so they don't bump into each other. I'm not sure if you're already doing this and that's what you call your "Location model"

Lastly, with this many different "virtual websites" to test, you would benefit immensely with adding automated unit testing (a la PHPUnit) combined with continuous integration to automatically kick off tests when software works its way up to the integration stage.