What is the best solution for handling multiplatform (dev/integ/valid/prod...) development? Delivery process

Question

I'm not so experienced but i worked on some big Java EE projects (using maven2) with very distinct ways to handle the installation / delivery on the different platforms.

1) One of them was to use snapshots for development and then make a maven release, of components and main webapplications. Thus the delivery is:

• war/ear files
• List item
• properties files
• sgdb files
• some others

And teams will use that files to put the new application versions in the different platforms. I think this process is strict and permits you to always keep easily the different configurations passed in production, but it's not really flexible, the process is a bit heavy and it conducted us to sometimes do some dirty things like overriding a class of a war to patch a regression... This is an e-commerce website with 10million unique visitors per month and a 99.89% availability.

2) Another i saw is to checkout the sources on each platform and then install the snapshot artifacts in a local repository. Then the application server will use these snapshots of the .m2 folder. There is not a real delivery process since to put a new version in production, we just have to update the sources of the components / webapps, do some maven clean install and restart the application server. I think it's more flexible but i see some drawbacks and this approach seems dangerous for me. This website has a frontoffice, i don't know the numbers but it's far less than the 1st one. It also has a big backoffice available for most employees of a 130 000 people company.

I guess depending on the website, its exposition to the public and the availability required, we have to adapt the delivery strategy to the needs.

I'm not here to ask which solution is the best but wonder if you have seen different things, and which strategy you would use in which case?

Answer 1

Without dealing dealing web sites, I had to participate in release management process for various big (Java) projects in heterogeneous environment:

• development on "PC", meaning in our case Windows -- sadly still Windows Xp for now -- (and unit testing)
• continuous integration and system testing on linux (because they are cheaper to setup)
• pre-production and production on Solaris (Sun Fire for instance)

The common method I saw was:

• binary dependency (each project uses the binaries produced by the other project, not their sources)
• no recompilation for integration testing (the jars produced on PC are directly used on linux farms)
• full recompilation on pre-production (meaning the binary stored on the Maven repo), at least to make sure that everything is recompiled with the same JDK and the sale options.
• no VCS (Version Control System, like SVN, Perforce, Git, Mercurial, ...) on a production system: everything is deployed from pre-prod through rsynch.

---

So the various parameters to take into account for a release management process is:

• when you develop your project, do you depend directly on the sources or the binaries of the other projects?
• where do you store your setting values?
  Do you parametrize them and, if yes, when do you replace the variables by their final values (only at startup, or also during runtime?)
• do you recompile everything on the final (pre-production) system?
• How do you access/copy/deploy on your production system?
• How do you stop/restart/patch your applications?

(and this is not an exhaustive list.
Depending on the nature of the application release, other concerns will have to be addressed)

Answer 2

The answer to this varies greatly depending on the exact requirements and team structures.

I've implemented processes for a few very large websites with similar availability requirements and there are some general principles I find have worked:

• Externalise any config such that the same built artifact can run on all your environments. Then only build the artifacts once for each release - Rebuilding for different environments is time consuming and risky e.g. it not the same app that you tested
• Centralise the place where the artifacts get built. - e.g. all wars for production must be packaged on the CI server (using the maven release plugin on hudson works well for us).
• All changes for release must be traceable (version control, audit table etc.), to ensure stability and allow for quick rollbacks & diagnostics. This doesn't have to mean a heavyweight process - see the next point
• Automate everything, building, testing, releasing, and rollbacks. If the process is dependable, automatable and quick the the same process can be used for everything from quick fixes to emergency changes. We use the same process for a quick 5 minute emergency fix and for a major release, because it is automated and quick.

Some additional pointers:

See my answer property-placeholder location from another property for a simple way to load different properties per environment with spring.

http://wiki.hudson-ci.org/display/HUDSON/M2+Release+Plugin If you use this plugin and ensure that only only the CI server has the correct credentials to perform maven releases, you can ensure that all releases are performed consistently.

http://decodify.blogspot.com/2010/10/how-to-build-one-click-deployment-job.html A simple way of deploying your releases. Although for large sites you will probably need something more complicated to ensure no downtime - e.g. deploying to half the cluster at a time and flip-flopping web traffic between the two halves - http://martinfowler.com/bliki/BlueGreenDeployment.html

http://continuousdelivery.com/ A good website and book with some very good patterns for releasing.

Hope this helps - good luck.