What's a practicable way for automated configuration, versioning and deployment of Maven-based java applications? [closed]

Question

we're maintaining a medium sized code base consolidated into a single multi(multi)-module maven project. Overall the whole build has up to ten output artifacts for different system components (web applications (.war), utilities (.jar), etc.).

Our deployment process so far is based on simple bash scripts that build the requested artifacts via maven, tag the scm repository with information regarding the artifacts, the target environment and the current build timestamp and then upload the artifacts to the chosen environments application servers and issue commands to restart the running daemons. Configuration of the built artifacts happens by means of maven profiles and resource filtering. So our builds are specific to the target environment.

This process has served us well but for different reasons I would like to move forward towards a more sophisticated approach. Especially I would like to get rid of the bash scripts.

So what are the best practices regarding configuration, versioning and deployment of Maven-based Java applications?

Should our builds be environment agnostic and the configuration be done via config files on the target systems? If so how would a developer take care that new configuration options are included in the deployed config files on the various application servers?

Should we use Maven-versioning a.k.a. Maven release plugin to tag the various builds?

Is it a good idea to configure a CI server like Jenkins or Teamcity to build and optionally deploy our artifacts for us?

Answer 1

I like to think of there being two problem spaces:

• building artifacts (ideally environment agnostic as that means QA can take a hash of the artifact, run their tests on that artifact and when it comes time to deploy, verify the ash and you know it's been QA'd. If your build produces different artifacts depending on whether for QA's env or the staging env, or the production env, then you have to do more work to ensure the artifact going into production has been tested by QA and staged in staging)

• shipping artifacts into an environment. Where that environment requires configuration of the artifacts, the shipping process should include that configuration, either by putting the appropriate configuration files in the target environment and letting the artifacts pick that up, or by cracking open the artifacts, configuring them, and sealing them back up (but in a repeatable and deterministic fashion)

Maven is designed for the first problem space. "The Maven way" is all about producing environment agnostic build artifacts and publishing them to a binary artifact store. If you look at the Maven lifecycles, you will see that the phases stop after the artifact is deployed to the Maven repository (a binary artifact store). In short, Maven sees its job as done at that point. Additionally, there are life cycle phases for unit testing and integration-testing both of which should be possible with an environment agnostic artifact, but that is not the full set of testing that you require... Rather to complete your testing you will need to actually deploy the built artifacts into a real environment.

Many people try to hijack Maven to move beyond its goal (myself included). For example you have the cargo-maven-plugin and the ship-maven-plugin which touch on aspects beyond the maven end game (ie after the artifact gets to the maven repository). Of these, I feel personally, that the ship-maven-plugin (which i wrote, hency my previous "myself included") is closest to use "after maven" because by default it is designed to operate, not on the -SNAPSHOT version of the project that you have checked out on disk, but rather on a release version of the same project that it pulls from the remote repository, eg

mvn ship:ship -DshipVersion=2.5.1

IMO, cargo is aimed at use around the integration-test phase in the life cycle, but again, you can hijack it for other purposes.

If you are producing shrink-wrapped software, ie the kind that a user buys, and installs on their system, the installer program itself is designed to configure the application for the end users environment. It is fine to have the Maven build produce the installer because the actual installer is (at least somewhat) environment agnostic. Ok it may be a Microsoft Windows only installer, or a Linux only installer, but it does not care about which users machine it gets installed on.

Now days, though, we tend to concentrate more on Software as a Service, so we are deploying the software onto servers that we control. It becomes more tempting tow to to the "Maven dark side" where build profiles are used to tweak the internal configuration of the build artifacts (after all we only have three environments we deploy to) and we are moving fast so don't want to take the time to make the application pick up the environment specific configuration from external to the built artifact (sound familiar?). The reason I call this the dark side is that you really are fighting the way maven wants to work... You are always wondering if the jar in the local repository was built with a different profile active, so you end up having to do full clean builds all the time. When it comes time to move from QA to staging or from staging to production, you need to do a full build of the software... And all the unit and integration tests end up being run again (or you end up skipping them and in turn skipping the sanity they may be providing on the artifacts they are building) so in effect you are making life harder and more complex... Just for the sake of putting a few profiles into the maven pom.xml... Just think, if you had followed the maven way you'd just take the artifact from the repository and move that along the different environments, unmodified, unmolested, and with MD5, SHA1 (and hopefully GPG) signatures to prove that it is the same artifact.

So, you ask, how do we code the shipping to production...

Well there are a number of ways to tackle this problem. All of them share a similar set of core principles, namely

• keep the recipe for shipping to an environment in a source control system

• the recipe should ideally have two parts, an environment agnostic part, and the environment specific part.

You can use good old bash scripts, or you can use more "modern" tools such as chef and puppet which are designed for this second problem space.

Recommendations

You should use the right tool for the right job.

If it were me, here's is what I would do:

• Cut releases with the Maven Release Plugin

• The built artifacts should always be environment agnostic.

• The built artifacts should contain "sensible defaults" for all configuration options. In other words, they should either blow up fast if a required configuration option with no sensible default is missing, or they should perform in a sensible way if an optional option is unspecified. An example of a required configuration option might be the database connection details (unless the app is happy to run with an in memory DB)

• Pick a side in the chef vs puppet war (doesn't matter which side, and you can change sides if you want. If you have and ANT mindset, chef may suit you better, if you like dependency management magic, puppet may suit you better)

• Developers should have a say in defining the chef/puppet scripts for deployment, at least the environment agnostic part of those scripts.

• Operations should define the production environment specific details of the chef/puppet deployment

• Keep all those scripts in SCM.

• Use Jenkins, or any CI, to automate as much of the steps as possible. The promoted builds plugin for Jenkins is your friend.

• Your end game is that every commit, providing that it passes all required tests, *could * get deployed into production automatically (or perhaps with the gate of a person saying "go ahead")... note not saying that you actually do this for every commit, only that you could