Deployment automation

Question

I have a Lift application that is packaged as a WAR archive and must be deployed under Jetty. However, I want to be able to perform a few tasks automatically:

• Specify the target server(or collection of servers). I have multiple servers, from development to testing and production servers and I would like to be able to control the destination of a deployment with great ease.
• A destination(for instance DEVELOPMENT), may mean a collection of servers, for load balancing purposes.
• Testing phase. Basically, on every deployment I would like to run the entire set of tests and prevent deployment if the application doesn't compile or if one or more tests failed.
• The WAR archive must deployed under Jetty, again on one or more Amazon EC2 machines running Linux.(Ubuntu 12.10)

I am using SBT and I have no idea how well this would play with Puppet or something similar. How would you go about this?

Answer 1

I've been watching this question hoping someone would come up with a clever answer. Since they haven't, I guess I'll chime in with my own feelings on the matter.

To my knowledge there is no existing tool to automate these types of deployments and I assume that's because there are so many different scenarios to deal with. At a minimum, you've got:

1. A single dev server, where it's just a matter of copying the WAR in place and then restarting the app server.
2. A single production server, where the process is similar, but you don't want the process to disrupt your users. Considerations here include retaining session data across restarts and planning the restart for a time with low usage.
3. A small cluster where you have a load balancer over many nodes. Now things really get complicated. You could be using a variety of different LB tools (HA Proxy, NGINX, Amazon's Elastic LB) and if you care about your users experience, you need to coordinate a rolling restart of the app servers. Migrating any session data between nodes is also a concern.
4. You have a large cluster consisting of smaller clusters in different geographic regions. Here you're dealing with #3 + whatever configuration needs to be done to coordinate between the regions as well.

I suppose 1 & 2 would be the easiest to find a generic tool for, and if those were the only situations I needed to deal with, I'd probably just deploy Jenkins along with the app. It can be configured pretty easily to pull from a git branch when changes occur, build the code and restart Jetty. By the time you get to 3 & 4 though, I think the number of different tools involved and the need to coordinate them has precluded any type of standard solution. I don't think this is just an issue in the Java/Scala world either since I've seen write-ups from the Github folks on the custom tools they've built to manage deployments of their Rails app.

As for Puppet, with the caveat that I've never used it, it seems like it could be a useful tool for this type of process. You'll need some type of central coordinator to deal with a cluster and I believe that Puppet can help with that.