Continuous Integration Advice?

Question

I'm working on setting up a Continuous Integration server (using Integrity) for my Rails app, and I'd like advice:

• Do most folks set up CI to build and test their app on every push to their central SCM repository, or only when pushing to their staging branch?
• I'll use the CI server to automatically run flay, flog, reek, and rcov -- are there any other testing or code-coverage tools I should run?
• I'm planning to deploy my app on Slicehost. Should I set up the CI server on a separate Slicehost slice that's set up to be identical (wrt installed gems, libraries, etc.) to my production slice?
• If I do use a separate slice for CI, is there any harm in using the CI slice for a staging server as well?

kind regards,

Jacob

Answer 1

Do most folks set up CI to build and test their app on every push to their central SCM repository, or only when pushing to their staging branch?

Depends on the team size and velocity of the project. The more people I have working on diverse branches of the code, the more often and vigorous I want the CI to run. I would recommend you start with as much CI as you can muster and gradually back it off as you see fit.

I'll use the CI server to automatically run flay, flog, reek, and rcov -- are there any other testing or code-coverage tools I should run?

Everything covered by metric-fu is a good start. If your team has a tech writer and/or documentation is part of your delivery, you can throw rdoc in there as well.

I'm planning to deploy my app on Slicehost. Should I set up the CI server on a separate Slicehost slice that's set up to be identical (wrt installed gems, libraries, etc.) to my production slice?

If you can afford it, yes. Usually small teams and fresh start-ups can't afford to have a dedicated server for every task, but I'm a huge proponent of isolation. Regarding the identical set-up, vendor everything that you can; setting up a fresh server should be quick, simple, and automated.

If I do use a separate slice for CI, is there any harm in using the CI slice for a staging server as well?

The only harm is cross-contamination between shared resources, e.g. gem versions, database resources, etc. If you're methodical you'll probably be OK, but if you can afford separate servers per environment I'd lean on the side of better safe than sorry.

Answer 2

We use Rackspace for deployment and testing. For development we have three slices arranged as follows:

Dev

Hosts a mercurial repository, continuously updated by developers (usually just merging their branch to the main branch). Tests are not triggered on check-in but scheduled hourly. This server has a MySQL installation with a test database.

Test

Hosts a mercurial repository, updated only by dev manager. Used for acceptance testing but also has nightly test runs. This server has a MyDQL installation with a test database.

Deploy

After a successful QA, we push the updates from Test to the Deploy server. This server can be considered a "setup.exe" - customer slices should initialise smoothly from a backup image of this slice. Testing on this slice really revolves around ensuring that this can happen.

New customer setup

Rackspace allows you to copies of a server so we can go back several incarnations of the system.

When a new customer needs to be setup we initialise their new slice from the latest version of Deploy, add a cert for HTTPS and register the new slice in the DNS.

We also store things like /etc/nginx/nginx.conf and /etc/mongrel_cluster under version control.

Chris