Example Puppet setup for a rails stack? (nginx, varnish, thin, postgres, memcached, redis)

Question

I'm just getting started with Puppet. The example walkthroughs and tutorials were good at helping me understand Puppet's usefulness and the basic toolset, but I'm having a hard time conceptualizing a full stack. Even the advanced tutorial didn't seem to give me a clear picture of what needs to happen.

Are there any full examples of a rails stack somewhere that I could learn from?

Answer 1

Examples of a full stack are hard to come by. You should be able to find examples of modules that manage some of those specific examples, however. One problem is that it can be a lot of extra work to create a module that has abstracted away all site-specific assumptions and that is truly cross-platform.

http://forge.puppetlabs.com/ is the canonical location for modules that people wish to share. With a quick scan I found modules for nginx, varnish, and postgres.

You'll want to start with the Puppet Best Practices for the basic setup.

From there, you're going to (at least), want a module for nginx, varnish, thin, postgres, memcached, redis, and a site module (probably named after your site).

In your nodes.pp, each system will have a fairly simple assignment to a role. ("include role")

In your "site" module, you'll want a sub-class for each system role (I'm assuming you'll have multiple sets of servers, and that within a set, they are intended to be basically identical to each other. I'm also assuming that you're likely to have more than one of the above included). You may also want a site::commonvariables class (or something like that) for things (such as lists of servers in a role, passwords, etc) that you may need across multiple other modules or classes. The best practices seem to have these site::role things in a /services secondary module area with names more like s_role, so you may want to follow that naming/placement scheme instead. These role classes will include the classes for the actual components that are needed on those roles, call defines, etc.

For each of the 6 components you mention, you'll have a module. Within that module, you're likely to want to have something like a "server" and "client" subclass. And possibly a third class included by client and server for things needed by both (common libraries, etc). And within the server subclass, a define that sets up specific instances (virtualhosts, databases, etc). (if it's absolutely only ever a server, maybe skip that level of subclassing).

So, for example:

• postgres module (manifests, files, templates, etc)
  • postgres class (in init.pp): maybe empty class, maybe things needed by client and server
    • postgres::client class: install postgres client libraries
    • postgres::server class: install postgres server code, make sure postgres service is running, configure it, set up backups, etc
      • postgres::server::database define: inside the server class, a define that takes parameters such as database name, username, password, and creates the database and user and gives the user access to the DB. Maybe this is two or three separate defines, depending on how you prefer to model things.

It's best if the component modules are kept fairly independent (and reusable) and your role classes is where all the more site-specific configuration happens, but it's not the end of the world if your component modules include some site-specific stuff.