How do you mock OpsWorks specific services/dependencies when developing locally with Kitchen and Chef?

Question

I'm writing Chef wrappers around some of the built in OpsWorks cookbooks. I'm using Berkshelf to clone the OpsWorks cookbooks from their github repo.

This is my Berksfile:

source 'https://supermarket.getchef.com'

metadata

def opsworks_cookbook(name)
  cookbook name, github: 'aws/opsworks-cookbooks', branch: 'release-chef-11.10', rel: name
end

%w(dependencies scm_helper mod_php5_apache2 ssh_users opsworks_agent_monit
   opsworks_java gem_support opsworks_commons opsworks_initial_setup
   opsworks_nodejs opsworks_aws_flow_ruby
   deploy mysql memcached).each do |cb|
  opsworks_cookbook cb
end

My metadata.rb:

depends 'deploy'
depends 'mysql'
depends 'memcached'

The problem is, when I try to override attributes that depend on the opsworks key in the node hash, I get a:

NoMethodError
-------------
undefined method `[]=' for nil:NilClass

OpsWorks has a whole bunch of pre-recipe dependencies that create these keys and do a lot of their setup. I'd like to find a way to either pull in those services and run them against my Kitchen instances or mock them in a way that I can actually test my recipes.

Is there a way to do this?

Answer 1

I would HIGHLY recommend you check out Mike Greiling's blog post Simplify OpsWorks Development With Packer and his github repo opsworks-vm which help you to mock the entire opsworks stack including the install of the opsworks agent so you can also test app deploy recipes, multiple layers, multiple instances at the same time, etc . It is extremely impressive.

Quick Start on Ubuntu 14.04

NOTE: This can NOT be done from an ubuntu virtual machine because virtualbox does not support nested virtualization of 64-bit machines.

1. Install ChefDK
   1. mkdir /tmp/packages && cd /tmp/packages
   2. wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chefdk_0.8.1-1_amd64.deb
   3. sudo dpkg -i chefdk_0.8.0-1_amd64.deb
   4. cd /opt/chefdk/
   5. chef verify
   6. which ruby
   7. echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile && source ~/.bash_profile
2. Install VirtualBox
   1. echo 'deb http://download.virtualbox.org/virtualbox/debian vivid contrib' > /etc/apt/sources.list.d/virtualbox.list
   2. wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
   3. sudo apt-get update -qqy
   4. sudo apt-get install virtualbox-5.0 dkms
3. Install Vagrant
   1. cd /tmp/packages
   2. wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
   3. sudo dpkg -i vagrant_1.7.4_x86_64.deb
   4. vagrant plugin install vagrant-berkshelf
   5. vagrant plugin install vagrant-omnibus
   6. vagrant plugin list
4. Install Packer
   1. mkdir /opt/packer && cd /opt/packer
   2. wget https://dl.bintray.com/mitchellh/packer/packer_0.8.6_linux_amd64.zip
   3. unzip packer_0.8.6_linux_amd64.zip
   4. echo 'PATH=$PATH:/opt/packer' >> ~/.bash_profile && source ~/.bash_profile
5. Build Mike Greiling's opsworks-vm virtualbox image using Packer
   1. mkdir ~/packer && cd ~/packer
   2. git clone https://github.com/pixelcog/opsworks-vm.git
   3. cd opsworks-vm
   4. rake build install
   5. This will install a new virtualbox vm to ~/.vagrant.d/boxes/ubuntu1404-opsworks/

To mock a single opsworks instance, create a new Vagrantfile like so:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu1404-opsworks"
  config.vm.provision :opsworks, type: 'shell', args: 'path/to/dna.json'
end

The dna.json file path is set relative to the Vagrantfile and should contain any JSON data you wish to send to OpsWorks Chef.

For example:

{
  "deploy": {
    "my-app": {
      "application_type": "php",
      "scm": {
        "scm_type": "git",
        "repository": "path/to/my-app"
      }
    }
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "path/to/my-cookbooks"
    },
    "recipes": [
      "recipe[opsworks_initial_setup]",
      "recipe[dependencies]",
      "recipe[mod_php5_apache2]",
      "recipe[deploy::default]",
      "recipe[deploy::php]",
      "recipe[my_custom_cookbook::configure]"
    ]
  }
}

To mock multiple opsworks instances and include layers see his AWS OpsWorks "Getting Started" Example which includes the stack.json below.

Vagrantfile (for multiple instances)

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu1404-opsworks"

  # Create the php-app layer
  config.vm.define "app" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/php-app.json'
    ]

    # Forward port 80 so we can see our work
    layer.vm.network "forwarded_port", guest: 80, host: 8080
    layer.vm.network "private_network", ip: "10.10.10.10"
  end

  # Create the db-master layer
  config.vm.define "db" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/db-master.json'
    ]

    layer.vm.network "private_network", ip: "10.10.10.20"
  end
end

stack.json

{
  "opsworks": {
    "layers": {
      "php-app": {
        "instances": {
          "php-app1": {"private-ip": "10.10.10.10"}
        }
      },
      "db-master": {
        "instances": {
          "db-master1": {"private-ip": "10.10.10.20"}
        }
      }
    }
  },
  "deploy": {
    "simple-php": {
      "application_type": "php",
      "document_root": "web",
      "scm": {
        "scm_type": "git",
        "repository": "dev/simple-php"
      },
      "memcached": {},
      "database": {
        "host": "10.10.10.20",
        "database": "simple-php",
        "username": "root",
        "password": "correcthorsebatterystaple",
        "reconnect": true
      }
    }
  },
  "mysql": {
    "server_root_password": "correcthorsebatterystaple",
    "tunable": {"innodb_buffer_pool_size": "256M"}
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "ops/cookbooks"
    }
  }
}

For those not familiar with vagrant you just do a vagrant up to start the instance(s). Then you can modify your cookbook locally and any changes can be applied by re-running chef against the existing instance(s) with vagrant provision. You can do a vagrant destroy and vagrant up to start from scratch.