How to use Play with custom modules and continuous integration

Question

How can I set up builds of Play apps and (custom) Play modules in a CI system so that when a module's build is good, the build installs the module artifacts in a local repository and/or deploys them to a remote repository, and the apps use the artifacts in that repository?

The solution should also work well for a developer who's working locally.

I'm using Jenkins, and running in trouble whatever way I try to do this.

Before I elaborate on all the problems I've encountered, I'll wait, because it is laborious, and maybe someone else can offer how they do it.

Answer 1

I have a setup in jenkins that works well from dev to production.

First here is the configuration in the dependencies.yml for the custom module repository

repositories:
    - modules:
        type: chain
        using:
            - localModules:
                type: local
                descriptor: "${application.path}/../[module]/conf/dependencies.yml"
                artifact: "${application.path}/../[module]"
            - repoModules:
                type: http
                artifact: "http://mynexus/nexus/content/repositories/releases/com/myorg/[module]/[revision]/[module]-[revision].zip"
        contains:
            - com.myorg -> *

With this developers and jenkins search firstly in the same repository to see if a module is present and if not, got to the nexus repository to download the artifact.

To build my module in jenkins I use a custom sh script like this

#!/bin/bash
APPLICATION="myModule"
PLAY_PATH="/usr/local/play"
set –xe

$PLAY_PATH/play deps --sync
$PLAY_PATH/play build-module --require 1.2.3
VERSION=`grep self conf/dependencies.yml | sed "s/.*$APPLICATION //"`
echo "Sending $APPLICATION-$VERSION.zip to nexus repository"
curl --request POST --user user:passwd http://mynexus/nexus/content/repositories/releases/com/myorg/$APPLICATION/$VERSION/$APPLICATION-$VERSION.zip -F "file=@dist/$APPLICATION-$VERSION.zip"  --verbose

With this script you are able to push your module to nexus on each jenkins build. This is not really what I do. I use jenkins release module to push it only when I build a release. For a release I have a special script

#!/bin/bash
APPLICATION="myModule"
PLAY_PATH="/usr/local/play"
set –xe

if [ -z "$RELEASE_VERSION" ]
then
  echo "Parameter RELEASE_VERSION is mandatory"
  exit 1
fi
if [ -z "$DEVELOPMENT_VERSION" ]
then
  echo "Parameter DEVELOPMENT_VERSION is mandatory"
  exit 1
fi
echo "Release version : $RELEASE_VERSION"
echo "Development version : $DEVELOPMENT_VERSION"
VERSION=`grep self conf/dependencies.yml | sed "s/.*$APPLICATION //"`
if [ "$RELEASE_VERSION" != "$VERSION" ]
then
  echo "Release version $RELEASE_VERSION and play version $VERSION in dependencies.yml does not match : release failed"
  exit 1
fi
REVISION=`svnversion .`
echo "Tag svn repository in revision $REVISION with version $VERSION"
svn copy -m "Version $VERSION" -r $REVISION http://mysvn/myRepo/$APPLICATION/trunk/ http://mysvn/myRepo/$APPLICATION/tags/$VERSION
echo "svn tag applied"
echo "Sending $APPLICATION-$VERSION.zip to nexus repository"
curl --request POST --user user:passwd http://mynexus/nexus/content/repositories/releases/com/myorg/$APPLICATION/$VERSION/$APPLICATION-$VERSION.zip -F "file=@dist/$APPLICATION-$VERSION.zip"  --verbose
echo "$APPLICATION-$VERSION.zip sent to nexus repository"
echo "Update module to version $DEVELOPMENT_VERSION"
sed -i "s/self\(.*\)$VERSION/self\1$DEVELOPMENT_VERSION/g" conf/dependencies.yml
svn commit -m "Version $DEVELOPMENT_VERSION" conf/dependencies.yml
svn update
echo "Version $DEVELOPMENT_VERSION créée"

This script put a tag in our svn repository, push the module to nexus and update dependencies.yml file.

With this jenkins can build an app which depends on a local version of the module while it is not released and after that can build the app by downloading the module artifcat from the nexus repository. It is the same thing for developers

Answer 2

I wrote this little Play! command that basically it does the same thing, but integrates nicely into Play!

http://www.playframework.org/community/snippets/25

from play.utils import *
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
import urllib
import urllib2
import yaml

COMMANDS = ['nexus-commit',]

HELP = {
    'nexus-commit': 'push built module to nexus'
}

def execute(**kargs):
    app = kargs.get("app")
    args = kargs.get("args")
    play_env = kargs.get("env")

    print '~ '
    print '~ Commiting to nexus server'
    print '~ '

    app.check()
    nexus = app.readConf('nexus.server.url')
    user = app.readConf('nexus.server.user')
    password = app.readConf('nexus.server.password')

    if nexus:
        print '~ Found nexus repository : %s' % nexus
    else:
        print '~ No nexus server configured'
        print '~ Set up the following on your application.conf:'
        print '~    nexus.server.url'
        print '~    nexus.server.user'
        print '~    nexus.server.password'
        sys.exit(0)

    #Getting module version from dependencies file
    deps_file = os.path.join(app.path, 'conf', 'dependencies.yml')
    if os.path.exists(deps_file):
        f = open(deps_file)
        deps = yaml.load(f.read())
        #Is this a Play~ module?
        if "self" in deps:
            d = deps["self"].split(" ")
            module_version = d.pop()
            app_name = d.pop()
        else:
            app_name = app.name()
            print '~ This is not a Play module'
            module_version = app.readConf('application.version')
            if not module_version:
                print '~ '
                print '~ No application.version found in application.conf file'
                print '~ '
                module_version = raw_input('~ Provide version number to be pushed to Nexus:')
        f.close

    if module_version:
        print '~ Module version : %s' % module_version
        print '~ '
    else:
        print '~ No module version configured.'
        print '~ Configure your dependencies file properly'
        sys.exit(0)

    dist_dir = os.path.join(app.path, 'dist')

    #Only interested on .zip files
    for root, dirs, files in os.walk(dist_dir):
        files = [ fi for fi in files if fi.endswith(".zip") ]

    #Loop over all found files
    if len(files) >0:
        for file in files:
            if "-a" in args:
                #We won't ask the user if he wants to commit
                resp = "Y"
            else:
                resp = raw_input('~ Do you want to post %s to nexus? (Y/N) ' % file)
            if resp == 'Y':
                url = '%s/%s/%s-SNAPSHOT/%s-%s-SNAPSHOT.zip' % (nexus, app_name, module_version, app_name, module_version)
                print '~ '
                print '~ Sending %s to %s' % (file, url)

                try:
                    data = open(os.path.join(dist_dir, file), 'rb')
                except:
                    print '~ Error: could not open file %s for reading' % file
                    continue 

                openers = register_openers()
                #post data to Nexus using configured credentials
                passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
                passman.add_password(None, url, user, password)
                authhandler = urllib2.HTTPBasicAuthHandler(passman)
                openers.add_handler(authhandler)
                openers.add_handler(urllib2.HTTPHandler(debuglevel=1))
                datagen, headers = multipart_encode({"file": data})
                request = urllib2.Request(url, datagen, headers)

                try:
                    print urllib2.urlopen(request).read()
                    print '~ File correctly sent'
                except urllib2.HTTPError, err:
                    print '~ Error: Nexus replied with -- %s' % err

            else:
                print '~ '
                print '~ Skiping %s' % file
    else:
        print '~ '
        print '~ No module build found.'
        print '~ Try "play build-module" command first'
        print '~ '