How to should I setup my CI (jenkins) for deb packages?

Question

I have a CI setup with Jenkins and Artifactory for Java. I would like also to build and deploy deb packages. For building deb packages, I might use a Maven plugin (called from Gradle), e.g., http://mojo.codehaus.org/deb-maven-plugin/.

I am now investigating Debian repository implementations. I would like to deploy a private Debian repository to host my packages (http://wiki.debian.org/HowToSetupADebianRepository).

Are there any plugin in Jenkins that would make it easier to deploy deb packages? Which debian repository implementation should I use?

Answer 1

Just adding my 2 cents to this post.

Internally we use Freight (https://github.com/rcrowley/freight#readme) as our Debian/Ubuntu repository.

A lot of us tend to use fpm (https://github.com/jordansissel/fpm#readme) by Jordan Sissel for creating debs for internal use. This can be easily scripted inside your source repository like I do here: https://github.com/stuart-warren/logit/blob/master/make-deb

#!/bin/bash

# SET SOME VARS
installdir='/usr/lib/logit'
NAME='logit-java'
VERSION='0.5.8'
ITERATION='1'
WEBSITE='https://github.com/stuart-warren/logit'
REPO='http://nexus.stuartwarren.com/nexus'

# REMOVE PREVIOUS BUILD IF PRESENT
echo "Delete ${installdir}"
rm -rf .${installdir}

# CREATE FOLDER STRUCTURE
echo "create base dir ${installdir}"
mkdir -p .${installdir}

# PUT FILES IN THE CORRECT LOCATIONS
wget ${REPO}/content/repositories/releases/com/stuartwarren/logit/${VERSION}/logit-${VERSION}-tomcatvalve.jar -O .${installdir}/logit-${VERSION}-tomcatvalve.jar
wget ${REPO}/content/repositories/releases/com/stuartwarren/logit/${VERSION}/logit-${VERSION}-jar-with-dependencies.jar -O .${installdir}/logit-${VERSION}-jar-with-dependencies.jar
wget https://raw.github.com/stuart-warren/logit/master/LICENSE -O .${installdir}/LICENCE
wget https://raw.github.com/stuart-warren/logit/master/README.md -O .${installdir}/README.md
pushd .${installdir}
ln -sf logit-${VERSION}-tomcatvalve.jar logit-tomcatvalve.jar
ln -sf logit-${VERSION}-jar-with-dependencies.jar logit-jar-with-dependencies.jar
popd

# REMOVE OLD PACKAGES
echo "Delete old packages"
rm ${NAME}_*_all.deb

# CREATE THE DEB
echo "Build new package"
fpm \
        -n $NAME \
        -v $VERSION \
        --iteration ${ITERATION} \
        -a all \
        -m "Stuart Warren <[email protected]>" \
        --description "Library to extend Log4J 1.2 (plus now Logback 1.0, 
Java.util.logging and Tomcat AccessLog Valve) by providing 
json layouts (for logstash/greylog) and a zeromq appender" \
        --url $WEBSITE \
        --license 'Apache License, Version 2.0' \
        --vendor 'stuartwarren.com' \
        -t deb \
        -s dir \
        ${installdir:1}

echo "Delete ${installdir}"
rm -rf .${installdir}
echo "Done!"

Obviously you could just copy in any compiled files directly rather than downloading from a server, maven repo in my case. Then you can SCP the deb upto some 'incoming' directory on your repository server.

Answer 2

I am not aware of a Debian package plugin for Jenkins, and I didn't find the maven-deb-plugin suitable for my needs (see "What doesn't work" on page you linked to). Where I have a maven build job in Jenkins I add a post step shell script which increments the version in debian/changelog and runs dpkg-buildpackage -b -nc.

-nc suppresses a clean before build, which is necessary because my debian/rules file will otherwise try to run the maven targets to build jars, which Jenkins has already done. Snippet from my debian/rules:

pre-built-stamp
        mvn package
        touch pre-built-stamp

override_dh_auto_build: pre-built-stamp

So after the maven steps in Jenkins it runs the following

touch pre-built-stamp
dpkg-buildpackage -b -nc

This part is personal preference, but I do not have Jenkins push built debs straight to my repository. Instead it saves the .deb and .changes files as build artifacts so I can use the Promoted Builds Plugin to sign the .changes file and copy it to the repository (rsync). This lets my developers download and test the deb out before approving it to be pushed to our staging repository. A second promotion can then be used to push the package to a live repository.

I chose reprepro as our repository manager. Its one major drawback is that it cannot handle more than one version of a package in a distribution at once which makes rollback more painful. Aside from this found it reliable and usable, and now use it to completely mirror the main Debian repositories as well as using it to host my private repos.

Reprepro uses inoticoming to spot new incoming packages and verifies the signature on the changes file, ensuring that only Jenkins can add new packages.

I find some of the reprepro documentation online lacking, but I recommend installing it and reading the reprepro and inoticoming man pages.