Build server: Best practices managing third party components?

Question

I'm maintaining a quite large legacy application. The source tree is a real mess. I'm trying to set up a build server.

On the source tree, I've third party component with sources (also in the project's include path). These components are also installed within the IDE.

My question is : How to manage those components ?

I thought to manage this way:

• Install the IDE on the build server
• Install all the third party component
• Remove the component sources from the project sources tree (and keep them on the project root on a dedicated folder each zipped)
• Each time we need to customize (or debug) a third party component we re-build the package and re-install it in the IDE of the build server (and on each developers workstation)

What's the difference between having the components installed in the IDE and having the sources in the include path ? How the linker handle that case ?

Answer 1

We have set up our daily builds using simple command files.

1. Each project (.dpr) has an associated Build.cmd file.
2. All Build.cmd files are called from our main BuildServerRun.cmd file.

The BuildServerRun.cmd file takes care of

1. Deleting the entire source tree on the buildserver.
2. Getting a latest version from our source control repository.
3. Call each Build.cmd and pipe the output to a file.
4. Mail the results to all developers.

All paths to external components are configured in the dcc32.cfg file

..    
-u"c:\Program files\Developer Express Inc\ExpressInplaceEditors\Delphi 5\Lib"
-u"c:\Program files\Developer Express Inc\ExpressQuantumGrid\Delphi 5\Lib"
..
-r"c:\Program Files\Borland\Delphi5\Lib"
-r"C:\Program Files\jvcl\jvcl\resources"
..
-i"C:\Program Files\jvcl\jvcl\run"
-i"C:\Program Files\jvcl\jcl\source"

---

Example of a Build.cmd.

Note: we have a policy to compile to bin\dcu, exe to bin, hence the -N, -E directives.

@echo on
dcc32speed -B -Q -W -H -Nbin\dcu -Ebin BpABA.dpr
@echo off

Example of snipped BuildServerRun.cmd

SET %Drive%=E:

:BuildServer
REM *************************************************
REM     Clear files
REM *************************************************
ECHO. > "%Temp%\BuildLieven.txt"
ECHO. > "%Temp%\TestRunLieven.txt"

REM *************************************************
REM     Set start time
REM *************************************************
echo | TIME | FIND "Huidige tijd" > "%Temp%\ResultLieven.txt"

REM *************************************************
REM     Get latest versions
REM *************************************************
IF %LatestVersion%==1 CALL %Drive%\buildserver\latestversion.cmd
ECHO "Latest versions opgehaald" >> "%Temp%\ResultLieven.txt"

REM *************************************************
REM     Build projects
REM *************************************************
CD %Drive%\Projects\

ECHO ***************************************************************** >> "%Temp%\BuildLieven.txt"
ECHO BpABA >> "%Temp%\BuildLieven.txt"
ECHO ***************************************************************** >> "%Temp%\BuildLieven.txt"
CD %Drive%\Projects\BPABA\production
ECHO Building BPABA\production
CALL Build.cmd >> "%Temp%\BuildLieven.txt"
CD %Drive%\Projects\BPABA\test
ECHO Building BPABA\test
CALL Build.cmd >> "%Temp%\BuildLieven.txt"
CD %Drive%\Projects\BPABA\test\dunit
ECHO Building BPABA\test\dunit
CALL Build.cmd >> "%Temp%\BuildLieven.txt"
ECHO BPABATests >> "%Temp%\TestRunLieven.txt"
ECHO Running BPABATests
CALL bin\BPABATests >> "%Temp%\TestRunLieven.txt"
CD %Drive%\Projects
ECHO. >> "%Temp%\BuildLieven.txt"
ECHO. >> "%Temp%\BuildLieven.txt"
ECHO. >> "%Temp%\BuildLieven.txt"

REM *****************************************************************
REM     Gather (Fatal)Errors/Hints/Warnings & Failures
REM *****************************************************************
ECHO ***************************************************************** >> "%Temp%\ResultLieven.txt"
ECHO (Fatal)Errors/Hints/Warnings en Failures >> "%Temp%\ResultLieven.txt"
ECHO ***************************************************************** >> "%Temp%\ResultLieven.txt"
ECHO Fatal errors during build >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND /c "Fatal:" >> "%Temp%\ResultLieven.txt"


ECHO Errors during build >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND /c "Error:" >> "%Temp%\ResultLieven.txt"

ECHO Warnings during build >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND /c "Warning:" >> "%Temp%\ResultLieven.txt"

ECHO Hints during build >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND /c "Hint:" >> "%Temp%\ResultLieven.txt"

ECHO Failures during test >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\TestRunLieven.txt" | FIND /c "Failures:" >> "%Temp%\ResultLieven.txt"
ECHO. >> "%Temp%\ResultLieven.txt"

ECHO ***************************************************************** >> "%Temp%\ResultLieven.txt"
ECHO Controle #Projecten = #Compiles >> "%Temp%\ResultLieven.txt"
ECHO ***************************************************************** >> "%Temp%\ResultLieven.txt"
ECHO #Projecten >> "%Temp%\ResultLieven.txt"
TYPE "%Drive%\buildserver\buildserverrun.cmd" | FIND /i "cmd >> " | FIND /i "Lieven" | FIND /i /v /c "FIND /i /v /c" >> "%Temp%\ResultLieven.txt"
ECHO #Compiles >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\buildLieven.txt" | FIND /i /c "dcc32" >> "%Temp%\ResultLieven.txt"
ECHO #Tests expected to run >> "%Temp%\ResultLieven.txt"
TYPE "%Drive%\buildserver\buildserverrun.cmd" | FIND /i "TestRunLieven" | FIND /i "CALL" | FIND /i /v /c "FIND /i /v /c" >> "%Temp%\ResultLieven.txt"
ECHO #Tests actually run >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\TestRunLieven.txt" | FIND /i /c "DUnit / Testing" >> "%Temp%\ResultLieven.txt"
ECHO. >> "%Temp%\ResultLieven.txt"
ECHO. >> "%Temp%\ResultLieven.txt"

ECHO ***************************************************************** >> "%Temp%\ResultLieven.txt"
ECHO Detail (Fatal)Errors/Hints/Warnings en Failures >> "%Temp%\ResultLieven.txt"
ECHO ***************************************************************** >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND "Fatal:" >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND "Error:" >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND "Warning:" >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\BuildLieven.txt" | FIND "Hint:" >> "%Temp%\ResultLieven.txt"
TYPE "%Temp%\TestRunLieven.txt" | FIND "Failures:" >> "%Temp%\ResultLieven.txt"

REM *************************************************
REM     Set stop time
REM *************************************************
ECHO | TIME | FIND "Huidige tijd" >> "%Temp%\ResultLieven.txt"

REM *************************************************
REM     Send results
REM *************************************************
CALL %drive%\buildserver\Blat.cmd

Answer 2

My answer is more general than Lieven's answer, which is Delphi-specific. I wrote this one shortly after the question, but went to a co-worker before submitting ;)

I refuse to install any IDE on our main Windows build agent. Sounds like a nightmare to me. The MSBuild engine handles all build scenarios well, and other than .NET, you just need the Windows SDK installed. Or you could use NAnt and even CMake, whatever. Just don't install IDEs. It's not fun on build servers.

Now you have tagged this as Delphi. I don't know how good it works there, but as Lieven wrote, Delphi comes with a command line compiler. I just don't have any experience how it works with third-party compnents, but I think Delphi supports MSBuild in the latest version.

I'm also unsure whether including thiry-party components into version control is a good thing, because of the space it takes - though you can also put them somewhere else and include them as external, which makes it much smaller, but also imposes the problem that upgrading the components for one app will upgrade them for all - so you better have good integration tests. But that is the point of a build server anyway.

Apart from that, it is always a good thing to check-out and have all components required to build the application available. You don't need to install components into an IDE if they were made well. Depending on what components they are, in many cases you don't even need to install them on developer machines. Many .NET components, for example, are available in the designer when you add a reference to them. And licensing is typically no more than "put the license file into the same directory". Well, that's how it should be, at least. If that's not how it works in Delphi today, that's likely one of the reasons Delphi is on its way out. Other than the Borland/Inprise/DevCo/Codegear/Embarcadero hassle.