Team Build now Painfully Slow

Question

We're running into performance issues with our implementation of Team Foundation Build Server and I'm running out of ideas on how to speed things up. We've already added a few PropertyGroup elements to increase the performance on several steps (SkipClean, SkipLabel, SkipInitializeWorkspace), but I think we need to undergo a major restructuring to fix things. Here's our setup:

• We've got about 40 web applications that are each very different, but run off a bunch of shared assemblies
• Each of these web applications has their own solution;
• There are around 10 to 25 shared assemblies referenced by each of these web applications;
• There exists a build definition containing all the solutions which is fired on each check-in to the trunk;

And here's the basic problems we're encountering

• During the build, it will build each shared assembly as many times as it is referenced, rather than building once and using for each app
• File copy time is reeeeally slow for the drop directory. It has to be over network share and won't take a local path.
• Every so many builds, one or more of the output files gets "locked" and causes the build to break even if compilation is fine.
• And another thing - I've also tried separate build definitions, but doing so will also force another workspace to be gotten on Get Latest version. I'd much rather have it be that the build server contains one version of the trunk to build off of.

Over the last several months we've given in to lethargy and ignored this problem, but now the build time is over an hour to an hour and a half.

I'm toying around with the idea of learning and switching to Cruise Control for the greater control I'd have. Anyone disagree with that?

Any help is most appreciated. Thanks!

Answer 1

So here's what I've done, and I've gotten the build down to 9 minutes. For the amount of projects I'm compiling, I'm fine with that.

• Created a solution which contains all shared libraries and all webs. There was a lot of extra work in this step because I had to fix a bunch of references that were legacy code or otherwise stored in multiple places.
• The thing that ended up saving the most time was to perform a file system move rather than a network copy for all output files. Since our drop location actually sits on the build server, it makes sense.

To perform the move, I just override the CoreDropBuild target within the TFSBuild.proj file:

<Target Name="CoreDropBuild"
        Condition=" '$(SkipDropBuild)'!='true' and '$(IsDesktopBuild)'!='true' "
        DependsOnTargets="$(CoreDropBuildDependsOn)" >
             <Exec Command="move $(BinariesRoot)\Release d:\BuildOutput\$(BuildNumber)\Release"/>    
</Target>