What is the best practice for "Copy Local" and with project references?

Question

In a previous project I worked with one big solution with project references and bumped into a performance problem as well. The solution was three fold:

1. Always set the Copy Local property to false and enforce this via a custom msbuild step

2. Set the output directory for each project to the same directory (preferably relative to $(SolutionDir)

3. The default cs targets that get shipped with the framework calculate the set of references to be copied to the output directory of the project currently being built. Since this requires calculating a transitive closure under the 'References' relation this can become VERY costly. My workaround for this was to redefine the GetCopyToOutputDirectoryItems target in a common targets file (eg. Common.targets ) that's imported in every project after the import of the Microsoft.CSharp.targets. Resulting in every project file to look like the following:

   <Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
     <PropertyGroup>
       ... snip ...
     </ItemGroup>
     <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
     <Import Project="[relative path to Common.targets]" />
     <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
          Other similar extension points exist, see Microsoft.Common.targets.
     <Target Name="BeforeBuild">
     </Target>
     <Target Name="AfterBuild">
     </Target>
     -->
   </Project>
   

This reduced our build time at a given time from a couple of hours (mostly due to memory constraints), to a couple of minutes.

The redefined GetCopyToOutputDirectoryItems can be created by copying the lines 2,438–2,450 and 2,474–2,524 from C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets into Common.targets.

For completeness the resulting target definition then becomes:

<!-- This is a modified version of the Microsoft.Common.targets
     version of this target it does not include transitively
     referenced projects. Since this leads to enormous memory
     consumption and is not needed since we use the single
     output directory strategy.
============================================================
                    GetCopyToOutputDirectoryItems

Get all project items that may need to be transferred to the
output directory.
============================================================ -->
<Target
    Name="GetCopyToOutputDirectoryItems"
    Outputs="@(AllItemsFullPathWithTargetPath)"
    DependsOnTargets="AssignTargetPaths;_SplitProjectReferencesByFileExistence">

    <!-- Get items from this project last so that they will be copied last. -->
    <CreateItem
        Include="@(ContentWithTargetPath->'%(FullPath)')"
        Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_EmbeddedResourceWithTargetPath->'%(FullPath)')"
        Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(Compile->'%(FullPath)')"
        Condition="'%(Compile.CopyToOutputDirectory)'=='Always' or '%(Compile.CopyToOutputDirectory)'=='PreserveNewest'">
        <Output TaskParameter="Include" ItemName="_CompileItemsToCopy"/>
    </CreateItem>
    <AssignTargetPath Files="@(_CompileItemsToCopy)" RootFolder="$(MSBuildProjectDirectory)">
        <Output TaskParameter="AssignedFiles" ItemName="_CompileItemsToCopyWithTargetPath" />
    </AssignTargetPath>
    <CreateItem Include="@(_CompileItemsToCopyWithTargetPath)">
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_NoneWithTargetPath->'%(FullPath)')"
        Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>
</Target>

With this workaround in place I found it workable to have as much as > 120 projects in one solution, this has the main benefit that the build order of the projects can still be determined by VS instead of doing that by hand by splitting up your solution.

I'll suggest you to read Patric Smacchia's articles on that subject :

• Partitioning Your Code Base Through .NET Assemblies and Visual Studio Projects --> Should every Visual Studio project really be in its own assembly? And what does 'Copy Local=True' really mean?
• Lessons learned from the NUnit code base --> The VisualStudio Project Reference + Copy Local true option is evil!)
• Analyzing the code base of CruiseControl.NET --> Bad usage of Copy Local Reference Assembly option set to True)

CC.Net VS projects rely on the copy local reference assembly option set to true. [...] Not only this increase significantly the compilation time (x3 in the case of NUnit), but also it messes up your working environment. Last but not least, doing so introduces the risk for versioning potential problems. Btw, NDepend will emit a warning if it founds 2 assemblies in 2 different directories with the same name, but not the same content or version.

The right thing to do is to define 2 directories $RootDir$\bin\Debug and $RootDir$\bin\Release, and configure your VisualStudio projects to emit assemblies in these directories. All project references should reference assemblies in the Debug directory.

You could also read this article to help you reduce your projects number and improve your compilation time.

I suggest having copy local = false for almost all projects except the one that is at the top of the dependency tree. And for all the references in the one at the top set copy local = true. I see many people suggesting sharing an output directory; I think this is a horrible idea based on experience. If your startup project holds references to a dll that any other project holds a reference to you will at some point experience an access\sharing violation even if copy local = false on everything and your build will fail. This issue is very annoying and hard to track down. I completely suggest staying away from a shard output directory and instead of having the project at the top of the dependency chain write the needed assemblies to the corresponding folder. If you don't have a project at the "top," then I would suggest a post-build copy to get everything in the right place. Also, I would try and keep in mind the ease of debugging. Any exe projects I still leave copy local=true so the F5 debugging experience will work.

You are correct. CopyLocal will absolutely kill your build times. If you have a large source tree then you should disable CopyLocal. Unfortunately it not as easy as it should be to disable it cleanly. I have answered this exact question about disabling CopyLocal at How do I override CopyLocal (Private) setting for references in .NET from MSBUILD. Check it out. As well as Best practices for large solutions in Visual Studio (2008).

Here is some more info on CopyLocal as I see it.

CopyLocal was implemented really to support local debugging. When you prepare your application for packaging and deployment you should build your projects to the same output folder and make sure you have all the references you need there.

I have written about how to deal with building large source trees in the article MSBuild: Best Practices For Creating Reliable Builds, Part 2.

In my opinion, having a solution with 100 projects is a BIG mistake. You could probably split your solution in valid logical small units, thus simplifying both maintenance and builds.

I am surprised no one has mentioned using hardlinks. Instead of copying the files, it creates a hardlink to the original file. This saves disk space as well as greatly speeding up build. This can enabled on the command line with the following properties:

/p:CreateHardLinksForAdditionalFilesIfPossible=true;CreateHardLinksForCopyAdditionalFilesIfPossible=true;CreateHardLinksForCopyFilesToOutputDirectoryIfPossible=true;CreateHardLinksForCopyLocalIfPossible=true;CreateHardLinksForPublishFilesIfPossible=true

You can also add this to a central import file so that all your projects can also get this benefit.