Self-compiled Roslyn build performance: Not as fast as originally shipped Roslyn version

Question

What I am doing in one sentence

Checking out the branch Update-1 from the Roslyn github repository, building the csc.exe, and compiling a random Solution with the csc.exe version I just built myself.

Expected result

I expect the performance to be comparable to the original Roslyn version which was shipped with VS 2015 Update 1, located in the path: C:\Program Files (x86)\MSBuild\14.0\Bin

Actual result

The build performance of the Roslyn version I built myself is significantly worse, compared to the original one. In the solution I used in my tests: roughly 30 seconds vs. 65 seconds.

My procedure in more detail

• Cloned https://github.com/dotnet/roslyn.git and checked out branch update-1
• Built Roslyn using Release configuration (tried with both, Mixed Configuration as well as x64 for the solution platform.)

• To compile a test solution, I altered all project files to use a specific location for the csc path:

  <CscToolPath>C:\Path\To\Output\Location\Of\Roslyn\Binaries\</CscToolPath>

• For testing purposes I am building the test solution from command line with

  MSBuild.exe /t:Rebuild /m:1 /verbosity:m MySolutionName.sln

• I am currently also doing a clean before:

  MSBuild.exe /t:Clean /m:1 /verbosity:m MySolutionName.sln

• To test against the original compiler, shipped with VS 2015 Update 1, I am changing the configuration in the project files to:

  <CscToolPath>C:\Program Files (x86)\MSBuild\14.0\Bin\</CscToolPath>

Questions

• What can I do to to achieve a similar performance with my self-compiled Roslyn version, like with the originally shipped Roslyn dlls?
• Are there any further things (like optimizations etc.) to be considered when building Roslyn itself?

Answer 1

In addition to Kevin's Answer which is totally correct, here are a few more details on signing/NGEN compiling Roslyn binaries, as this may be interesting for other people.

• To NGEN compile the binaries, they must have a strong name, which means they have to be signed
• Normally, Roslyn binaries are delay-signed with the Microsoft private key, as Kevin also pointed out.
• So we have to use our own key pair for signing
• The corresponding configuration can be found in Roslyn repo in the file build\Targets\VSL.Imports.targets
• For testing purposes, we can then replace this section:

<Choose>
    <When Condition="'$(SignAssembly)' == 'true'">
      <Choose>
        <!-- Shipping binaries in an "official" build are delay-signed with the MS key; later, the signing
             system will finish the strong-name signing. -->
        <When Condition="'$(NonShipping)' != 'true'">
          <PropertyGroup>
            <AssemblyOriginatorKeyFile>$(VSLToolsPath)\Strong Name Keys\35MSSharedLib1024.snk</AssemblyOriginatorKeyFile>
            <DelaySign>true</DelaySign>
            <PublicKey>0024000004800000940000000602000000240000525341310004000001000100b5fc90e7027f67871e773a8fde8938c81dd402ba65b9201d60593e96c492651e889cc13f1415ebb53fac1131ae0bd333c5ee6021672d9718ea31a8aebd0da0072f25d87dba6fc90ffd598ed4da35e44c398c454307e8e33b8426143daec9f596836f97c8f74750e5975c64e2189f45def46b2a2b1247adc3652bf5c308055da9</PublicKey>
            <PublicKeyToken>31BF3856AD364E35</PublicKeyToken>
          </PropertyGroup>
        </When>

        <!-- Non-shipping binaries are simply signed with the Roslyn internal key. -->
        <Otherwise>
          <PropertyGroup>
            <AssemblyOriginatorKeyFile>$(VSLToolsPath)\Strong Name Keys\RoslynInternalKey.Private.snk</AssemblyOriginatorKeyFile>
            <DelaySign>false</DelaySign>
            <PublicKey>$(RoslynInternalKey)</PublicKey>
            <PublicKeyToken>fc793a00266884fb</PublicKeyToken>
          </PropertyGroup>
        </Otherwise>
      </Choose>
    </When>
</Choose>

with, for example, this one:

<Choose>  
<When Condition="'$(SignAssembly)' == 'true'">
  <PropertyGroup>
    <AssemblyOriginatorKeyFile>C:\path\to\keyfile\TestKey.snk</AssemblyOriginatorKeyFile>
    <DelaySign>false</DelaySign>
    <PublicKey>0024000004800000940000000602000000240000525341310004000001000100B15B00E697DB995031A740A3E07A0B1DBE16AAEA61E615A013E0381B4D875F97F1792965D58810893F6D4B1C10CBD991FB8E9F1118D9C0C6F0EBCB50462FC25056E194667CB59822C18E9CB0C17DBC573291F05F7C87B51C48B377C9EEE12F6D5B331B235E5D6E3669737B210F7BE245A76B118C23EAD90FC392E4ED9F6CDFAB/PublicKey>
    <PublicKeyToken>6E0B9EF75D28854E</PublicKeyToken>
  </PropertyGroup>
</When>
</Choose>

• ...using your own key file, public key as well as public key token.
• The key file can be created inside Visual Studio or with the help of the sn.exe tool.
• sn.exe can also be used to export the public key as well as extract the public key token (for example from an assembly you could test-sign with the key)

• Then NGEN can be called like this:

  ngen.exe install "C:\path\to\Roslyn\Release\csc.exe"

(for example located in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\ngen.exe)