dotnet core 2 long build time because of long restore time

Question

I noticed that building in dotnet core 2 seemed a lot slower.
But the timing after the build always showed 'only' 15 seconds.
I couldn't believe that so I timed it with time.

> time dotnet build
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  hrm -> /Users/r/dev/hrm/bin/Debug/netcoreapp2.0/hrm.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:15.45

real    0m52.366s
user    0m36.851s
sys     0m15.458s

That seemed more correct. Almost a minute.
I then tried without restore and it was a lot faster:

> time dotnet build --no-restore
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  hrm -> /Users/r/dev/hrm/bin/Debug/netcoreapp2.0/hrm.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:15.39

real    0m15.795s
user    0m11.397s
sys     0m4.238s

But dotnet also shows 15 seconds.
Could it be that only building is counted in the timings?
Not sure why a restore is always slow when everything is already restored.

Are there other ways I could speed up the building process? Disable telemetry? (I'm using osx, my environment is set to development)

I prefer to use dotnet watch run but that seems even slower. Running dotnet watch to view the parameters is taking 12 seconds.

> time dotnet watch
Microsoft DotNet File Watcher 2.0.0-rtm-26452

Usage: dotnet watch [options] [[--] <arg>...]

Options:
  ....


real    0m12.631s
user    0m8.880s
sys     0m3.816s

Is this only on my system?

Update:

Here is the result from dotnet restore /clp:PerformanceSummary

> dotnet restore /clp:PerformanceSummary
  Restore completed in 43.95 ms for /Users/roeland/dev/hrm/hrm.csproj.
  Restore completed in 52.73 ms for /Users/roeland/dev/hrm/hrm.csproj.
  Restore completed in 38.48 ms for /Users/roeland/dev/hrm/hrm.csproj.

Project Evaluation Performance Summary:
    36252 ms  /Users/roeland/dev/hrm/hrm.csproj          3 calls

Project Performance Summary:
    36424 ms  /Users/roeland/dev/hrm/hrm.csproj          9 calls
              24359 ms  Restore                                    1 calls
                  1 ms  _IsProjectRestoreSupported                 2 calls
              12011 ms  _GenerateRestoreProjectPathWalk            1 calls
                  1 ms  _GenerateRestoreProjectPathItemsPerFramework   1 calls
                 43 ms  _GenerateRestoreGraphProjectEntry          1 calls
                  0 ms  _GetRestoreSettingsPerFramework            1 calls
                  6 ms  _GenerateProjectRestoreGraph               1 calls
                  3 ms  _GenerateProjectRestoreGraphPerFramework   1 calls

Target Performance Summary:
        0 ms  _GenerateRestoreGraphProjectEntry          1 calls
        0 ms  _GenerateProjectRestoreGraph               1 calls
        0 ms  _GetRestoreTargetFrameworksAsItems         1 calls
        0 ms  _GetRestoreProjectStyle                    2 calls
        0 ms  CheckForImplicitPackageReferenceOverridesBeforeRestore   2 calls
        0 ms  _CheckForUnsupportedNETCoreVersion         1 calls
        0 ms  _IsProjectRestoreSupported                 1 calls
        0 ms  _GetRestoreSettingsPerFramework            1 calls
        0 ms  _GetProjectJsonPath                        2 calls
        0 ms  _GetRestoreSettingsOverrides               1 calls
        1 ms  _GenerateRestoreProjectPathWalk            1 calls
        1 ms  _GenerateRestoreProjectPathItemsPerFramework   1 calls
        1 ms  _GenerateRestoreSpecs                      1 calls
        1 ms  _GenerateRestoreProjectSpec                1 calls
        2 ms  _GenerateProjectRestoreGraphPerFramework   1 calls
        2 ms  _GetRestoreTargetFrameworksOutput          1 calls
        5 ms  _GenerateRestoreDependencies               1 calls
       10 ms  _LoadRestoreGraphEntryPoints               1 calls
       20 ms  _GenerateDotnetCliToolReferenceSpecs       1 calls
       21 ms  _GetRestoreSettings                        1 calls
       54 ms  _GenerateRestoreGraph                      1 calls
      216 ms  Restore                                    1 calls
    12007 ms  _GenerateRestoreProjectPathItems           1 calls
    12014 ms  _GetAllRestoreProjectPathItems             1 calls
    12058 ms  _FilterRestoreGraphProjectInputItems       1 calls

Task Performance Summary:
        1 ms  Message                                    3 calls
        1 ms  ConvertToAbsolutePath                      2 calls
        1 ms  GetRestorePackageReferencesTask            1 calls
        1 ms  GetRestoreProjectReferencesTask            1 calls
        2 ms  GetRestoreProjectFrameworks                1 calls
        3 ms  RemoveDuplicates                           5 calls
        4 ms  WarnForInvalidProjectsTask                 1 calls
       18 ms  GetRestoreSettingsTask                     1 calls
       20 ms  GetRestoreDotnetCliToolsTask               1 calls
      216 ms  RestoreTask                                1 calls
    36121 ms  MsBuild                                    9 calls

Answer 1

Long story short: MSBuild scans the entire folder structure based on glob patterns defined by the SDK used. This is done for each project evaluation and the NuGet restore seems to trigger at least three full evaluations.

Since it is slow to scan large directories, the SDKs define globbing patterns used to exclude some known large directories that are usually not wanted as part of the project anyway (node_modules, bower_components etc.).

It has been known that special circumstances may circumvent these optimisations and or even trigger performance bugs in the include/exclude glob pattern expansion / matching.

As a precaution, add all folders known to be excluded to the DefaultItemExcludes property (inside of a <PropertyGroup> element):

<DefaultItemExcludes>custom\node_modules\**;$(DefaultItemExcludes)</DefaultItemExcludes>