Breaking change in GetFiles with 0.15.2 can't exclude folders

Question

I can't remember where but found an example how to exclude folder from being search. Our issue was search node_modules would cause long path exceptions.

Func<IFileSystemInfo, bool> exclude_node_modules = fileSystemInfo=>!fileSystemInfo.Path.FullPath.Contains("node_modules");

var solutions = GetFiles("./**/*.sln", exclude_node_modules);

Any help to resolve this issue would be helpful.

Answer 1

To speed up the recursive walking of the file system Cake utilizes the .NET built in features to do so, it's limited by the old 260 char limit of Windows though. So when it's a magnitude faster in most use cases it fails on too deep folder structures like i.e. Node modules can introduce.

You can solve this by iterating folder by folder and applying predicate on which folder to exclude before entering it.

In my example the below folder structure is used

Repo directory
    |   build.cake
    |   test.sln
    |
    \---src
        |   test.sln
        |
        +---proj1
        |   |   test.sln
        |   |
        |   \---node_modules
        |           node.sln
        |
        +---proj2
        |   |   test.sln
        |   |
        |   \---node_modules
        |           node.sln
        |
        +---proj3
        |   |   test.sln
        |   |
        |   \---node_modules
        |           node.sln
        |
        \---proj4
            |   test.sln
            |
            \---node_modules
                    node.sln

What we want is to find all solutions recursively from repo directory not entering the the node_modules directory and not finding the node.sln

Below is suggested solution for this would be to create a utility method called RecursiveGetFile that does this for you:

// find and iterate all solution files
foreach(var filePath in RecursiveGetFile(
    Context,
    "./",
    "*.sln",
    path=>!path.EndsWith("node_modules", StringComparison.OrdinalIgnoreCase)
    ))
{
    Information("{0}", filePath);
}


// Utility method to recursively find files
public static IEnumerable<FilePath> RecursiveGetFile(
    ICakeContext context,
    DirectoryPath directoryPath,
    string filter,
    Func<string, bool> predicate
    )
{
    var directory = context.FileSystem.GetDirectory(context.MakeAbsolute(directoryPath));
    foreach(var file in directory.GetFiles(filter, SearchScope.Current))
    {
        yield return file.Path;
    }
    foreach(var file in directory.GetDirectories("*.*", SearchScope.Current)
        .Where(dir=>predicate(dir.Path.FullPath))
        .SelectMany(childDirectory=>RecursiveGetFile(context, childDirectory.Path, filter, predicate))
        )
    {
        yield return file;
    }
}

The output of this script would be something like

RepoRoot/test.sln
RepoRoot/src/test.sln
RepoRoot/src/proj1/test.sln
RepoRoot/src/proj2/test.sln
RepoRoot/src/proj3/test.sln
RepoRoot/src/proj4/test.sln

This voids the 260 char issue by skipping known trouble makers, won't fix if other unknown paths has same issue.