C# How to loop a large set of folders and files recursively without using a huge amount of memory

Question

I want to index all my music files and store them in a database. I have this function that i call recusively, starting from the root of my music drive.

i.e.

start > ReadFiles(C:\music\);

ReadFiles(path){
   foreach(file)
      save to index;

   foreach(directory)
      ReadFiles(directory);
}

This works fine, but while running the program the amount of memory that is used grows and grows and.. finally my system runs out of memory.

Does anyone have a better approach that doesnt need 4GB of RAM to complete this task?

Best Regards, Tys

Answer 1

Alxandr's queue based solution should work fine.

If you're using .NET 4.0, you could also take advantage of the new Directory.EnumerateFiles method, which enumerates files lazily, without loading them all in memory:

void ReadFiles(string path)
{
    IEnumerable<string> files =
        Directory.EnumerateFiles(
            path,
            "*",
            SearchOption.AllDirectories); // search recursively

    foreach(string file in files)
        SaveToIndex(file);
}