How does EnumerateFiles() work?

Question

I am writing a program that, when given a date, folder path, and file extension, will look into the folder and will go find all files that had a last access time from the beginning of the month to the current date, and only files with the file extension that was passed.

The files that I am looking for are always in the same level in the folder tree so I can code into the program how far to dig to find the files.

Currently my program takes about a minute for a day so today (The 16th) takes about sixteen and a half minutes.

I want to make a program that fill find all files for a date range in a folder path and will extract information from the file. I just don't want to code how deep the program has to look in case my business changes how they store their files.

I managed to make code that, if given a folder, the program will display the names of all the files for the date range but that took 25 minutes. Here is the code

TimeSpan BeginningTime = DateTime.Now.TimeOfDay;
DateTime BeginningDate = new DateTime(DateTime.Now.Year, DateTime.Now.Month, 1);
DateTime EndingDate = DateTime.Now;
string[] FoldersToLookAt = { @"e:\", @"e:\Kodak Images\", @"e:\images\", @"e:\AFSImageMerge\" };
foreach (string FolderPath in FoldersToLookAt)
{
    for (DateTime Date = BeginningDate; Date <= EndingDate; Date = Date.AddDays(1))
    {
        string DateString = Date.ToString("yyMMdd");
        string FilePath = (FolderPath + DateString);
        DirectoryInfo FilesToLookThrough = new DirectoryInfo(FilePath);
        if (FilesToLookThrough.Exists)
        {
            foreach (var MyFile in FilesToLookThrough.EnumerateFiles("*.dat", SearchOption.AllDirectories))
            {
                if (MyFile.LastAccessTime >= BeginningDate)
                {
                    Console.WriteLine(MyFile.FullName);
                }
            }
        }
    }
}

From what I see this first obtains all the files, then goes through all files and prints out all files that have a last access time greater than the beginning date.

Is their a way in C# that will extract information from a file and NOT store it in a list? Or am I going to have to build the program from scratch?

Answer 1

Your question is not very clear, but looking at your code, and what you state you want to achieve, I would recommend that you get rid of the loop that takes you through the date folders. Just use the "AllDirectories" option, under each of the top-level folders. It is recursive, so it will go down through as many levels as there are.

TimeSpan BeginningTime = DateTime.Now.TimeOfDay;
DateTime BeginningDate = new DateTime(DateTime.Now.Year, DateTime.Now.Month, 1);
DateTime EndingDate = DateTime.Now;
string[] FoldersToLookAt = { @"e:\", @"e:\Kodak Images\", @"e:\images\", @"e:\AFSImageMerge\" };
foreach (string FolderPath in FoldersToLookAt)
{
    FilesToLookThrough = new DirectoryInfo(FolderPath);
    if (FilesToLookThrough.Exists)
    {
        foreach (var MyFile in FilesToLookThrough.EnumerateFiles("*.dat", SearchOption.AllDirectories))
        {
            if (MyFile.LastAccessTime >= BeginningDate)
            {
                Console.WriteLine(MyFile.FullName);
            }
        }
    }
}

EDIT: The other answer makes a good point, since you're going through "e:\", you probably don't need to go through the other "FoldersToLookAt", since they will all be searched anyway. What you might end up with here is multiple listings of the same file. If you take those out, it will run quite a bit faster.

You see, your code was pretty close in the first place. Using this approach you cut out a whole loop, and the "AllDirectories" search option will make sure that you look through all the sub-folders recursively. You are also protected against your organisation deciding not to store stuff in folders named by date etc, and now the runtime of your program is proportional only to the number of files.

Now, for extra credit, another big performance improvement can be made by not using Console.WriteLine for each item. A faster way would be to use a StringBuilder, and then spit the results out at the end.

// At the top of your program
StringBuilder sb = new StringBuilder();
// BeginningTime, BeginningDate, etc...

// Before the first loop
Console.WriteLine("Working...");

// Inside the very inner if, where your Console.WriteLine was
sb.AppendLine(MyFile.FullName);


// After the end of the outer loop
Console.WriteLine(sb.ToString());

Why does this make it better? Writing to the console is notoriously slow, it actually involves sending Windows into kernel mode and back, it's really, really slow. Doing it once, event with a much larger chunk of text, is much, much quicker, than doing it lots. Now, why use a StringBuilder instead of just doing a good old:

string output;

for(...)
{
     output += filename + Environment.NewLine;
}

In C# adding two strings to eachother creates a new string. Doing this over and over again is also slow, especially as the new string gets larger. StringBuilder just maintains a list of all the strings, and creates a new buffer and copies them all in, once, when you call ToString().