Using C# 5.0 async to read a file

Question

I'm just starting out with C#'s new async features. I've read plenty of how-to's now on parallel downloads etc. but nothing on reading/processing a text file.

I had an old script I use to filter a log file and figured I'd have a go at upgrading it. However I'm unsure if my usage of the new async/await syntax is correct.

In my head I see this reading the file line by line and passing it on for processing in different thread so it can continue without waiting for a result.

Am I thinking about it correctly, or what is the best way to implement this?

static async Task<string[]> FilterLogFile(string fileLocation)
{
    string line;

    List<string> matches = new List<string>();

    using(TextReader file = File.OpenText(fileLocation))
    {        
        while((line = await file.ReadLineAsync()) != null)
        {
            CheckForMatch(line, matches);
        }
    }

    return matches.ToArray();
}

The full script: http://share.linqpad.net/29kgbe.linq

Answer 1

In my head I see this reading the file line by line and passing it on for processing in different thread so it can continue without waiting for a result.

But that's not what your code does. Instead, you will (asynchronously) return an array when all reading is done. If you actually want to asynchronously return the matches one by one, you would need some sort of asynchronous collection. You could use a block from TPL Dataflow for that. For example:

ISourceBlock<string> FilterLogFile(string fileLocation)
{
    var block = new BufferBlock<string>();

    Task.Run(async () =>
    {
        string line;

        using(TextReader file = File.OpenText(fileLocation))
        {        
            while((line = await file.ReadLineAsync()) != null)
            {
                var match = GetMatch(line);

                if (match != null)
                    block.Post(match);
            }
        }

        block.Complete();
    });

    return block;
}

(You would need to add error handling, probably by faulting the returned block.)

You would then link the returned block to another block that will process the results. Or you could read them directly from the block (by using ReceiveAsync()).

---

But looking at the full code, I'm not sure this approach would be that useful to you. Because of the way you process the results (grouping and then ordering by count in each group), you can't do much with them until you have all of them.