Optimizing a LINQ reading from System.Diagnostics.EventLog

Question

I have a performance problem on certain computers with the following query:

System.Diagnostics.EventLog log = new System.Diagnostics.EventLog("Application");

var entries = log.Entries
    .Cast<System.Diagnostics.EventLogEntry>()
    .Where(x => x.EntryType == System.Diagnostics.EventLogEntryType.Error)
    .OrderByDescending(x => x.TimeGenerated)
    .Take(cutoff)
    .Select(x => new
    {
        x.Index,
        x.TimeGenerated,
        x.EntryType,
        x.Source,
        x.InstanceId,
        x.Message
    }).ToList();

Apparently ToList() can be quite slow in certain queries, but with what should I replace it?

Answer 1

log.Entries collection works like this: it is aware about total number of events (log.Entries.Count) and when you access individual element - it makes a query to get that element.

That means when you enumerate over whole Entries collection - it will query for each individual element, so there will be Count queries. And structure of your LINQ query (for example, OrderBy) forces full enumeration of that collection. As you already know - it's very inefficient.

Much more efficient might be to query only log entries you need. For that you can use EventLogQuery class. Suppose you have simple class to hold event info details:

private class EventLogInfo {
    public int Id { get; set; }
    public string Source { get; set; }
    public string Message { get; set; }
    public DateTime? Timestamp { get; set; }
}

Then you can convert your inefficient LINQ query like this:

// query Application log, only entries with Level = 2 (that's error)
var query = new EventLogQuery("Application", PathType.LogName, "*[System/Level=2]");
// reverse default sort, by default it sorts oldest first
// but we need newest first (OrderByDescending(x => x.TimeGenerated)
query.ReverseDirection = true;            
var events = new List<EventLogInfo>();
// analog of Take
int cutoff = 100;
using (var reader = new EventLogReader(query)) {
    while (true) {
        using (var next = reader.ReadEvent()) {
            if (next == null)
                // we are done, no more events
                break;
            events.Add(new EventLogInfo {
                Id = next.Id,
                Source = next.ProviderName,
                Timestamp = next.TimeCreated,
                Message = next.FormatDescription()
            });
            cutoff--;
            if (cutoff == 0)
                // we are done, took as much as we need
                break;
        }
    }
}

It will be 10-100 times faster. However, this API is more low-level and returns instances of EventRecord (and not EventLogEntry), so for some information there might be different ways to obtain it (compared to EventLogEntry).

If you decide that you absolutely must use log.Entries and EventLogEntry, then at least enumerate Entries backwards. That's because newest events are in the end (its sorted by timestamp ascending), and you need top X errors by timestamp descended.

EventLog log = new System.Diagnostics.EventLog("Application");
int cutoff = 100;
var events = new List<EventLogEntry>();
for (int i = log.Entries.Count - 1; i >= 0; i--) {
    // note that line below might throw ArgumentException
    // if, for example, entries were deleted in the middle
    // of our loop. That's rare condition, but robust code should handle it
    var next = log.Entries[i];
    if (next.EntryType == EventLogEntryType.Error) {
        // add what you need here
        events.Add(next);
        // got as much as we need, break
        if (events.Count == cutoff)
            break;
    }
}

That is less efficient, but still should be 10 times faster than your current approach. Note that it's faster because Entries collection is not materialized in memory. Individual elements are queried when you access them, and when enumerating backwards in your specific case - there is high chance to query much less elements.