Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parse IIS log file - is there an alternative to LogParser

I need to parse an IIS log file. Is there any alternative to LogParser, a simple class to query a log file ?

I only need to know how many request I receive between 2 dates.

Here is an example of iis log file :

#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2014-08-26 12:20:57
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
2014-08-26 12:20:57 W3SVC1 QXXXSXXXX 172.25.161.53 POST /XXXX/XXX/XXXX/XXXXX/1.0/XXXX/XXXXXXXX/xxxxxx.svc - 443 - 999.99.999.999 HTTP/1.1 - - - xxxx.xxxx.xxx.xxx.xxxx.xxxx.xxx.com 200 0 0 4302 5562 1560
like image 962
Anas Avatar asked Aug 20 '15 14:08

Anas


People also ask

How do I extract IIS logs?

Go to Windows Start and run “inetmgr”. Alternatively, you can go to Administrative Tools → Internet Information Services (IIS) Manager. Click on “Sites” in the left-hand tree menu to display a list of sites on the right of the screen. Note your site ID number, which is what IIS saves logs based on.

Which is the recommended log file format for Logging IIS events?

By default, IIS 7 and later uses the World Wide Web Consortium (W3C) log file format. You can change this setting by modifying the logFormat attribute to IIS, NCSA, or Custom.

What is an IIS parser log used for?

IIS log parser is a command-line tool that takes the SQL-like expression as input and outputs the data that matches the user's query. Log parser can be used to query the log files, XML files, CSV files, and all other major data sources in Windows OS like Event log, Active directory, the Registry, and the file system.


3 Answers

You can use Tx (LINQ to Logs and Traces) , you can install it via nuget

and use it like this:

var iisLog = W3CEnumerable.FromFile(pathToLog);
int nbOfLogsForLastHour = iisLog.Where(x => x.dateTime > DateTime.Now.AddHours(-1)).Count();

If the log file is used by another process, you can use W3CEnumerable.FromStream

like image 145
Anas Avatar answered Oct 27 '22 20:10

Anas


You can use IISLogParser , and install it via nuget, it has support for large files (> 1Gb)

List<IISLogEvent> logs = new List<IISLogEvent>();
using (ParserEngine parser = new ParserEngine([filepath]))
{
    while (parser.MissingRecords)
    {
        logs = parser.ParseLog().ToList();
    }
}
like image 30
Kabindas Avatar answered Oct 27 '22 18:10

Kabindas


It's 2017 and the LogParser is still closed source. Moreover, all the instrumentation provided by cloud solutions appears to be making the need for parsing IIS logs a thing of the past. But since I am also dealing with legacy apps, I wrote this simple parser using .NET core.

using System;
using System.IO;
using W3CParser.Extensions;
using W3CParser.Instrumentation;
using W3CParser.Parser;

namespace W3CParser
{
    class Program
    {
        static void Main(string[] args)
        {            
            var reader = new W3CReader(File.OpenText(args.Length > 0 ? args[0] : "Data/foobar.log"));

            using (new ConsoleAutoStopWatch())
            {
                foreach (var @event in reader.Read())
                {
                    Console.WriteLine("{0} ({1}):{2}/{3} {4} (bytes sent)",
                                      @event.Status.ToString().Red().Bold(),
                                      @event.ToLocalTime(),
                                      @event.UriStem.Green(),
                                      @event.UriQuery,
                                      @event.BytesSent);
                }
            }
        }
    }
}

Source code: https://github.com/alexnolasco/32120528

like image 11
Alex Nolasco Avatar answered Oct 27 '22 20:10

Alex Nolasco