Read last 30,000 lines of a file [duplicate]

Question

If has a csv file whose data will increase by time to time. Now what i need to do is to read the last 30,000 lines.

Code :

string[] lines = File.ReadAllLines(Filename).Where(r => r.ToString() != "").ToArray();

 int count = lines.Count();

 int loopCount = count > 30000 ? count - 30000 : 0;

  for (int i = loopCount; i < lines.Count(); i++)
  {
      string[] columns = lines[i].Split(',');
      orderList.Add(columns[2]);
  }

It is working fine but the problem is

File.ReadAllLines(Filename)

Read a complete file which causes performance lack. I want something like it only reads the last 30,000 lines which iteration through the complete file.

PS : i am using .Net 3.5 . Files.ReadLines() not exists in .Net 3.5

Answer 1

You can Use File.ReadLines() Method instead of using File.ReadAllLines()

From MSDN:File.ReadLines()

The ReadLines and ReadAllLines methods differ as follows:
When you use ReadLines, you can start enumerating the collection of strings before the whole collection is returned; when you use ReadAllLines, you must wait for the whole array of strings be returned before you can access the array.

Therefore, when you are working with very large files, ReadLines can be more efficient.

Solution 1 :

        string[] lines = File.ReadAllLines(FileName).Where(r => r.ToString() != "").ToArray();

        int count = lines.Count();
        List<String> orderList = new List<String>();
        int loopCount = count > 30000 ? 30000 : 0;

        for (int i = count-1; i > loopCount; i--)
        {
            string[] columns = lines[i].Split(',');
            orderList.Add(columns[2]);
        }

Solution 2: if you are using .NET Framework 3.5 as you said in comments below , you can not use File.ReadLines() method as it is avaialble since .NET 4.0 .

You can use StreamReader as below:

        List<string> lines = new List<string>();
        List<String> orderList = new List<String>();
        String line;
        int count=0;
        using (StreamReader reader = new StreamReader("c:\\Bethlehem-Deployment.txt"))
        {
            while ((line = reader.ReadLine()) != null)
            {
                lines.Add(line);
                count++;
            }
        }

        int loopCount = (count > 30000) ? 30000 : 0;

        for (int i = count-1; i > loopCount; i--)
        {
            string[] columns = lines[i].Split(',');
            orderList.Add(columns[0]);
        }