Optimizing Listview in C# for large files

Question

I have a C# program that is pulling in a .csv file that is approx. 42,000 lines long. All data in the file is stored as follows:

Zipcode,City,State

I am pulling all the information into three different columns in a listview.

Currently this data takes about 30 - 50 seconds to be brought into my program. My question is how can I better optimize my code to get this time down?

The following is a snippet of my code. The commented code is code I previously tried, but had no success in reducing the time, therefore I rewrote it in a way that was easier to read.

 //These are globally declared.
lvZip.Columns.Add("Zipcode", 150, HorizontalAlignment.Left);
lvZip.Columns.Add("City", 150, HorizontalAlignment.Left);
lvZip.Columns.Add("State", 150, HorizontalAlignment.Left);
lvZip.View = View.Details;

lvZip.Items.Clear();

        //string dir = System.IO.Path.GetDirectoryName(
        //  System.Reflection.Assembly.GetExecutingAssembly().Location);

        //string path = dir + @"\zip_code_database_edited.csv";
        //var open = new StreamReader(File.OpenRead(path));

        //foreach (String s in File.ReadAllLines(path))
        //{
        //    Zipinfo = s.Split(',');
        //    Zipinfo[0] = Zipinfo[0].Trim();
        //    Zipinfo[1] = Zipinfo[1].Trim();
        //    Zipinfo[2] = Zipinfo[2].Trim();
        //    lvItem = new ListViewItem(Zipinfo);
        //    lvZip.Items.Add(lvItem);
        //}
        //open.Close();

        StreamReader myreader = File.OpenText(path);
        aLine = myreader.ReadLine();

        while (aLine != null)
        {
            Zipinfo = aLine.Split(',');
            Zipinfo[0] = Zipinfo[0].Trim();
            Zipinfo[1] = Zipinfo[1].Trim();
            Zipinfo[2] = Zipinfo[2].Trim();
            lvItem = new ListViewItem(Zipinfo);
            lvZip.Items.Add(lvItem);
            aLine = myreader.ReadLine();
        }
        myreader.Close();

Answer 1

What you should be doing is using the ListView.BeginUpdate() and the ListView.EndUpdate() before and after you add anything into the ListView. The second thing would be to use the ListView.AddRange() instead of ListView.Add(). By using the Add method, you will redraw the ListView every time you use it. However, using ListView.AddRange() you will only redraw it once. That should optimize it a little for you.

Answer 2

You can try:

lvZip.BeginUpdate();

before you start adding all the items.

Then:

lvZip.EndUpdate();

when you've finished.

This will prevent the control from drawing each item as you add it, which makes the whole process very slow.

Answer 3

Golden Rule: Don't use String.Split() to read CSV data.

The .NET Framework already has a built-in dedicated CSV parser called TextFieldParser.

It's located in the Microsoft.VisualBasic.FileIO namespace.

Not only are there many edge cases that String.Split() is not properly equipped to handle, but it's also much slower to use StreamReader.

One last remark: A tip is to use using statements in order to ensure that your disposable objects get disposed (release unmanaged resources). I see that you are not using them (pun not intended) in the above code.

It's actually not that far outside the scope of this question since efficient memory management can boost the performance of your code.