CSV Reading as empty

Question

I've got a page where by intervals of around 10 minutes a csv file is uploaded to a folder (received by an http link). This csv file has to be uploaded to sql. I've managed to get the csv files and save them in the folder, but the problem that I've got is that when I try and read the data it shows that the file is empty (but it is not)... It doesn't throw any errors, but when I run it with a debug, it shows an "Object reference not set to an instance of an object".

This is my code...

Method of what has to happen for this entire process:

    private void RunCSVGetToSqlOrder()
    {
        DAL d = new DAL();

        GetGeoLocations();

        string geoLocPath = Server.MapPath("~\\GeoLocations\\GeoLocation.csv");
        string assetListPath = Server.MapPath("~\\GeoLocations\\AssetList.csv");

        d.InsertAssetGeoLocation(ImportGeoLocations(geoLocPath));
        d.InsertAssetList(ImportAssetList(assetListPath));

        DeleteFileFromFolder();
    }

Getting the csv and saving into a folder (working):

    private void GetGeoLocations()
    {
        string linkGeoLoc = "http://app03.gpsts.co.za/io/api/asset_group/APIAssetGroupLocation.class?zqcEK60SxfoP4fVppcLoCXFWUfVRVkKS@auth_token@auth_token";
        string filepathGeoLoc = Server.MapPath("~\\GeoLocations\\GeoLocation.csv");

        using (WebClient wc = new WebClient())
        {
            wc.DownloadFileAsync(new System.Uri(linkGeoLoc), filepathGeoLoc);
        }
    }

Read csv file and import to sql:

    private static DataTable ImportGeoLocations(string csvFilePath)
    {
        DataTable csvData = new DataTable();
        try
        {
            using (TextFieldParser csvReader = new TextFieldParser(csvFilePath))
            {
              //  csvReader.TextFieldType = FieldType.Delimited;
                csvReader.SetDelimiters(new string[] { "," });
                csvReader.HasFieldsEnclosedInQuotes = true;
                csvReader.TrimWhiteSpace = true;

                string[] colFields = csvReader.ReadFields();

                foreach (string column in colFields)
                {
                    DataColumn datecolumn = new DataColumn(column);
                    datecolumn.AllowDBNull = true;
                    csvData.Columns.Add(datecolumn);
                }
                while (!csvReader.EndOfData)
                {
                    string[] fieldData = csvReader.ReadFields();
                    //Making empty value as null
                    for (int i = 0; i < fieldData.Length; i++)
                    {
                        if (fieldData[i] == "")
                        {
                            fieldData[i] = null;
                        }
                    }
                    csvData.Rows.Add(fieldData);
                }
            }
        }
        catch (Exception ex)
        {
        }
        return csvData;
    }

The above code gives the error of "Object reference not set to an instance of an object" on the line, but this is most probably due because it's reading the csv as empty(null)...

    string[] colFields = csvReader.ReadFields();

I'm not sure what I'm doing wrong... Any advice would be greatly appreciated...

------------ EDIT --------------

The csv file after the download looks as follows:

-------- Solution ------------

Below is the solution:

     private void RunCSVGetToSqlOrder()
    {
        GetGeoLocations();
        DeleteFileFromFolder();
    }

    private void GetGeoLocations()
    {
        string linkGeoLoc = "http://app03.gpsts.co.za/io/api/asset_group/APIAssetGroupLocation.class?zqcEK60SxfoP4fVppcLoCXFWUfVRVkKS@auth_token@auth_token";
        string filepathGeoLoc = Server.MapPath("~\\GeoLocations\\GeoLocation.csv");

        using (WebClient wc = new WebClient())
        {
            wc.DownloadFileAsync(new System.Uri(linkGeoLoc), filepathGeoLoc);
            wc.DownloadFileCompleted += new AsyncCompletedEventHandler(wc_DownloadFileCompletedGeoLoc);
        }

        string linkAssetList = "http://app03.gpsts.co.za/io/api/asset_group/APIAssetGroupLocation.class?zqcEK60SxfoP4fVppcLoCXFWUfVRVkKS@auth_token@auth_token";
        string filepathAssetList = Server.MapPath("~\\GeoLocations\\AssetList.csv");

        using (WebClient wc = new WebClient())
        {
            wc.DownloadFileAsync(new System.Uri(linkAssetList), filepathAssetList);
            wc.DownloadFileCompleted += new AsyncCompletedEventHandler(wc_DownloadFileCompletedAssetList);
        }
    }

    void wc_DownloadFileCompletedGeoLoc(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
    {
        DAL d = new DAL();

        string geoLocPath = Server.MapPath("~\\GeoLocations\\GeoLocation.csv");
        d.InsertAssetGeoLocation(ImportGeoLocations(geoLocPath));
    }

    void wc_DownloadFileCompletedAssetList(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
    {
        DAL d = new DAL();

        string assetListPath = Server.MapPath("~\\GeoLocations\\AssetList.csv");
        d.InsertAssetList(ImportAssetList(assetListPath));
    }

Answer 1

The call to wc.DownloadFileAsync returns a Task object and the downloaded file is only complete after the Task is completed.

This issue is hard to catch with the debugger, because the task will have enough time to complete when a breakpoint is reached or the file is manually inspected later. However, when the code runs without break point, the call d.InsertAssetGeoLocation(ImportGeoLocations(geoLocPath)) will be reached before the download is complete and therefore the csv file will be empty.

Possible solutions:

• Redesign the code with async / await
• Use wd.DownloadFileCompleted event for continuation
• Use the synchronous wd.DownloadFile method

... just to name a few.