Index was outside the bounds of the array when adding class to list C#

Question

Exception: Index was outside the bounds of the array.

First off, I'm familiar with this exception and I have fixed it before but I'm getting this exception at a very strange line in my code. It is being thrown when I'm adding a user created class to a list of classes in my code. I'm completely lost as to why it is throwing this exception and how to fix it.

public static async Task getData()
    {
        // initialize everything
        List<StockData> stockData = new List<StockData>();
        List<StockMarketCompare> stockCompareData = new List<StockMarketCompare>();
        List<StockData> sandpInfo = new List<StockData>();
        List<StockData> sandpDateInfo = new List<StockData>();
        List<StockData> amexList = new List<StockData>();
        List<DateTime> completedDates = new List<DateTime>();
        SymbolInfo symbolClass = new SymbolInfo();
        List<SymbolInfo> ratingSymbols = new List<SymbolInfo>();
        List<StockRating> ratingList = new List<StockRating>();
        bool isGoodToGo = false;
        string symbol, market;
        int activeSymbolsCount = 0;
        int rowCount = 0, completedRowCount = 0;
        DateTime date = new DateTime();
        DateTime searchDate = new DateTime();

        using (SqlConnection connection = new SqlConnection("connectionstring"))
        using (SqlCommand sandpCommand = new SqlCommand("select * from dbo.DailyGlobalData where Symbol='" + Calculations.sp500 + "'", connection))
        using (SqlDataAdapter sandpAdapter = new SqlDataAdapter(sandpCommand))
        using (DataTable sandpTable = new DataTable("sandp"))
        using (SqlCommand stockRatingsCommand = new SqlCommand("select * from dbo.StockRatings", connection))
        using (SqlDataAdapter stockRatingsAdapter = new SqlDataAdapter(stockRatingsCommand))
        using (DataTable stockRatingsTable = new DataTable("stockratings"))
        {
            try
            {
                // fill the sandptable
                sandpAdapter.Fill(sandpTable);

                if (sandpTable != null)
                {
                    var sandpQuery = from c in sandpTable.AsEnumerable()
                                     select new StockData { Close = c.Field<decimal>("Close"), Date = c.Field<DateTime>("Date"), High = c.Field<decimal>("High"), Low = c.Field<decimal>("Low"), Volume = c.Field<Int64>("Volume") };
                    sandpInfo = sandpQuery.AsParallel().ToList();
                }

                // fill the stockratingstable
                stockRatingsAdapter.Fill(stockRatingsTable);

                if (stockRatingsTable != null)
                {
                    activeSymbolsCount = stockRatingsTable.Rows.Count;

                    var symbolsAmountQuery = from c in stockRatingsTable.AsEnumerable()
                                             select new SymbolInfo { Symbol = c.Field<string>("Symbol"), Market = c.Field<string>("Market") };
                    ratingSymbols = symbolsAmountQuery.AsParallel().ToList();
                }

                for (int i = 0; i < activeSymbolsCount; i++)
                {
                    symbol = ratingSymbols.AsParallel().ElementAtOrDefault(i).Symbol;
                    market = ratingSymbols.AsParallel().ElementAtOrDefault(i).Market;
                    ratingList = new List<StockRating>();

                    using (SqlCommand historicalRatingsCommand = new SqlCommand("select * from dbo.OldStockRatings where Symbol='" + symbol + "' and Market='" + market + "'", connection))
                    using (SqlDataAdapter historicalRatingsAdapter = new SqlDataAdapter(historicalRatingsCommand))
                    using (DataTable historicalRatingsTable = new DataTable("historicalratings"))
                    {
                        // fill the historical ratings table
                        historicalRatingsAdapter.Fill(historicalRatingsTable);

                        if (historicalRatingsTable != null)
                        {
                            completedRowCount = historicalRatingsTable.AsEnumerable().AsParallel().Count();
                            completedDates = historicalRatingsTable.AsEnumerable().AsParallel().Select(d => d.Field<DateTime>("Date")).ToList();
                        }
                    }
                            using (SqlCommand amexCommand = new SqlCommand("select * from dbo.DailyAmexData where Symbol='" + symbol + "'", connection))
                            using (SqlDataAdapter amexAdapter = new SqlDataAdapter(amexCommand))
                            using (DataTable amexTable = new DataTable("amexdata"))
                            {
                                // fill the amex data table
                                amexAdapter.Fill(amexTable);

                                if (amexTable != null)
                                {
                                    var amexFillQuery = from c in amexTable.AsEnumerable()
                                                        select new StockData { Close = c.Field<decimal>("Close"), Date = c.Field<DateTime>("Date"), High = c.Field<decimal>("High"), Low = c.Field<decimal>("Low"), Volume = c.Field<Int64>("Volume") };
                                    amexList = amexFillQuery.AsParallel().ToList();

                                    rowCount = amexList.AsParallel().Count();
                                }
                            }

                    Parallel.For(0, rowCount - 30, new ParallelOptions
                    {
                        MaxDegreeOfParallelism = Environment.ProcessorCount
                    }, async j =>
                    {
                                if (amexList.AsParallel().Count() > 0)
                                {
                                    date = amexList.AsParallel().ElementAtOrDefault(j).Date;
                                    searchDate = date.Subtract(TimeSpan.FromDays(60));

                                    if (completedDates.Contains(date) == false)
                                    {
                                        var amexQuery = from c in sandpInfo
                                                        where c.Date >= searchDate && c.Date <= date
                                                        join d in amexList on c.Date equals d.Date
                                                        select new StockMarketCompare { stockClose = d.Close, marketClose = c.Close };

                                        var amexStockDataQuery = from c in amexList
                                                                 where c.Date >= searchDate && c.Date <= date
                                                                 select new StockData { Close = c.Close, High = c.High, Low = c.Low, Volume = c.Volume, Date = c.Date };

                                        stockCompareData = amexQuery.AsParallel().ToList();
                                        stockData = amexStockDataQuery.AsParallel().ToList();
                                        isGoodToGo = true;
                                    }
                                    else
                                    {
                                        isGoodToGo = false;
                                    }
                                }

                        if (completedDates.Contains(date) == false)
                        {
                            var sandpDateQuery = from c in sandpInfo
                                                 where c.Date >= searchDate && c.Date <= date
                                                 select c;
                            sandpDateInfo = sandpDateQuery.AsParallel().ToList();
                            symbolClass = new SymbolInfo() { Symbol = symbol, Market = market };
                            isGoodToGo = true;
                        }
                        else
                        {
                            isGoodToGo = false;
                        }

                        if (isGoodToGo)
                        {
                            StockRating rating = performCalculations(symbolClass, date, sandpInfo, stockData, stockCompareData);

                                if (rating != null)
                                {
                                    **ratingList.Add(rating);** // getting the exception thrown here
                                }
                        }
                    });

                    // now save the results to the table outside the parallel for loop

                    ratingList.RemoveAll(item => item == null);
                    List<StockRating> masterList = ratingList.DistinctBy(j => j.date).ToList();
                    saveToTable(masterList, symbol, market);
                    // close the connection
                    connection.Close();
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            finally
            {
                // close the connection
                connection.Close();
            }
        }
    }

Answer 1

List<T> is not thread safe and you are calling .Add from inside your Parallel.For. You either need to lock on the Add or use a threadsafe collection in the System.Collections.Concurrent namespace.

This is not your only threading error you have, for example also inside your Parallel.For you assign several variables that are all declared in the scope outside of the loop. Your various threads are going to be writing over each other assigning those values.

Answer 2

ratingsList is not thread safe because List<T> is not guaranteed to be thread safe (except for static methods), yet you are modifying it from multiple threads.

Public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe. It is safe to perform multiple read operations on a List, but issues can occur if the collection is modified while it’s being read. To ensure thread safety, lock the collection during a read or write operation. To enable a collection to be accessed by multiple threads for reading and writing, you must implement your own synchronization. For collections with built-in synchronization, see the classes in the System.Collections.Concurrent namespace. For an inherently thread–safe alternative, see the ImmutableList class.

https://msdn.microsoft.com/en-us/library/6sh2ey19%28v=vs.110%29.aspx

Use a thread safe collection instead.