Min-Max DataPoint Normilization

Question

I have a list of DataPoint such as

List<DataPoint> newpoints=new List<DataPoint>(); 

where the DataPoint is a class consists of nine double features from A to I , and the

newpoints.count=100000 double points (i.e each point consists of nine double features from A to I)

I need to apply the normilization for List newpoints using Min-Max normilization method and the scale_range between 0 and 1.

I have implemneted so far the following steps

1. each DataPoints feature is assigned to one dimensional array. for example, the code for feature A

   for (int i = 0; i < newpoints.Count; i++)
       {  array_A[i] = newpoints[i].A;} and so on for all nine double features
   

2. I have applied the max-min normilization method. for example, the code for feature A:

   normilized_featureA= (((array_A[i] - array_A.Min()) * (1 - 0)) / 
                     (array_A.Max() - array_A.Min()))+0;
   

the method is succssfuly done but it takes more time (i.e. 3 minutes and 45 seconds)

how can i apply the Max_min normilization using the LINQ code in C# to reduce my time to a few seconds? I found this question in Stackoverflow How to normalize a list of int values but my problem is

double valueMax = list.Max(); // I need Max point for feature A  for all 100000
double valueMin = list.Min(); //I need Min point for feature A  for all 100000

and so on for all others nine features your help will be highly appreciated.

Answer 1

As an alternative to modelling your 9 features as double properties on a class "DataPoint", you could also model a datapoint of 9 doubles as an array, with the benefit being that you can do all 9 calculations in one pass, again, using LINQ:

var newpoints = new List<double[]>
{
    new []{1.23, 2.34, 3.45, 4.56, 5.67, 6.78, 7.89, 8.90, 9.12},
    new []{2.34, 3.45, 4.56, 5.67, 6.78, 7.89, 8.90, 9.12, 12.23},
    new []{3.45, 4.56, 5.67, 6.78, 7.89, 8.90, 9.12, 12.23, 13.34},
    new []{4.56, 5.67, 6.78, 7.89, 8.90, 9.12, 12.23, 13.34, 15.32}
};

var featureStats = newpoints
// We make the assumption that all 9 data points are present on each row.
.First()
// 2 Anon Projections - first to determine min / max as a function of column
.Select((np, idx) => new
{ 
   Idx = idx,
   Max = newpoints.Max(x => x[idx]),
   Min = newpoints.Min(x => x[idx])
})
// Second to add in the dynamic Range
.Select(x => new {
  x.Idx,
  x.Max,
  x.Min,
  Range = x.Max - x.Min
})
// Back to array for O(1) lookups.
.ToArray();

// Do the normalizaton for the columns, for each row.
var normalizedFeatures = newpoints
   .Select(np => np.Select(
      (i, idx) => (i - featureStats[idx].Min) / featureStats[idx].Range));

foreach(var datapoint in normalizedFeatures)
{
  Console.WriteLine(string.Join(",", datapoint.Select(x => x.ToString("0.00"))));
}

Result:

0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
0.33,0.33,0.33,0.33,0.34,0.47,0.23,0.05,0.50
0.67,0.67,0.67,0.67,0.69,0.91,0.28,0.75,0.68
1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00