If we have concrete range of max..min value it is quite easy to normalize it to 0..1 float values, but if we don't have concrete limits? Is it possible to build universal function to have output between 0 and 1? In my mind I think it is impossible but I am not math expert.
I'm searching for implementation on JavaScript or PHP, but any code on C/C++/Python/Delphi is OK to provide examples ( if there are some )
Normalization using sklearn MinMaxScaler In Python, sklearn module provides an object called MinMaxScaler that normalizes the given data using minimum and maximum values. Here fit_tranform method scales the data between 0 and 1 using the MinMaxScaler object.
In statistics, the term “normalization” refers to the scaling down of the data set such that the normalized data falls in the range. Range = maximum value – minimum value read more between 0 and 1.
There are many ways to do this. I'll leave out mapping -inf
and +inf
, which can be done with conditional statements.
exp(x) / (1 + exp(x))
or the equivalent 1 / (1 + exp(-x))
where exp
is the exponential function. This is a logistic function.atan(x) / pi + 1 / 2
(tanh(x) + 1) / 2
(1 + x / sqrt(1 + x*x)) / 2
(1 + x / (1 + abs(x)) / 2
(erf(x) + 1) / 2
You have probably noticed that most of these take a mapping to (-1, 1) and change it to (0, 1). The former is usually easier. Here is a graph of these functions:
In my Python 3.5.2, the fastest was (1 + x / (1 + abs(x)) * 0.5
.
With nearly all programming of floating point numbers, the values are distributed logarithmically. Therefore first take the log()
of the value to begin mapping paying attention to edge case concerns.
double map(double x, double x0, double x1, double y0, double y1) {
return (x - x0) / (x1 - x0) * (y1 - y0) + y0;
}
double noramlize01(double x) {
assert(x == x); // fail is x is NaN
// These values only need to be calculated once.
double logxmin = log(DBL_TRUE_MIN); // e.g. -323.306...
double logxmax = log(DBL_MAX); // e.g. 308.254...
double y;
if (x < -DBL_MAX) y = 0.0;
else if (x < 0.0) {
y = map(log(-x), logxmax, logxmin, nextafter(0.0,1.0), nextafter(0.5,0.0));
} else if (x == 0.0) {
y = 0.5;
} else if (x <= DBL_MAX) {
y = map(log(x), logxmin, logxmax, nextafter(0.5,1.0), nextafter(1.0,0.5));
} else {
y = 1.0;
}
return y;
}
double round_n(double x, unsigned n) {
return x * n;
}
void testr(double x) {
printf("% 20e %#.17g\n", x, noramlize01(x));
//printf("% 20e %.17f\n", -x, noramlize01(-x));
}
int main(void) {
double t[] = {0.0, DBL_TRUE_MIN, DBL_MIN, 1/M_PI, 1/M_E,
1.0, M_E, M_PI, DBL_MAX, INFINITY};
for (unsigned i = sizeof t/sizeof t[0]; i > 0; i--) {
testr(-t[i-1]);
}
for (unsigned i = 0; i < sizeof t/sizeof t[0]; i++) {
testr(t[i]);
}
}
Sample output
-inf 0.0000000000000000
-1.797693e+308 4.9406564584124654e-324
-3.141593e+00 0.24364835649917244
-2.718282e+00 0.24369811843639441
-1.000000e+00 0.24404194470924687
-3.678794e-01 0.24438577098209935
-3.183099e-01 0.24443553291932130
-2.225074e-308 0.48760724499523350
-4.940656e-324 0.49999999999999994
-0.000000e+00 0.50000000000000000
0.000000e+00 0.50000000000000000
4.940656e-324 0.50000000000000011
2.225074e-308 0.51239275500476655
3.183099e-01 0.75556446708067870
3.678794e-01 0.75561422901790065
1.000000e+00 0.75595805529075311
2.718282e+00 0.75630188156360556
3.141593e+00 0.75635164350082751
1.797693e+308 0.99999999999999989
inf 1.0000000000000000
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With