Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Rescaling after feature scaling, linear regression

Seems like a basic question, but I need to use feature scaling (take each feature value, subtract the mean then divide by the standard deviation) in my implementation of linear regression with gradient descent. After I'm finished, I'd like the weights and regression line rescaled to the original data. I'm only using one feature, plus the y-intercept term. How would I change the weights, after I get them using the scaled data, so that they apply to the original unscaled data?

like image 506
Cartesian Theater Avatar asked Jan 16 '14 17:01

Cartesian Theater


People also ask

Does feature scaling affect linear regression?

Centering/scaling does not affect your statistical inference in regression models — the estimates are adjusted appropriately and the p-values will be the same.

Is linear regression sensitive to feature scaling?

Summary. We need to perform Feature Scaling when we are dealing with Gradient Descent Based algorithms (Linear and Logistic Regression, Neural Network) and Distance-based algorithms (KNN, K-means, SVM) as these are very sensitive to the range of the data points.

What happens if feature scaling is not done?

If feature scaling is not done, then a machine learning algorithm tends to weigh greater values, higher and consider smaller values as the lower values, regardless of the unit of the values.

Why feature scaling is not used in multiple linear regression?

For example, to find the best parameter values of a linear regression model, there is a closed-form solution, called the Normal Equation. If your implementation makes use of that equation, there is no stepwise optimization process, so feature scaling is not necessary.


1 Answers

Suppose your regression is y = W*x + b with x the scaled data, with the original data it is

y = W/std * x0 + b - u/std * W

where u and std are mean value and standard deviation of x0. Yet I don't think you need to transform back the data. Just use the same u and std to scale the new test data.

like image 147
lennon310 Avatar answered Oct 17 '22 22:10

lennon310