Why is logistic regression called regression? [closed]

Question

According to what I have understood, linear regression predicts the outcome which can have continuous values, whereas logistic regression predicts outcome which is discrete. It seems to me that logistic regression is similar to a classification problem. So, why is it called regression?

There is also a related question: What is the difference between linear regression and logistic regression?

Answer 1

There is a strict link between linear regression and logistic regression.

With linear regression you're looking for the k_i parameters:

h = k₀ + Σ k_i ˙ X_i = K^t ˙ X

With logistic regression you've the same aim but the equation is:

h = g(K^t ˙ X)

Where g is the sigmoid function:

g(w) = 1 / (1 + e^-w)

So:

h = 1 / (1 + e^{-Kt ˙ X})

and you need to fit K to your data.

Assuming a binary classification problem, the output h is the estimated probability that the example x is a positive match in the classification task:

P(Y = 1) = 1 / (1 + e^{-Kt ˙ X})

When the probability is greater than 0.5 then we can predict "a match".

The probability is greater than 0.5 when:

g(w) > 0.5

and this is true when:

w = K^t ˙ X ≥ 0

The hyperplane:

K^t ˙ X = 0

is the decision boundary.

In summary:

• logistic regression is a generalized linear model using the same basic formula of linear regression but it is regressing for the probability of a categorical outcome.

This is a very abridged version. You can find a simple explanation in these videos (third week of Machine Learning by Andrew Ng).

You can also take a look at http://www.holehouse.org/mlclass/06_Logistic_Regression.html for some notes on the lessons.

Answer 2

As explained earlier,logistic regression is a generalized linear model using the same basic formula of linear regression but it is regressing for the probability of a categorical outcome.

As you can see, we get similar type of equation for both linear and logistic regression. Difference lies in fact that linear regression give continous values of y for given x where logistic regression also gives continous values of p(y=1) for given x which is coverted later to y=0 or y=1 based on threshold value(0.5).