Great answer by Leevo, just let me point out one thing: Perfect multicollinearity means that one variable is a **linear combination** of another. Say you have $x_1$ and $x_2$, where $x_2 = \gamma x_1$. This causes various problems as discussed in this post.

The main takeway (to put it simple) is, that $x_1$ and $x_2$ basically carry the same information (just "scaled" by $\gamma$ in the case of $x_1$). So there is no benefit of including both. In fact there is a problem with including both since multicollinearity will "confuse" the model because there is no unique effect of $x_1, x_2$, when considered jointly, on some outcome $y$.

Look the following situation (R code):

```
y = c(5,2,9,10)
x1 = c(2,4,6,8) ### = 2 * x2
x2 = c(1,2,3,4) ### = 0.5 * x1
cor(x1, x2, method = c("pearson"))
```

The correlation between $x_1$ and $x_2$ equals 1 (so of course a linear combination). Now when I try to make a simple linear OLS regression:

```
lm(y~x1+x2)
```

The result is:

```
Coefficients:
(Intercept) x1 x2
1.0 1.1 NA
```

The second term has been dropped by `R`

(due to perfect multicollinearity).

We can run a regression on each term separately:

```
Call:
lm(formula = y ~ x1)
Coefficients:
(Intercept) x1
1.0 1.1
```

...and...

```
Call:
lm(formula = y ~ x2)
Coefficients:
(Intercept) x2
1.0 2.2
```

Now you can see that the coefficient for $\beta_2$ is simply $2\beta_1$ because $x_1$ is $2 x_2$. So nothing to learn from including both, $x_1, x_2$ since there is no additional information.

Basically the same problem can occur if correlation between $x_1,x_2$ is really high. See some more discussion in this post. Thus given strong correlation, one should be cautious to include both variables. The reason is that in this case, your model cannot really tell apart the effect of $x_1$ and $x_2$ on some outcome $y$, so that you may end up with weak predictions (among other problems).

Thanks, Leevo. About the last point, I am still confused. Do you mean that collinearity gives a quantifiable measurement to the change in one variable which is brought by a uit in change in another variable. – Payal Bhatia – 2019-08-07T06:37:25.033

To my understanding, multicollinearity happens when the variables are "way too similar". That's probably an excess of correlation, to such an extent that the parameters of your regressions are tilted and not reliable enough. – Leevo – 2019-08-07T06:39:49.410

Leevo, can you site an example or ellaborate the point where you say

`Two variables can be highly correlated but their scatterplot can be very "spread out"`

? – GadaaDhaariGeek – 2019-08-21T11:00:31.940