Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I happen to do a lot of linear regression, the typical use case is predicting $Y$ with a few predictors say $X = [X_1;X_2;X_3]$. So we we find $\hat{\beta}$ such that $$\hat{\beta} = \underset{\beta}{argmin} ||Y-X.\beta||^2$$ But actually what I am more interested in in finding the linear estimation of $Y$ such that $$corr(Y,\hat{Y}) = corr(Y,X.\hat{\beta})$$ is maximal.

I am bothered by the fact that for a given $\beta_0$, $corr(Y,X.a.\beta_0)$ is the same for all $a$, questions are then

Is the optimal linear estimator of $\beta$ in the least square sense also optimal in my correlation term, and if yes should I be bothered that the optimal $\beta$ is not unique ?

share|improve this question

1 Answer 1

Think about $R^2$ of linear regression:

$R^2=1-\frac{\|Y-X\hat\beta_{OLS}\|^2}{\|Y-\bar Y\|^2}=[Corr(Y,X\hat\beta_{OLS})]^2$

Since denominator does not depend on $\hat\beta$ and LS minimizes the numerator the "result follows".


You correctly mentioned that the correlation coefficient is invariant under linear transformations (which preserve sign). It means that you cannot use correlation to recover the underlying parameters.

Consider the next univariate model:

$Y_i=X_i\beta+u_i$

For any $b\not=0, |Corr(Y_i. X_ib)|=|Corr(Y_i, X_i)|$

So, it meaningless to maximize sample covariance w.r.t to $b$.

On the other hand, $R^2(b)\le Corr(Y_i, X_ib)^2$ with equality for $b=b_{OLS}$. In this sense LS yields the optimal estimator.

share|improve this answer
    
good that's what I thought, did not know the correl formula. Any thought about the second question (should I be bothered that an infinite number of $\beta$ maximize the correl) ? –  statquant Apr 10 at 9:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.