How to derive the slope of an optimal linear regression?
Hello,
How is alpha found for this problem? I understand that
\alpha = cov(x,y)/var(x)
I achieve something similar calculating the correlation coefficient but cannot quite get the same.
It is implied that the loss function wrt w is \( \frac{1}{N} \Sigma(y_n-\alpha x_n)^2 \).
Deriving L wrt to alpha and setting the derivative to zero, we get \(dL/d\alpha= 0 = \frac{1}{N}\Sigma(-x_n)*2*(y_n-\alpha x_n) = \frac{2}{N} \Sigma (x_n^2 \alpha - x_n y_n) \) and solving for alpha (getting rid of 2 but keeping 1/N) we obtain \( \alpha = \frac{ \frac{1}{N} \Sigma x_n y_n }{\frac{1}{N} \Sigma x_n^2} = \frac{E[XY]}{E[X^2]} \).
However if anyone knows what the link with correlation between X and Y is, please share!
EDIT: I think the link is that \(corr(X,Y) = \frac{E[(X-\mu_x)(Y-\mu_y)]}{\sqrt{E[X^2]-E[X]^2}\sqrt{E[Y^2]-E[Y]^2}} \) which implies that \( \alpha = corr(X,Y) E[Y^2] \) and linear regression has sense only if \(\alpha \neq 0\), implying \(corr(X,Y) \neq 0 \) as well.
How to derive the slope of an optimal linear regression?
Hello,
How is alpha found for this problem? I understand that
\alpha = cov(x,y)/var(x)
I achieve something similar calculating the correlation coefficient but cannot quite get the same.
Thank you in advance,
4
Hello Sélène,
Could you explain a bit more in detail why \alpha is cov(x,y)/var(x)?
Cov(E,Y) = E(X,Y) - E(X)E(Y) = Cov(X,Y) since x and y are 0 mean here
Var(X) E(X^2)-E(X)^2 = E(X^2)
But I don't know if this is the path to take, because I don't get to the solution
It is implied that the loss function wrt w is \( \frac{1}{N} \Sigma(y_n-\alpha x_n)^2 \).
Deriving L wrt to alpha and setting the derivative to zero, we get \(dL/d\alpha= 0 = \frac{1}{N}\Sigma(-x_n)*2*(y_n-\alpha x_n) = \frac{2}{N} \Sigma (x_n^2 \alpha - x_n y_n) \) and solving for alpha (getting rid of 2 but keeping 1/N) we obtain \( \alpha = \frac{ \frac{1}{N} \Sigma x_n y_n }{\frac{1}{N} \Sigma x_n^2} = \frac{E[XY]}{E[X^2]} \).
However if anyone knows what the link with correlation between X and Y is, please share!
EDIT: I think the link is that \(corr(X,Y) = \frac{E[(X-\mu_x)(Y-\mu_y)]}{\sqrt{E[X^2]-E[X]^2}\sqrt{E[Y^2]-E[Y]^2}} \) which implies that \( \alpha = corr(X,Y) E[Y^2] \) and linear regression has sense only if \(\alpha \neq 0\), implying \(corr(X,Y) \neq 0 \) as well.
4
(edit)
Add comment