In lecture 3_c they go trough the the defintion for least squares loss function derived from log of maximum likelyhood. Question 11 is the same formula but now with a probality density p(w) that acts a regularizer when you take the log of it.
So it has variance sigma^2 but here sigma = 1, and the reason why we want that is beacuase as you see in lecture note 3_c we get an 1/2sigma in front of the sum and we just want 1/2 so we put the variance to 1. Makes sense?
The reason why we want the variance of for p(w) to be 1/lambda is for the exact same reason, before the sum over the weights we then get 1/2/lambda = lambda/2 which is exactly what we want! (btw log(p(y|x,w)p(w)) = log(p(y|x,w)) + log(p(w)) the first part is the least squares and second the regulizer.
Hope this clears things up and I advise you to go trough the lecture notes of least likelyhood again to get a bit of intution for how the variance affects it if you still feel a bit confused about it. Good luck!
Pb 11 exam 2018
Hello,
Can you explain why regularized least-squares problem is equivalent to the formula with -log?
Thank you,
In lecture 3_c they go trough the the defintion for least squares loss function derived from log of maximum likelyhood. Question 11 is the same formula but now with a probality density p(w) that acts a regularizer when you take the log of it.
Okay, thank you, but first:
and then:
1
For me, I don't see how the first answer (the one with alpha) is equivalent to the regularized least square?
So it has variance sigma^2 but here sigma = 1, and the reason why we want that is beacuase as you see in lecture note 3_c we get an 1/2sigma in front of the sum and we just want 1/2 so we put the variance to 1. Makes sense?
The reason why we want the variance of for p(w) to be 1/lambda is for the exact same reason, before the sum over the weights we then get 1/2/lambda = lambda/2 which is exactly what we want! (btw log(p(y|x,w)p(w)) = log(p(y|x,w)) + log(p(w)) the first part is the least squares and second the regulizer.
Hope this clears things up and I advise you to go trough the lecture notes of least likelyhood again to get a bit of intution for how the variance affects it if you still feel a bit confused about it. Good luck!
2
Add comment