Hello I can't figure out how to find the same gradients derived in lab 13, So i figured it would be one of the following options :
Did I do an error in my derivation ?
Or is there a missing 1/2 in the loss function provided in the lab ? ( in the course notes there's a 1/2 term but not in the notebook)
Or is there an error in the derivative of the loss function in the solution ? Maybe the 2 was just factorized and just considered part of the learning rate ?
I think the 1/2 factor indeed should be present in the original objective. Otherwise one immediately gets (as in your derivations) the 2 factor in the derivative/gradient due to the derivative of the square function.
lab 13 gradients of Loss wrt to w and u
Hello I can't figure out how to find the same gradients derived in lab 13, So i figured it would be one of the following options :
Did I do an error in my derivation ?
Or is there a missing 1/2 in the loss function provided in the lab ? ( in the course notes there's a 1/2 term but not in the notebook)
Or is there an error in the derivative of the loss function in the solution ? Maybe the 2 was just factorized and just considered part of the learning rate ?
Thank you for your time,
I think the 1/2 factor indeed should be present in the original objective. Otherwise one immediately gets (as in your derivations) the 2 factor in the derivative/gradient due to the derivative of the square function.
1
Okay, thank you for the clarification!
Add comment