Hello,
I have a question concerning the question 9 of the exam. I can't wrap my head around the solution. I agree that h and q shouldn't modify the accuracy of your NN. However, for g shouldn't multiplying the weight by 2 change the accuracy ? I agree that the accuracy would not change if we assume that the sigmoid is centered around the origin, but what if we add a bias ? Or do we simply never add a bias to this kind of NN ?
Thank you in advance
\( w \) or \( 2w \) only make the sigmoid steeper a little. When \( w^Tx \) is positive or negative, \( 2w^Tx \) retains the same sign. Thus, this doesn’t make the \( \sigma(z) \) change the decision.
This is also true when changing the Sigmoid to ReLU using the threshold 1/2 and 0. When the \( w^Tx \) is positive or negative the ReLU becomes that positive number or 0 respectively.
Exam 2021 Q9
Hello,
I have a question concerning the question 9 of the exam. I can't wrap my head around the solution. I agree that h and q shouldn't modify the accuracy of your NN. However, for g shouldn't multiplying the weight by 2 change the accuracy ? I agree that the accuracy would not change if we assume that the sigmoid is centered around the origin, but what if we add a bias ? Or do we simply never add a bias to this kind of NN ?
Thank you in advance
\( w \) or \( 2w \) only make the sigmoid steeper a little. When \( w^Tx \) is positive or negative, \( 2w^Tx \) retains the same sign. Thus, this doesn’t make the \( \sigma(z) \) change the decision.
This is also true when changing the Sigmoid to ReLU using the threshold 1/2 and 0. When the \( w^Tx \) is positive or negative the ReLU becomes that positive number or 0 respectively.
Add comment