Hello,
I am kind of confused by the statement: MSE is convex to the model's output. I totally get why the MSE is not necessary convex for any function as in this question. However, from my understanding, we have always looked at the convexity with respect to the weights and not with regards to the output, no?
Thank you for your help
Right. For optimization, we care about convexity with regards to model parameters.
The solution states that 'MSE is convex with respect to the model output', because if the model output is an affine function of the model weights, the composition of "MSE after model" will be convex. This is an argument why linear least squares regression is a convex problem, for example. Convexity of the loss w.r.t. the model output can be part of an argument.
Exam 2020 Q 18
Hello,
I am kind of confused by the statement: MSE is convex to the model's output. I totally get why the MSE is not necessary convex for any function as in this question. However, from my understanding, we have always looked at the convexity with respect to the weights and not with regards to the output, no?
Thank you for your help
1
Right. For optimization, we care about convexity with regards to model parameters.
The solution states that 'MSE is convex with respect to the model output', because if the model output is an affine function of the model weights, the composition of "MSE after model" will be convex. This is an argument why linear least squares regression is a convex problem, for example. Convexity of the loss w.r.t. the model output can be part of an argument.
Add comment