Hi,
When D>N, then there are many solutions to the problem. Therefore the problem is not well-defined. However, regularization shrinks the space of possible solutions. If you set it properly, then the solution can even be a unique point, as in w_star in the figure.
Regression and over-parametrized systems
Good afternoon,
Could you please explain why we say that regularization "helps" with over-parametrized systems. I don't see the link...
Will we penalize the extra parameters so much that they will have to be zero ? (isn't the "regulator" lambda a constant)
Thanks
Hi,
When D>N, then there are many solutions to the problem. Therefore the problem is not well-defined. However, regularization shrinks the space of possible solutions. If you set it properly, then the solution can even be a unique point, as in w_star in the figure.
3
Super clear, thanks !
Add comment