this is not possible in general. it only can be done in very specific cases, such as the 1-parameter example with squared loss we have seen, where we can write down exactly that the trajectories will be. in practical ML, you have to empirically try different gammas=stepsizes, as those are crucial for the overall training performance
GD: gammas that guarantee convergence
Can you explain further in detail how to calculate/compute the range of gammas that guarantee convergence?
this is not possible in general. it only can be done in very specific cases, such as the 1-parameter example with squared loss we have seen, where we can write down exactly that the trajectories will be. in practical ML, you have to empirically try different gammas=stepsizes, as those are crucial for the overall training performance
1
Add comment