I have a question about the grid search implementation on the exercise sheet. In the sheet it's written that we have to play with a parameter "grid spacing". Actually when you look at the grid_search.py code, the parameter "num_interval" in the function generate_w corresponds to the third parameter of the function np.linspace which corresponds to the number of points and not the spacing as mentioned in the sheet. And by defaut this parameter is set to 10 in the notebook and when we increase it to 50, we have more points so, more accurate fit. But shouldn't we implement a larger spacing to see that the fit is getting worse when we increase the spacing ?
Yes, you're right. I also noticed that it is a bit confusing. Changing spacing from 50 to 10 is like adding more points to linspace, so here we need to increase the num_interval argument.
Yes, in the sheet this should have read 'Change the num_intervals to 50 instead of 10 to reduce the grid spacing'. The idea was precisely that you will see a better fit than what you see initially (but at a cost, of course). You could definitely do an experiment in the other direction as well (increase grid spacing / reduce num_intervals) and see if you get a worse fit.
Hi !
No problem :) Below are my results.. Even though our curves look almost the same, we don't have the same range for the loss values and I don't know why. It should be a mistake from my side.
Best
Quick question - Grid Search
Hi there !
I have a question about the grid search implementation on the exercise sheet. In the sheet it's written that we have to play with a parameter "grid spacing". Actually when you look at the grid_search.py code, the parameter "num_interval" in the function generate_w corresponds to the third parameter of the function np.linspace which corresponds to the number of points and not the spacing as mentioned in the sheet. And by defaut this parameter is set to 10 in the notebook and when we increase it to 50, we have more points so, more accurate fit. But shouldn't we implement a larger spacing to see that the fit is getting worse when we increase the spacing ?
Best
1
Yes, you're right. I also noticed that it is a bit confusing. Changing spacing from 50 to 10 is like adding more points to
linspace, so here we need to increase thenum_intervalargument.2
Yes, in the sheet this should have read 'Change the num_intervals to 50 instead of 10 to reduce the grid spacing'. The idea was precisely that you will see a better fit than what you see initially (but at a cost, of course). You could definitely do an experiment in the other direction as well (increase grid spacing / reduce num_intervals) and see if you get a worse fit.
2
@younes would love to compare results of both experiments :)
This is what I got from increasing the number of intervals:
2
Hi !
No problem :) Below are my results.. Even though our curves look almost the same, we don't have the same range for the loss values and I don't know why. It should be a mistake from my side.
Best
1
I have the same range as well.
edit : I haven't divided by N the loss. So that's probably the first thing that differs from
Add comment