About the final submission, it is mentioned that the run.py should get the same best prediction percentages we got. Does this mean this only uses our best overall (i.e. with Ridge method), or our best for each method tested? Also do you want us to re-do the crossvalidation, and output any of the plots we used? Since I know the settings and everything, can the run.py just a script that outputs a prediction.csv file, i.e., my run.py script can just be something like this :
For clarification, I can re-run the entire process (cross-validation included) just like I did to get the same sets of parameters. I just wanted to know if running speed mattered, and whether you want the whole thing or just something that gets the results (i.e. no need to run for some time to get some weights I already have, also because said scripts I wrote to find parameters are included in the submission folder)
the run.py should reproduce the score you get on the AIcrowd submission.
so make sure you fix the random seed. for cross validation you can fix its outcomes (as you suggest) and provide the crossvalidation in a separate code-file, or you can include it in run.py (mention running time in the readme for convenience). either is fine but it needs reproducibility.
this is independent of the first part where you implement the individual methods. i just sent an email to everyone with a submission checklist, hope it clarifies a bit.
Hello,
Should we also include code and Jupiter notebooks where we worked on multiple methods until finding the best one ?
Also, in the run.py, how should we proceed to recreate our results if for example we worked with ridge regression and found some best weights ? Should we manually save the weights and use them to redo our prediction ? That begs the first question answer, as this doesn't show at all how we coded -> Jupiter notebooks are really how much work we did, step by step. So should we copy some of the code and steps from Jupiter notebooks into python files, and reference them in the report if you just want python files ? Or in general how will you evaluate the code ? Will you look if we have multiple Jupiter notebooks for different methods ?
Final submission
Hello,
About the final submission, it is mentioned that the run.py should get the same best prediction percentages we got. Does this mean this only uses our best overall (i.e. with Ridge method), or our best for each method tested? Also do you want us to re-do the crossvalidation, and output any of the plots we used? Since I know the settings and everything, can the run.py just a script that outputs a prediction.csv file, i.e., my run.py script can just be something like this :
from Whatever import *
process_data(known settings)
weights = ridge_regression(known settings)
getting_prediction (...)
...
For clarification, I can re-run the entire process (cross-validation included) just like I did to get the same sets of parameters. I just wanted to know if running speed mattered, and whether you want the whole thing or just something that gets the results (i.e. no need to run for some time to get some weights I already have, also because said scripts I wrote to find parameters are included in the submission folder)
1
the
run.pyshould reproduce the score you get on the AIcrowd submission.so make sure you fix the random seed. for cross validation you can fix its outcomes (as you suggest) and provide the crossvalidation in a separate code-file, or you can include it in
run.py(mention running time in the readme for convenience). either is fine but it needs reproducibility.this is independent of the first part where you implement the individual methods. i just sent an email to everyone with a submission checklist, hope it clarifies a bit.
1
There is no "Project 1" topic, could you please confirm we should select the "mini-project optML course" topic?
Hello,
Should we also include code and Jupiter notebooks where we worked on multiple methods until finding the best one ?
Also, in the run.py, how should we proceed to recreate our results if for example we worked with ridge regression and found some best weights ? Should we manually save the weights and use them to redo our prediction ? That begs the first question answer, as this doesn't show at all how we coded -> Jupiter notebooks are really how much work we did, step by step. So should we copy some of the code and steps from Jupiter notebooks into python files, and reference them in the report if you just want python files ? Or in general how will you evaluate the code ? Will you look if we have multiple Jupiter notebooks for different methods ?
Also, in the run.py where should we import the data from ? Will it be in the same directory as the file ?
you have to specify in the readme where the data needs to go (and put an understandable error message in the code if it doesn't find it)
Add comment