The true error corresponds to the expected error achieved on fresh samples from the (infinitely large) data distribution. If your model is powerful enough, you can overfit on the training set, and get zero training loss (thus it can go below the true error as it is measured on different data)
The larger your training dataset, the harder it is to overfit on them.
Pb 22 exam 2017
Hello,
For this problem, I agree with the test error for the 5k and 100k data sets. But I do not understand the following:
Thanks for your help.
Hi,
The true error corresponds to the expected error achieved on fresh samples from the (infinitely large) data distribution. If your model is powerful enough, you can overfit on the training set, and get zero training loss (thus it can go below the true error as it is measured on different data)
The larger your training dataset, the harder it is to overfit on them.
3
Add comment