I don't understand at the end of the lecture about GMM why you said that the model is non-identifiable because it has non-unique optima ? I thought that a model was identifiable if f(theta_1) = f(theta_2) <-> theta_1 = theta_2, and I don't understand the link with the number of optima here.
In addition, I didn't understand how you prove the fact that it has non-unique optima by permutating the k ?
Thank you for your clarifications !
Identifiable is exactly what you wrote, one model per parameter.
In the case of GMM and more generally models with clusters, permuting the clusters (with their related parameters) does not change your loss. Basically, the name that you give to the cluster is not important. Using this fact, you can see that if you have an optimum, by permuting the clusters you construct a different optimum and thus why you shouldn't/can't have a unique optimum in this case.
Identifiable model <-> non unique optima ?
Hello,
I don't understand at the end of the lecture about GMM why you said that the model is non-identifiable because it has non-unique optima ? I thought that a model was identifiable if f(theta_1) = f(theta_2) <-> theta_1 = theta_2, and I don't understand the link with the number of optima here.
In addition, I didn't understand how you prove the fact that it has non-unique optima by permutating the k ?
Thank you for your clarifications !
Identifiable is exactly what you wrote, one model per parameter.
In the case of GMM and more generally models with clusters, permuting the clusters (with their related parameters) does not change your loss. Basically, the name that you give to the cluster is not important. Using this fact, you can see that if you have an optimum, by permuting the clusters you construct a different optimum and thus why you shouldn't/can't have a unique optimum in this case.
Add comment