Hi,
The two cases comes from the fact that the derivative at point \(w\) of the soft margin cost function is \(-a \mathbf{1}_{w \leq 1 / a }+ \lambda w\). Setting this to zero you need to be careful on how \(\lambda\) and \(a\) compare. In your reasoning this amounts to seeing that in your case (1) you get \(w^* = a / \lambda\), but this can only hold if your initial condition \(a w^* \leq 1\) is indeed true, this leads to \(a^2 / \lambda \leq 1\) which gives the disjunction boundary
Also, for linearly separable data, do the soft svm and the hard svm have the same weights ?, no the solutions of the two in general are not the same, but can be for \( \lambda \) small enough as seen in this exercise.
Exam 2020 Q25
Hi,
My answer is the following :
L(w) = [1-aw]_+ + lambda / 2 ||W||²
1) if aw < 1 <=> w < 1/a :
L(w) = (1-aw) + lambda / 2 ||W||²
L'(w) = -a + lambda * w
L'(w) = 0 <=> w = a / lambda
2) if aw >= 1 <=> w >= 1/a we are minimizing C * w² so we are taking w as small as possible (w = 1/a).
Could you please explain why the two cases are lambda <= a² and lambda > a².
Also, for linearly separable data, do the soft svm and the hard svm have the same weights ?
1
Hi,
The two cases comes from the fact that the derivative at point \(w\) of the soft margin cost function is \(-a \mathbf{1}_{w \leq 1 / a }+ \lambda w\). Setting this to zero you need to be careful on how \(\lambda\) and \(a\) compare. In your reasoning this amounts to seeing that in your case (1) you get \(w^* = a / \lambda\), but this can only hold if your initial condition \(a w^* \leq 1\) is indeed true, this leads to \(a^2 / \lambda \leq 1\) which gives the disjunction boundary
Also, for linearly separable data, do the soft svm and the hard svm have the same weights ?, no the solutions of the two in general are not the same, but can be for \( \lambda \) small enough as seen in this exercise.
3
Add comment