I have a hard time seeing how we derive the constraints for alpha in the dual formulation of SVM in lecture 7 . And is alpha 0 or 1 or can it also be a value in between? Does this condition change or stay the same if we enforce the SVM to be hard margin?
Secondly,I have a hard time seeing how to use the information from the lecture to calculate the closed form in coordinate descent in the lab.
"I have a hard time seeing how we derive the constraints for alpha in the dual formulation of SVM in lecture 7. " Can you be more precise on what exactly is causing you problems ? the constraints on \( \alpha \) are given by noticing the trick \( \max \{ 0, z \}=\max _{\alpha \in[0,1]} \alpha z \).
And is alpha 0 or 1 or can it also be a value in between?
The value of \( \alpha \) can be any number between 0 and 1. As seen in lecture 7a, if \( \alpha_n = 0 \) the feature \(x_n\) is on the correct side of the margin, if \( \alpha_n \in (0, 1) \) then \( x_n \) is exactly on the (correct) margin, and if \( \alpha_n = 1 \) then \(x_n\) is inside the margin, or on the wrong side.
Does this condition change or stay the same if we enforce the SVM to be hard margin?
You have not seen in the lectures the dual formulation for the hard margin problem, but the constraints on alpha will be different from the soft margin framework . Also note that the hard margin problem only makes sense for linearly separable data. If you assume that the data is indeed linearly separable, then for \( \lambda \) small enough (small enough so that we recover the hard margin solution, see lecture for more details) then all the \( \alpha_n \) will be either 0 or in \( (0, 1) \) since in this case no data points are wrongly labeled or inside the margin.
Secondly,I have a hard time seeing how to use the information from the lecture to calculate the closed form in coordinate descent in the lab.
This seems to have posed some problems to many of you, I will shortly release some hints that should help you.
Let me know it my explanations were clear (or not !) ,
soft vs hard margin dual and alpha
Good Evening,
I have a hard time seeing how we derive the constraints for alpha in the dual formulation of SVM in lecture 7 . And is alpha 0 or 1 or can it also be a value in between? Does this condition change or stay the same if we enforce the SVM to be hard margin?
Secondly,I have a hard time seeing how to use the information from the lecture to calculate the closed form in coordinate descent in the lab.
Thanks!
Hello,
"I have a hard time seeing how we derive the constraints for alpha in the dual formulation of SVM in lecture 7. " Can you be more precise on what exactly is causing you problems ? the constraints on \( \alpha \) are given by noticing the trick \( \max \{ 0, z \}=\max _{\alpha \in[0,1]} \alpha z \).
And is alpha 0 or 1 or can it also be a value in between?
The value of \( \alpha \) can be any number between 0 and 1. As seen in lecture 7a, if \( \alpha_n = 0 \) the feature \(x_n\) is on the correct side of the margin, if \( \alpha_n \in (0, 1) \) then \( x_n \) is exactly on the (correct) margin, and if \( \alpha_n = 1 \) then \(x_n\) is inside the margin, or on the wrong side.
Does this condition change or stay the same if we enforce the SVM to be hard margin?
You have not seen in the lectures the dual formulation for the hard margin problem, but the constraints on alpha will be different from the soft margin framework . Also note that the hard margin problem only makes sense for linearly separable data. If you assume that the data is indeed linearly separable, then for \( \lambda \) small enough (small enough so that we recover the hard margin solution, see lecture for more details) then all the \( \alpha_n \) will be either 0 or in \( (0, 1) \) since in this case no data points are wrongly labeled or inside the margin.
Secondly,I have a hard time seeing how to use the information from the lecture to calculate the closed form in coordinate descent in the lab.
This seems to have posed some problems to many of you, I will shortly release some hints that should help you.
Let me know it my explanations were clear (or not !) ,
Scott
4
Add comment