Hi,
I have a question in Exercise 7:
Why should the gap between primal and dual objectives reach 0 (or a value close to zero)?
I understand we want to minimize the primal objective (using the parameter \(w \in \mathcal{R}^D\)) and maximize the dual objective (using the parameter \(\alpha \in \mathcal{R}^N\)), and that the two parameters are related via the equation \(w = \frac{1}{\lambda}X^TY\alpha\).
But what is the relation between primal and dual objectives in terms of magnitude? (i.e. why should they converge to the same value?).
In general the dual objective is a lower bound of the primal, this is what is called weak duallity and is always true. In some conditions we have strong duality : optimal dual= optimal primal. These conditions are true for SVM (in general strong duality holds if your objective is convex and you have linear constraints).
Ex7 - Primal and Dual Objectives
Hi,
I have a question in Exercise 7:
Why should the gap between primal and dual objectives reach 0 (or a value close to zero)?
I understand we want to minimize the primal objective (using the parameter \(w \in \mathcal{R}^D\)) and maximize the dual objective (using the parameter \(\alpha \in \mathcal{R}^N\)), and that the two parameters are related via the equation \(w = \frac{1}{\lambda}X^TY\alpha\).
But what is the relation between primal and dual objectives in terms of magnitude? (i.e. why should they converge to the same value?).
Thank you!
In general the dual objective is a lower bound of the primal, this is what is called weak duallity and is always true. In some conditions we have strong duality : optimal dual= optimal primal. These conditions are true for SVM (in general strong duality holds if your objective is convex and you have linear constraints).
Thank you. It is much clearer now!
Add comment