I have a question regarding the solution of the problem set 10. Here :
The cauchy schwartz inequality states that : \(|<u,v>| \leq ||u|| ||v||\)
I am a bit confused with the solution's inequality, maybe it's trivial but I didn't completely get it.. If anyone can help
|<u,v>| <= ||u|| ||v|| equivalently implies that if we remove the absolute value then we get the following two inequalities:
-||u|| ||v|| <= <u,v> <= ||u|| ||v||
in particular, that <u,v> >= -||u|| ||v|| as in the exercise solution.
Indeed, you are right that y_i w^t delta is not necessary positive, but we don't make any assumption on that.
Hello,
I have question regarding adversarial example with L-infinity norm. How have we obtained sign(w) in expression for delta*? Shouldn't it be like this w/||w||_1? But how then did we derive sign(w) from that?
Thank you in advance!
One can verify that if delta*=-w/||w||_1 (which also has to be divided over max_i |w_i|/||w||_1 then to meet the Linf constraint on delta), then the objective value would be suboptimal compared to what is given by delta*=-eps*sign(w).
In order to derive the expression for delta*, we need to know when the equality is attained in the Hölder's inequality. For the general conditions, see e.g. http://www.math.ubc.ca/~feldman/m421/ineq.pdf.
However, here we have a particularly easy case. We need to know when:
<delta, w> = - eps ||w||_1
which means that:
sum_i delta_i * w_i = - sum_i eps * |w_i|.
So we can select the delta* such that each term in both sums is equal. This can be achieved if delta_i = - eps * sign(w_i) for each i. This solution is not necessarily unique as, e.g., if w_i=0, then we can set any delta_i. But we are not interested in uniqueness, we just need to find some delta* that would lead to the optimal objective value. This can be always achieved when we set delta = -eps * sign(w).
Clarification adversarial example
Hi,
I have a question regarding the solution of the problem set 10. Here :
The cauchy schwartz inequality states that :
\(|<u,v>| \leq ||u|| ||v||\)
I am a bit confused with the solution's inequality, maybe it's trivial but I didn't completely get it.. If anyone can help
Thanks !
What I mean here is that in my opinion we cannot be sure that \(y_{i}w^{t}\delta\) is positive.. ( \(y_{i} \in {-1, 1}\) and \(w \in R^{d}\)
Hi,
|<u,v>| <= ||u|| ||v|| equivalently implies that if we remove the absolute value then we get the following two inequalities:
-||u|| ||v|| <= <u,v> <= ||u|| ||v||
in particular, that <u,v> >= -||u|| ||v|| as in the exercise solution.
Indeed, you are right that y_i w^t delta is not necessary positive, but we don't make any assumption on that.
I hope that helps.
1
It's perfectly clear ! Thank you very much
Hello,
I have question regarding adversarial example with L-infinity norm. How have we obtained sign(w) in expression for delta*? Shouldn't it be like this w/||w||_1? But how then did we derive sign(w) from that?
Thank you in advance!
1
Hi,
One can verify that if delta*=-w/||w||_1 (which also has to be divided over max_i |w_i|/||w||_1 then to meet the Linf constraint on delta), then the objective value would be suboptimal compared to what is given by delta*=-eps*sign(w).
In order to derive the expression for delta*, we need to know when the equality is attained in the Hölder's inequality. For the general conditions, see e.g. http://www.math.ubc.ca/~feldman/m421/ineq.pdf.
However, here we have a particularly easy case. We need to know when:
<delta, w> = - eps ||w||_1
which means that:
sum_i delta_i * w_i = - sum_i eps * |w_i|.
So we can select the delta* such that each term in both sums is equal. This can be achieved if delta_i = - eps * sign(w_i) for each i. This solution is not necessarily unique as, e.g., if w_i=0, then we can set any delta_i. But we are not interested in uniqueness, we just need to find some delta* that would lead to the optimal objective value. This can be always achieved when we set delta = -eps * sign(w).
I hope that helps.
Add comment