I have a question for the last exercise serie problem 2.1. I have been able to get the following equation for the adverserial problem like the solutionary, \( min_w \frac{1}{n} \sum_{i=1}^n l(y \cdot w^t x - \epsilon ||w||_2) \)
however I am not sure how to get \(\delta^*\) and \(\hat{x}^*\) from this. Sorry if it is straightforward, however i just do not see it - and also do not understand in the solutionary which equality is referred to in order to obtain it!
Thanks a lot for the question. I think this reference can be helpful: https://math.berkeley.edu/~arash/54/notes/6_7.pdf
In particular Theorem 16 therein that discusses when do we have equality in the Cauchy-Schwartz inequality: when vector \(v\) is a scalar multiple of \(w\) or vice versa. Applying this to our notation from lab 10 it would mean that we need to select our adversarial perturbation \(\delta\) to be collinear with the weight vector, i.e. \(\delta = \alpha w\) where \(\alpha\) is an appropriately chosen scalar to achieve the equality. In our case, by solving wrt \(\alpha\) one gets \(\alpha = - \frac{y_i \varepsilon}{||w||_2}\) which recovers the desired {\delta^\star).
I still don't completely understand how to solve with respect to α. I'm guessing we could use the property in equation 1 (from the Berkeley notes) but I don't understand what <u,v> is. For this example to work, <u,v> would have to be ε. Is that correct? Could you explain in detail how to solve with respect to α? Thank you so much in advance :))
The notation \(\langle u, v\rangle\) denotes the inner product between vectors \(u\), \(v\). For usual Euclidean spaces, it's just \(\langle u, v\rangle = u^T v = \sum_{i=1}^d u_i v_i\).
For this example, it should hold that \(y_i \langle \delta, w \rangle = -\epsilon ||w||_2\). But then we also know that \(\delta^* = \alpha w\), thus we get that \(y_i \langle \alpha w, w \rangle = -\epsilon ||w||_2\) or equivalently
Serie 10 Problem 2.1 Retrieving delta*
Hi,
I have a question for the last exercise serie problem 2.1. I have been able to get the following equation for the adverserial problem like the solutionary,
\( min_w \frac{1}{n} \sum_{i=1}^n l(y \cdot w^t x - \epsilon ||w||_2) \)
however I am not sure how to get \(\delta^*\) and \(\hat{x}^*\) from this. Sorry if it is straightforward, however i just do not see it - and also do not understand in the solutionary which equality is referred to in order to obtain it!
Many thanks
1
Hi,
Thanks a lot for the question. I think this reference can be helpful:
https://math.berkeley.edu/~arash/54/notes/6_7.pdf
In particular Theorem 16 therein that discusses when do we have equality in the Cauchy-Schwartz inequality: when vector \(v\) is a scalar multiple of \(w\) or vice versa. Applying this to our notation from lab 10 it would mean that we need to select our adversarial perturbation \(\delta\) to be collinear with the weight vector, i.e. \(\delta = \alpha w\) where \(\alpha\) is an appropriately chosen scalar to achieve the equality. In our case, by solving wrt \(\alpha\) one gets \(\alpha = - \frac{y_i \varepsilon}{||w||_2}\) which recovers the desired {\delta^\star).
I hope that helps.
1
Thank you very much! Have a great weekend
1
I still don't completely understand how to solve with respect to α. I'm guessing we could use the property in equation 1 (from the Berkeley notes) but I don't understand what <u,v> is. For this example to work, <u,v> would have to be ε. Is that correct? Could you explain in detail how to solve with respect to α? Thank you so much in advance :))
The notation \(\langle u, v\rangle\) denotes the inner product between vectors \(u\), \(v\). For usual Euclidean spaces, it's just \(\langle u, v\rangle = u^T v = \sum_{i=1}^d u_i v_i\).
For this example, it should hold that \(y_i \langle \delta, w \rangle = -\epsilon ||w||_2\). But then we also know that \(\delta^* = \alpha w\), thus we get that \(y_i \langle \alpha w, w \rangle = -\epsilon ||w||_2\) or equivalently
$$\alpha = -\frac{\epsilon ||w||_2}{y_i \langle w, w \rangle} = -\frac{\epsilon ||w||_2}{y_i ||w||_2^2} = -\frac{\epsilon}{y_i ||w||_2} = -\frac{y_i \epsilon}{||w||_2},$$
where we used that \(\langle w, w \rangle = ||w||_2^2\) and \(1/y_i = y_i\) since \(y_i \in \{-1, 1\}\).
I hope that helps.
2
Add comment