Lecture 7b Proof that k(x,x') = k1(x,x') * k2(x,x')
Hi,
I didn't fully understand the proof. I am trying to do my own proof but I don't get to exactly the same result.
k1(x,x') can be decomposed as a(x).T . a(x') where a(x) can be of dimension d1. I prefer not to use a phi notation yet.
k2(x,x') can be decomposed as b(x).T . b(x') where b(x) can be of dimension d2
k(x,x') = k1(x,x') . k2(x,x')
<=>
k(x,x') = a(x).T @ a(x') . b(x).T @ b(x')
k(x,x') = sum_i(a(x)_i . a(x')_i) . sum_j(b(x)_j . b(x')_j) Note : it's the sum from i = 1 to d1 and from j = 1 to d2
k(x,x') = sum_i,j( [a(x)_i . b(x)_j] . [a(x')_i . b(x')_j] ) Note : I just regroup the terms concerning x and x' in the same parenthesis
k(x,x') = sum_i,j( phi(x)_i,j . phi(x')_i,j) Note : Here I introduce the phi matrix that was presented in class
where phi(x)_i,j = a(x)_i . b(x)_j
k(x, x') = sum(phi(x).T @ phi(x))
I don't know how can we get to
k(x,x') = phi(x).T @ phi(x)
considering that k(x,x') should be a scalar and phi(x) is a matrix of dimensions d1 . d2 then phi(x).T @ phi(x) is a matrix of dimension d2 . d2.
Lecture 7b Proof that k(x,x') = k1(x,x') * k2(x,x')
Hi,
I didn't fully understand the proof. I am trying to do my own proof but I don't get to exactly the same result.
k1(x,x') can be decomposed as a(x).T . a(x') where a(x) can be of dimension d1. I prefer not to use a phi notation yet.
k2(x,x') can be decomposed as b(x).T . b(x') where b(x) can be of dimension d2
k(x,x') = k1(x,x') . k2(x,x')
<=>
k(x,x') = a(x).T @ a(x') . b(x).T @ b(x')
k(x,x') = sum_i(a(x)_i . a(x')_i) . sum_j(b(x)_j . b(x')_j) Note : it's the sum from i = 1 to d1 and from j = 1 to d2
k(x,x') = sum_i,j( [a(x)_i . b(x)_j] . [a(x')_i . b(x')_j] ) Note : I just regroup the terms concerning x and x' in the same parenthesis
k(x,x') = sum_i,j( phi(x)_i,j . phi(x')_i,j) Note : Here I introduce the phi matrix that was presented in class
where phi(x)_i,j = a(x)_i . b(x)_j
k(x, x') = sum(phi(x).T @ phi(x))
I don't know how can we get to
k(x,x') = phi(x).T @ phi(x)
considering that k(x,x') should be a scalar and phi(x) is a matrix of dimensions d1 . d2 then phi(x).T @ phi(x) is a matrix of dimension d2 . d2.
Best regards,
Ali
Hint: a quick workaround is to reshape your \(\phi(x)\) into a vector. For a matrix \(A\) of size \(m\) by \(n\), define
$$\mathrm{vec}(A) = (A_{11}, A_{12},..., A_{1n}, A_{21}, ..., A_{2n}, ..., A_{m1}, ..., A_{mn})$$
a vector of length \(mn\).
1
Thank you for your answer.
I also thought of using a vectorization for phi however it would be different from what is on the slides.
They are actually the same :)
1
Oh alright thank you I just noticed that. It is more clear on the poly. On the slides phi(x) looks like a R^d1 vector.
1
Thank you Ali for your feedback. I have updated the slide. It should be more clear now.
1
Add comment