Could a neural network, a knn or a ridge regression with a kernel trick solve the XOR problem ?
Is it possible to have this kind of decision boundary with a kernel trick ?
I guess that yes since a neural network could do it and a neural network performs feature extraction but I would like to confirm.
Finally, will the ridge regression and the kernelized ridge regression with a linear kernel that we have seen in class always give the same decision boundary ?
NN yes, I think you've seen in the course. Knn I would say yes, but it does not make a lot of sense to solve the XoR gate. Plain ridge regression, No, this was one of the reasons why polynomial feature augmentation was introduced. With polynomial features XoR gate can be solved using ridge regression (a polynomial kernel will do the job).
As for ridge regression and linear kernel ridge regression, they are one and the same, the two formulations are equivalent, however depending on which dimension N or D is the biggest, you would prefer one or the other (to avoid inverting a big matrix) .
Decision boundaries
Hi,
Could a neural network, a knn or a ridge regression with a kernel trick solve the XOR problem ?
Is it possible to have this kind of decision boundary with a kernel trick ?
I guess that yes since a neural network could do it and a neural network performs feature extraction but I would like to confirm.
Finally, will the ridge regression and the kernelized ridge regression with a linear kernel that we have seen in class always give the same decision boundary ?
Thank you for your time.
NN yes, I think you've seen in the course. Knn I would say yes, but it does not make a lot of sense to solve the XoR gate. Plain ridge regression, No, this was one of the reasons why polynomial feature augmentation was introduced. With polynomial features XoR gate can be solved using ridge regression (a polynomial kernel will do the job).
As for ridge regression and linear kernel ridge regression, they are one and the same, the two formulations are equivalent, however depending on which dimension N or D is the biggest, you would prefer one or the other (to avoid inverting a big matrix) .
1
Add comment