I think I did not get when you presented the example with shared weights what was the "convention" for the correct way of making the convolution with the filter. Indeed, doing the Hadamard product, we can see that the filter is flipped along the axis where we do the convolution according to the convolution definition. However, are the weights between the input and the convolved feature corresponding to the values of the filter flipped or in the way it is presented on the slide 4? (Like is it 1 on top left or 0 on bottom right that will be the weight between the node input 11 and convolved feature 11)?
Convolutions as used in neural networks are typically (confusingly) 'correlation' operations, so the filters are multiplied pointwise with different patches of the image, without any flipping. This typical convention is different from what is shown on the slide you are referring to.
In the end, it doesn't matter much how you define your convolutions (flipping things or not), because the weights are learned anyway, and you would effectively learn the same model if you choose another convention.
Lecture 9: Flipping the filter for CNNs
Hi,
I think I did not get when you presented the example with shared weights what was the "convention" for the correct way of making the convolution with the filter. Indeed, doing the Hadamard product, we can see that the filter is flipped along the axis where we do the convolution according to the convolution definition. However, are the weights between the input and the convolved feature corresponding to the values of the filter flipped or in the way it is presented on the slide 4? (Like is it 1 on top left or 0 on bottom right that will be the weight between the node input 11 and convolved feature 11)?
Thanks a lot for the answer,
Best
Convolutions as used in neural networks are typically (confusingly) 'correlation' operations, so the filters are multiplied pointwise with different patches of the image, without any flipping. This typical convention is different from what is shown on the slide you are referring to.
In the end, it doesn't matter much how you define your convolutions (flipping things or not), because the weights are learned anyway, and you would effectively learn the same model if you choose another convention.
Add comment