Non-squared convolution kernel size

Question

It is very common to use squared_sized kernels for convolutional neural network, i.e. (3,3), (5,5), etc.

What would be the cons and pros of using non-squared kernel sizes? meaning (3,7), (3,9), etc.

Answer 1

I can not think of any cons. It really depends what you wannado and what your data is.

When you use a squared size kernel, you use that kernel to translate that area to one point in the output of the conv. So using a square, each point in the output are obtained from a fair set of weighted neighbours of an input point( same number of vertical neighbours as the horizontal one).

However, if you use a non square kernel size, let's say a 3×9 kernel size,you map each input point using 3 times more of horizontal than vertical ( or vice versa). Depending on the nature of the data, that might simplify your training process and increase the accuracy. ( if you are trying to detect very large thin crocodiles for example^_^). After all, these are all my opinions, not a 100% scientific facts.

Answer 2

The reason behind squared sized kernels is that you in general have no idea what orientation the learned features will have. So you don't want to restrict the network. The optimal shape for a filter would be a circle, so it can learn any feature with an arbitrary orientation inside some region with a given radius. Since this is not really feasible because of implementation problems a square it the next best shape.

If you would know e.g. that all learned features will have the ratio 1x3 (heightxwidth) you could use a kernel size like 2x6. But you just don't know this. Even if you say that the objects you want to detect/classify look like this, it doesn't translate to the features the network will learn to identify it. The whole advantage is that you can let the network learn the features and imo you should restrict this as little as possible.

But I don't want to discourage you. Deep learning is a lot of experimentation and trial and error. So just try it out and see for yourself. Maybe for some kind of problem it actually performs better, who knows.