Do image have to have the same size for deep learning?

Question

I'm new to deep learning. I'm planning to use caffe and preparing a dataset for the training.

Do all the images have to have the same size? And does it have to be a square?

If so, what would be the ideal size or how to choose it?

Answer 1

Normally for deep learning this does not have to be the case. Convolutional Neural Networks do not depend on the image size and filters can be applied on all image sizes.

Still many frameworks and literally all papers use the same image sizes for training. In https://arxiv.org/pdf/1409.1556/ they used different sizes for evaluating the network. To achieve this you can use either resizing or crops or a combination of the both. Keep in mind that changing the aspect ratio is almost always a bad idea.

To choose a good image size it is important to note that a bigger image sizes will give you better accuracy normally. However all the filter take longer and the memory requirements rise with the image size. Additionally larger sizes yield diminishing improvements. I normally use 224x224, because it is often divisible through 2 and imagenet uses it too.

Finally the image size does not have to be square, but it is most of the time a good idea, because CNNs often cut the image size in half and often end up at something like 4x4 or 6x6. Doing this with a non square starting size will give you an akward ending size like 4x2 or 6x3.