Can Keras deal with input images with different size?

Question

Can the Keras deal with input images with different size? For example, in the fully convolutional neural network, the input images can have any size. However, we need to specify the input shape when we create a network by Keras. Therefore, how can we use Keras to deal with different input size without resizing the input images to the same size? Thanks for any help.

Answer 1

Yes. Just change your input shape to shape=(n_channels, None, None). Where n_channels is the number of channels in your input image.

I'm using Theano backend though, so if you are using tensorflow you might have to change it to (None,None,n_channels)

You should use:

input_shape=(1, None, None)

None in a shape denotes a variable dimension. Note that not all layers will work with such variable dimensions, since some layers require shape information (such as Flatten). https://github.com/fchollet/keras/issues/1920

For example, using keras's functional API your input layer would be:

For a RGB dataset

inp = Input(shape=(3,None,None))

For a Gray dataset

inp = Input(shape=(1,None,None))

Answer 2

Implementing arbitrarily sized input arrays with the same computational kernels can pose many challenges - e.g. on a GPU, you need to know how big buffers to reserve, and more weakly how much to unroll your loops, etc. This is the main reason that Keras requires constant input shapes, variable-sized inputs are too painful to deal with.

This more commonly occurs when processing variable-length sequences like sentences in NLP. The common approach is to establish an upper bound on the size (and crop longer sequences), and then pad the sequences with zeros up to this size.

(You could also include masking on zero values to skip computations on the padded areas, except that the convolutional layers in Keras might still not support masked inputs...)

I'm not sure if for 3D data structures, the overhead of padding is not prohibitive - if you start getting memory errors, the easiest workaround is to reduce the batch size. Let us know about your experience with applying this trick on images!