Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to format the image data for training/prediction when images are different in size?

I am trying to train my model which classifies images. The problem I have is, they have different sizes. how should i format my images/or model architecture ?

like image 707
Asif Mohammed Avatar asked Jan 28 '17 07:01

Asif Mohammed


People also ask

What should I do if I have different sizes of images in CNN?

Conventionally, when dealing with images of different sizes in CNN(which happens very often in real world problems), we resize the images to the size of the smallest images with the help of any image manipulation library (OpenCV, PIL etc) or some times, pad the images of unequal size to desired size.

Does image size matter in machine learning?

Smaller images will train significantly faster, and possibly even converge quicker (all other factors held constant) as you will be able to train on bigger batches (e.g. 100-1000 images in one pass, which you might not be able to do on a single machine with high res imagery).


2 Answers

You didn't say what architecture you're talking about. Since you said you want to classify images, I'm assuming it's a partly convolutional, partly fully connected network like AlexNet, GoogLeNet, etc. In general, the answer to your question depends on the network type you are working with.

If, for example, your network only contains convolutional units - that is to say, does not contain fully connected layers - it can be invariant to the input image's size. Such a network could process the input images and in turn return another image ("convolutional all the way"); you would have to make sure that the output matches what you expect, since you have to determine the loss in some way, of course.

If you are using fully connected units though, you're up for trouble: Here you have a fixed number of learned weights your network has to work with, so varying inputs would require a varying number of weights - and that's not possible.

If that is your problem, here's some things you can do:

  • Don't care about squashing the images. A network might learn to make sense of the content anyway; does scale and perspective mean anything to the content anyway?
  • Center-crop the images to a specific size. If you fear you're losing data, do multiple crops and use these to augment your input data, so that the original image will be split into N different images of correct size.
  • Pad the images with a solid color to a squared size, then resize.
  • Do a combination of that.

The padding option might introduce an additional error source to the network's prediction, as the network might (read: likely will) be biased to images that contain such a padded border. If you need some ideas, have a look at the Images section of the TensorFlow documentation, there's pieces like resize_image_with_crop_or_pad that take away the bigger work.

As for just don't caring about squashing, here's a piece of the preprocessing pipeline of the famous Inception network:

# This resizing operation may distort the images because the aspect # ratio is not respected. We select a resize method in a round robin # fashion based on the thread number. # Note that ResizeMethod contains 4 enumerated resizing methods.  # We select only 1 case for fast_mode bilinear. num_resize_cases = 1 if fast_mode else 4 distorted_image = apply_with_random_selector(     distorted_image,     lambda x, method: tf.image.resize_images(x, [height, width], method=method),     num_cases=num_resize_cases) 

They're totally aware of it and do it anyway.

Depending on how far you want or need to go, there actually is a paper here called Spatial Pyramid Pooling in Deep Convolution Networks for Visual Recognition that handles inputs of arbitrary sizes by processing them in a very special way.

like image 129
sunside Avatar answered Oct 02 '22 20:10

sunside


Try making a spatial pyramid pooling layer. Then put it after your last convolution layer so that the FC layers always get constant dimensional vectors as input . During training , train the images from the entire dataset using a particular image size for one epoch . Then for the next epoch , switch to a different image size and continue training .

like image 32
Pranay Mukherjee Avatar answered Oct 02 '22 21:10

Pranay Mukherjee