Keras ImageDataGenerator with center crop for rotation and translation shift

Question

I need to do data augmentation but not with any fill modes, constant, reflect, nearest, wrap. Instead everytime the image is rotated or translated, I would like to have it center-cropped (shown below) so as not have any black, white, reflected, or constant edges/borders as explained here.

How do I extend the ImageDataGenerator class (if that's the only way to do it and no center crop is available out of the box) with these points taken into account?

1. Keep existing parts of the ImageDataGenerator other than the augmentation part, and write a custom augmentation function

2. It would be efficient to retain the images of original size without resizing before augmentation happens because center crop would result in huge loss of data after resize. Translate/Rotate -> Center crop -> Resize should be more efficient than Resize -> Translate/Rotate -> Center crop

Answer 1

In case someone is looking for a solution, here's how I managed to solve the problem. The main idea is to wrap the ImageDataGenerator in a custom generator, like this:

def crop_generator(batches, new_size):
    while True:
        batch_x, batch_y = next(batches)
        x= batch_x.shape[1] // 2
        y= batch_x.shape[2] // 2
        size = new_size // 2
        yield (batch_x[:, x-size:x+size, y-size:y+size], batch_y)

x_train = HDF5Matrix(...)
y_train = HDF5Matrix(...)

datagen = ImageDataGenerator(rotation_range=180, ...)

model = create_model()

training_gen = crop_generator(datagen.flow(x_train, y_train, batch_size=128), new_size=64)

model.fit_generator(training_gen, ...)

Using numpy indexing batch_x[:, x-size:x+size, y-size:y+size, :] we only alter x and y dimensions of the images, leaving batch size and channel dimensions the same. This allows us to avoid the for-loop.

Answer 2

This may be helpful,

Extending Keras' ImageDataGenerator to Support Random Cropping

https://jkjung-avt.github.io/keras-image-cropping/

github code:

https://github.com/jkjung-avt/keras-cats-dogs-tutorial/blob/master/train_cropped.py

train_datagen = ImageDataGenerator(......)
train_batches = train_datagen.flow_from_directory(DATASET_PATH + '/train',
                                              target_size=(256,256),
                                              ......)
train_crops = crop_generator(train_batches, 224)
net_final.fit_generator(train_crops, ......)