imgaug: load and save images

Question

I am using Python+Tensorflow for CNN training on a high-performance computing cluster. I am training a convolutional neural network, but have a relatively small dataset. So I am implementing techniques to augment it. Now this is the first time i am working on a core computer vision problem so am relatively new to it.

As raising the number of images for training, the imgaug package (https://github.com/aleju/imgaug) which requires cv2 seems to be perfect and a simple solution to multiply the number of images. I installed opencv2 using python 2.7, all required python packages are running locally (no env/Anaconda/Docker are used here).

Practically, I’ am trying to run this code from here:

import os
import imgaug as ia
from imgaug import augmenters as iaa
import scipy
import cv2
import numpy as np
# optional check my hint import scipy.misc import imwrite
# optional check my hint import scipy.misc import imsave

ia.seed(1)

# Example batch of images.
# The array has shape (32, 64, 64, 3) and dtype uint8.
images = np.array(
    [ia.quokka(size=(64, 64)) for _ in range(32)],
    dtype=np.uint8
)

seq = iaa.Sequential([
    iaa.Fliplr(0.5), # horizontal flips
    iaa.Crop(percent=(0, 0.1)), # random crops
    # Small gaussian blur with random sigma between 0 and 0.5.
    # But we only blur about 50% of all images.
    iaa.Sometimes(0.5,
        iaa.GaussianBlur(sigma=(0, 0.5))
    ),
    # Strengthen or weaken the contrast in each image.
    iaa.ContrastNormalization((0.75, 1.5)),
    # Add gaussian noise.
    # For 50% of all images, we sample the noise once per pixel.
    # For the other 50% of all images, we sample the noise per pixel AND
    # channel. This can change the color (not only brightness) of the
    # pixels.
    iaa.AdditiveGaussianNoise(loc=0, scale=(0.0, 0.05*255), per_channel=0.5),
    # Make some images brighter and some darker.
    # In 20% of all cases, we sample the multiplier once per channel,
    # which can end up changing the color of the images.
    iaa.Multiply((0.8, 1.2), per_channel=0.2),
    # Apply affine transformations to each image.
    # Scale/zoom them, translate/move them, rotate them and shear them.
    iaa.Affine(
        scale={"x": (0.8, 1.2), "y": (0.8, 1.2)},
        translate_percent={"x": (-0.2, 0.2), "y": (-0.2, 0.2)},
        rotate=(-25, 25),
        shear=(-8, 8)
    )
], random_order=True) # apply augmenters in random order

images_aug = seq.augment_images(images)

After creating a .py file with this code, I have copied this file into a folder of 50 images (.jpg). The code itself runs perfectly (no error encounter), but it seems like that the .jpg are not loaded into the.py file, even if I set the folder path, it will not load the images. Also, no new files were written.

Question 1: I guess I must load the images as an array into the .py, or into the cv2 but I can’t find any example how to load and write them, as I have never done something like this. Any help?

Question 2: What is the best interface to use in this context? All the images are stored in folders or .xlsx files.

(Hint: May be this is an option unfortunately its based on Keras (Link)? Or these 2 option I’ve found, but I am not able to use them scipy.ndimage.imread and scipy.misc.imsave)

Answer 1

images_aug = seq.augment_images(images)

Your code does augmentation on images, not on your files. So you have read your files content first. The author also have these lines in their README.

for batch_idx in range(1000):
    # 'images' should be either a 4D numpy array of shape (N, height, width, channels)
    # or a list of 3D numpy arrays, each having shape (height, width, channels).
    # Grayscale images must have shape (height, width, 1) each.
    # All images must have numpy's dtype uint8. Values are expected to be in
    # range 0-255.

Question 1: I guess I must load the images as an array into the .py, or into the cv2 but I can’t find any example how to load and write them, as I have never done something like this. Any help?

Simply create a 4d array with size (N, height, width, channels). Then read each image and push it into that array will work. For example

import numpy as np
import cv2
images = np.zeros((N, height, width, channels))
for idx, img_path in enumerate(img_paths):
    img = cv2.imread(img_path, 1)
    images[idx, :, :, :] = img

Question 2: What is the best interface to use in this context? All the images are stored in folders or .xlsx files.

Data argumentation is used to increase the robustness of training data. If your "interface" means deep learning framework then any framework which has a python interface should work well.

Hope that helps.

Answer 2

I have read the source code of imgaug, the method 'ia.quokka' return (H,W,3) ndarray(the image array of dtype uint8.),so you can change the example to read and save images.

This is my use:

import imgaug as ia
from imgaug import augmenters as iaa
import numpy as np
import imageio

ia.seed(1)

img = imageio.imread("test.jpg") #read you image
images = np.array(
    [img for _ in range(32)], dtype=np.uint8)  # 32 means creat 32 enhanced images using following methods.

seq = iaa.Sequential(
    [
        iaa.Fliplr(0.5),  
        iaa.Crop(percent=(0, 0.1)),            
        iaa.Sometimes(0.5, iaa.GaussianBlur(sigma=(0, 0.5))),        
        iaa.ContrastNormalization((0.75, 1.5)),         
        iaa.AdditiveGaussianNoise(
            loc=0, scale=(0.0, 0.05 * 255), per_channel=0.5),    
        iaa.Multiply((0.8, 1.2), per_channel=0.2),
        iaa.Affine(
            scale={
                "x": (0.8, 1.2),
                "y": (0.8, 1.2)
            },
            translate_percent={
                "x": (-0.2, 0.2),
                "y": (-0.2, 0.2)
            },
            rotate=(-25, 25),
            shear=(-8, 8))
    ],
    random_order=True)  # apply augmenters in random order

images_aug = seq.augment_images(images)

for i in range(32):
    imageio.imwrite(str(i)+'new.jpg', images_aug[i])  #write all changed images