What are possible values for data_augmentation_options in the TensorFlow Object Detection pipeline configuration?

Question

I have successfully trained an object detection model with TensorFlow with the sample configurations given here: https://github.com/tensorflow/models/tree/master/object_detection/samples/configs

Now I want to fine tune my configuration to get better results. One of the promising options I see in there is "data_augmentation_options" under "train_config". Currently, it looks like this:

train_config: {   batch_size: 1   ...   data_augmentation_options {     random_horizontal_flip {     }   } } 

Are there other options to do random scaling, cropping or tweaking of brightness?

Answer 1

The list of options is provided in preprocessor.proto:

NormalizeImage normalize_image = 1; RandomHorizontalFlip random_horizontal_flip = 2; RandomPixelValueScale random_pixel_value_scale = 3; RandomImageScale random_image_scale = 4; RandomRGBtoGray random_rgb_to_gray = 5; RandomAdjustBrightness random_adjust_brightness = 6; RandomAdjustContrast random_adjust_contrast = 7; RandomAdjustHue random_adjust_hue = 8; RandomAdjustSaturation random_adjust_saturation = 9; RandomDistortColor random_distort_color = 10; RandomJitterBoxes random_jitter_boxes = 11; RandomCropImage random_crop_image = 12; RandomPadImage random_pad_image = 13; RandomCropPadImage random_crop_pad_image = 14; RandomCropToAspectRatio random_crop_to_aspect_ratio = 15; RandomBlackPatches random_black_patches = 16; RandomResizeMethod random_resize_method = 17; ScaleBoxesToPixelCoordinates scale_boxes_to_pixel_coordinates = 18; ResizeImage resize_image = 19; SubtractChannelMean subtract_channel_mean = 20; SSDRandomCrop ssd_random_crop = 21; SSDRandomCropPad ssd_random_crop_pad = 22; SSDRandomCropFixedAspectRatio ssd_random_crop_fixed_aspect_ratio = 23; 

You can see the details about each option in preprocessor.py. Arguments can be provided as key-value pairs.

  data_augmentation_options {     ssd_random_crop {     }   }   data_augmentation_options {     random_pixel_value_scale {       minval: 0.6     }   } 

Answer 2

Adding to solution, here is the entire list added inside

data_augmentation_options {  ...   } 

Augmentation options (found here):

normalize_image {   original_minval: 0.0   original_maxval: 255.0   target_minval: -1.0   target_maxval: 1.0 }   random_horizontal_flip {   keypoint_flip_permutation: 1   keypoint_flip_permutation: 0   keypoint_flip_permutation: 2   keypoint_flip_permutation: 3   keypoint_flip_permutation: 5   keypoint_flip_permutation: 4   probability: 0.5 }   random_vertical_flip {   keypoint_flip_permutation: 1   keypoint_flip_permutation: 0   keypoint_flip_permutation: 2   keypoint_flip_permutation: 3   keypoint_flip_permutation: 5   keypoint_flip_permutation: 4   probability: 0.5 }   random_rotation90 {   keypoint_rot_permutation: 3   keypoint_rot_permutation: 0   keypoint_rot_permutation: 1   keypoint_rot_permutation: 2   probability: 0.5 }   random_pixel_value_scale {   minval: 0.8   maxval: 1.2 }   random_image_scale {   min_scale_ratio: 0.8   max_scale_ratio: 2.2 }   random_rgb_to_gray {   probability: 0.8 }   random_adjust_brightness {   max_delta: 0.2 }   random_adjust_contrast {   min_delta: 0.7   max_delta: 1.1 }   random_adjust_hue {   max_delta: 0.01 }   random_adjust_saturation {   min_delta: 0.75   max_delta: 1.15 }   random_distort_color {   color_ordering: 1 }   random_jitter_boxes {   ratio: 0.1   jitter_mode: SHRINK }   random_crop_image {   min_object_covered: 0.75   min_aspect_ratio: 0.75   max_aspect_ratio: 1.5   min_area: 0.25   max_area: 0.875   overlap_thresh: 0.5   clip_boxes: False   random_coef: 0.125 }   random_pad_image { }   random_absolute_pad_image {   max_height_padding: 50   max_width_padding: 100 }   random_crop_pad_image {   min_object_covered: 0.75   min_aspect_ratio: 0.75   max_aspect_ratio: 1.5   min_area: 0.25   max_area: 0.875   overlap_thresh: 0.5   clip_boxes: False   random_coef: 0.125 }   random_crop_pad_image {   min_object_covered: 0.75   min_aspect_ratio: 0.75   max_aspect_ratio: 1.5   min_area: 0.25   max_area: 0.875   overlap_thresh: 0.5   clip_boxes: False   random_coef: 0.125   min_padded_size_ratio: 0.5   min_padded_size_ratio: 0.75   max_padded_size_ratio: 0.5   max_padded_size_ratio: 0.75 }   random_crop_to_aspect_ratio {   aspect_ratio: 0.85   overlap_thresh: 0.35   clip_boxes: False }   random_black_patches {   max_black_patches: 20   probability: 0.95   size_to_image_ratio: 0.12 }   random_jpeg_quality {   random_coef: 0.5   min_jpeg_quality: 40   max_jpeg_quality: 90 }   random_downscale_to_target_pixels {   random_coef: 0.5   min_target_pixels: 200   max_target_pixels: 900 }   random_patch_gaussian {   random_coef: 0.5   min_patch_size: 10   max_patch_size: 300   min_gaussian_stddev: 0.2   max_gaussian_stddev: 1.5 }   autoaugment_image {   policy_name: 'v0' }   drop_label_probabilistically{   label: 2   drop_probability: 0.5 }   remap_labels{   original_labels: 1   original_labels: 2   new_label: 3 }   random_resize_method {   target_height: 75   target_width: 100 }   resize_image {   new_height: 75   new_width: 100   method: BICUBIC }    random_self_concat_image {   concat_vertical_probability: 0.5   concat_horizontal_probability: 0.25 }   ssd_random_crop {   operations {     min_object_covered: 0.0     min_aspect_ratio: 0.875     max_aspect_ratio: 1.125     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.0     clip_boxes: False     random_coef: 0.375   }   operations {     min_object_covered: 0.25     min_aspect_ratio: 0.75     max_aspect_ratio: 1.5     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.25     clip_boxes: True     random_coef: 0.375   } }   ssd_random_crop { }   ssd_random_crop_pad {   operations {     min_object_covered: 0.0     min_aspect_ratio: 0.875     max_aspect_ratio: 1.125     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.0     clip_boxes: False     random_coef: 0.375     min_padded_size_ratio: [1.0, 1.0]     max_padded_size_ratio: [2.0, 2.0]     pad_color_r: 0.5     pad_color_g: 0.5     pad_color_b: 0.5   }   operations {     min_object_covered: 0.25     min_aspect_ratio: 0.75     max_aspect_ratio: 1.5     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.25     clip_boxes: True     random_coef: 0.375     min_padded_size_ratio: [1.0, 1.0]     max_padded_size_ratio: [2.0, 2.0]     pad_color_r: 0.5     pad_color_g: 0.5     pad_color_b: 0.5   } }   ssd_random_crop_fixed_aspect_ratio {   operations {     min_object_covered: 0.0     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.0     clip_boxes: False     random_coef: 0.375   }   operations {     min_object_covered: 0.25     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.25     clip_boxes: True     random_coef: 0.375   }   aspect_ratio: 0.875 }   ssd_random_crop_pad_fixed_aspect_ratio {   operations {     min_object_covered: 0.0     min_aspect_ratio: 0.875     max_aspect_ratio: 1.125     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.0     clip_boxes: False     random_coef: 0.375   }   operations {     min_object_covered: 0.25     min_aspect_ratio: 0.75     max_aspect_ratio: 1.5     min_area: 0.5     max_area: 1.0     overlap_thresh: 0.25     clip_boxes: True     random_coef: 0.375   }   aspect_ratio: 0.875   min_padded_size_ratio: [1.0, 1.0]   max_padded_size_ratio: [2.0, 2.0] }   convert_class_logits_to_softmax {     temperature: 2 }   random_square_crop_by_scale {   scale_min: 0.25   scale_max: 2.0   num_scales: 8 }   adjust_gamma {   gamma: 2.2   gain: 2.0 }