How do you load, label, and feed jpeg data into Tensorflow?

Question

I have been trying to feed 1750 * 1750 images into Tensorflow, but I do not know how to label and feed the data after I convert the images into a Tensor using the tf.image.decode_jpeg() function.

Currently, my code is:

import tensorflow as tf
import numpy as np

import imageflow
import os, glob

sess = tf.InteractiveSession()

def read_jpeg(filename_queue):
 reader = tf.WholeFileReader()
 key, value = reader.read(filename_queue)

 my_img = tf.image.decode_jpeg(value)
 my_img.set_shape([1750, 1750, 1])
 print(value)
 return my_img

#####################################################
def read_image_data():
 jpeg_files = []
 images_tensor = []

 i = 1
 WORKING_PATH = "/Users/Zanhuang/Desktop/NNP/DATA"
 jpeg_files_path = glob.glob(os.path.join(WORKING_PATH, '*.jpeg'))

 for filename in jpeg_files_path:
    print(i)
    i += 1
    jpeg_files.append(filename)


 filename_queue = tf.train.string_input_producer(jpeg_files)

 mlist = [read_jpeg(filename_queue) for _ in range(len(jpeg_files))]

 init = tf.initialize_all_variables()

 sess = tf.Session()
 sess.run(init)

 images_tensor = tf.convert_to_tensor(images_tensor)


 sess.close()

Now, as I said earlier, I need to feed and label the data. I have seen the CIFAR-10 tutorial files, but they stored the labels in a file and I plan on not doing that way.

I am quite new to Tensorflow so please keep the response as detailed as possible.

Thanks!

Answer 1

Depending on what you are trying to do, there are several directions to consider.

1. If you just wish to run inference on an arbitrary JPEG file (i.e. labels are not required), then you can follow the example of classify_image.py which feeds in a JPEG image into a pre-trained Inception network:

   github.com/tensorflow/models/blob/master/tutorials/image/imagenet/classify_image.py

2. If you do wish to train (or fine-tune) a model on a small custom data set of JPEG images, then take a look at this example for how to train a model off a small set of JPEG images.

   github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

3. If you do wish to train (or fine-tune) a model on a large custom data set of JPEG images, then reading many individual JPEG files will be inefficient and slow down training tremendously.

I would suggest following the procedure of described in the inception/ model library that converts a directory of JPEG images into sharded RecordIO containing serialized JPEG images.

github.com/tensorflow/models/blob/master/research/inception/inception/data/build_image_data.py

Instructions for running the conversion script are available here:

github.com/tensorflow/models/blob/master/research/inception/README.md#how-to-construct-a-new-dataset-for-retraining

After running the conversion, you may then employ/copy the image preprocessing pipeline used by the inception/ model.

github.com/tensorflow/models/blob/master/research/inception/inception/image_processing.py