Writing tfrecords with images and multilabels for classification

Question

I want to perform a multi-label classification with TensorFlow. I have about 95000 images and for each image there is a corresponding label vector. For every image there are 7 labels. These 7 labels are represented as a tensor with size 7. Each image has the shape of (299,299,3).

How can I now write the image with the corresponding label vector/tensor to the .tfrecords File

my current code/approach:

def get_decode_and_resize_image(image_id):
    image_queue = tf.train.string_input_producer(['../../original-data/'+image_id+".jpg"])
    image_reader = tf.WholeFileReader()
    image_key, image_value = image_reader.read(image_queue)
    image = tf.image.decode_jpeg(image_value,channels=3)
    resized_image= tf.image.resize_images(image, 299, 299, align_corners=False)
    return resized_image



init_op = tf.initialize_all_variables()
with tf.Session() as sess:
 # Start populating the filename queue.

 sess.run(init_op)
 coord = tf.train.Coordinator()
 threads = tf.train.start_queue_runners(coord=coord)

 # get all labels and image ids
 csv= pd.read_csv('../../filteredLabelsToPhotos.csv')

 #create a writer for writing to the .tfrecords file
 writer = tf.python_io.TFRecordWriter("tfrecords/data.tfrecords")

 for index,row in csv.iterrows():

     # the labels
     image_id = row['photo_id']
     lunch = tf.to_float(row["lunch"])
     dinner= tf.to_float(row["dinner"])
     reservations= tf.to_float(row["TK"])
     outdoor = tf.to_float(row["OS"])
     waiter = tf.to_float(row["WS"])
     classy = tf.to_float(row["c"])
     gfk = tf.to_float(row["GFK"])

     labels_list = [lunch,dinner,reservations,outdoor,waiter,classy,gfk]
     labels_tensor = tf.convert_to_tensor(labels_list)

     #get the corresponding image
     image_file= get_decode_and_resize_image(image_id=image_id)

     #here : how do I now create a TFExample and write it to the .tfrecords file






 coord.request_stop()
 coord.join(threads)

And after I´ve created the .tfrecords file, can i then read it from my TensorFlow Training Code and batch the data automatically?

Answer 1

To expand on Alexandre's answer, you can do something like this:

# Set this up before your for-loop, you'll use this repeatedly
tfrecords_filename = 'myfile.tfrecords'
writer = tf.python_io.TFRecordWriter(tfrecords_filename)

# Then within your for-loop, you can write like so:
for ...:

  #here : how do I now create a TFExample and write it to the .tfrecords file

  example = tf.train.Example(features=tf.train.Features(feature={
    'image_raw': tf.train.Feature(bytes_list=tf.train.BytesList(value=[image_file])),
    # the other features, labels you wish to include go here too
  }))
  writer.write(example.SerializeToString())

# then finally, don't forget to close the writer.
writer.close()

This assumes you have already converted the image into a byte array in the image_file variable.

I adapted this from this very helpful post that goes into detail on serialising images & may be helpful to you if my assumption above is false.