Saving image files in Tensorflow

Question

I'm just starting with Tensorflow and I have a newbie question.

I know that Tensorflow is all about neural nets but I'm starting with just the mechanics of it. I'm trying to get it to load, resize, flip, and save two images. Should be a simple operation, right, and it gets me started with the basics.

Here's my code so far:

import tensorflow as tf
import numpy as np

print("resizing images")

filenames = ['img1.png', 'img2.png' ]
filename_queue = tf.train.string_input_producer(filenames, num_epochs=1)

reader = tf.WholeFileReader()
key,value = reader.read(filename_queue)
images = tf.image.decode_png(value)

resized = tf.image.resize_images(images, 180,180, 1)
resized.set_shape([180,180,3])

flipped_images = tf.image.flip_up_down(resized)

resized_encoded = tf.image.encode_jpeg(flipped_images,name="save_me")

init = tf.initialize_all_variables()
sess = tf.Session()

with sess.as_default():
  tf.train.start_queue_runners()
  sess.run(init)

  f = open("/tmp/foo1.jpeg", "wb+")
  f.write(resized_encoded.eval())
  f.close()

  f = open("/tmp/foo2.jpeg", "wb+")
  f.write(resized_encoded.eval())
  f.close()

It works fine, resizing the two images and saving them. But it always ends with an error:

W tensorflow/core/common_runtime/executor.cc:1076] 0x7f97240e7a40
 Compute status: Out of range: Reached limit of 1

I'm obviously doing something wrong. If I take off the num_epochs=1, then it ends with no error.

I have a few questions:

How do I do this correctly?

Also, if I want to preserve the original file names all the way from the filename_queue through to the end so I can save them with the original names, how do I do that? And how do I know how many files I need to save? Let's say I'm making the list of file names by reading a directory. I tried many different things but I could never find out how I know when I reach the end.

It seems strange to me that I'm calling resized_encoded.eval() twice.

Thank you and I'm sure this is a very basic question but I'm not understanding how this works.

Edit: I created an even simpler demonstration of the behavior:

import tensorflow as tf
import numpy as np

filenames = ['file1.png', 'file2.png' ]

filename_queue = tf.train.string_input_producer(filenames, 
                        num_epochs=1, name="my_file_q")

reader = tf.WholeFileReader()
key,value = reader.read(filename_queue)
init = tf.initialize_all_variables()

sess = tf.Session()

with sess.as_default():
  print("session started")

  sess.run(init)

  coord = tf.train.Coordinator()
  threads = tf.train.start_queue_runners(coord=coord)

  for i in range (2):
    print(key.eval())

  coord.request_stop()
  coord.join(threads)

This gives the same warning. I can't understand why.

Answer 1

This warning is perfectly normal. As stated in the TensorFlow API

num_epochs: An integer (optional). If specified, string_input_producer produces each string from string_tensor num_epochs times before generating an OutOfRange error. If not specified, string_input_producer can cycle through the strings in string_tensor an unlimited number of times.

Why is this important, you may ask. I have refactored your code into something perhaps more understandable, in my opinion. Let me explain.

import tensorflow as tf
import numpy as np
import os
from PIL import Image

cur_dir = os.getcwd()
print("resizing images")
print("current directory:",cur_dir)

def modify_image(image):
    resized = tf.image.resize_images(image, 180, 180, 1)
    resized.set_shape([180,180,3])
    flipped_images = tf.image.flip_up_down(resized)
    return flipped_images

def read_image(filename_queue):
    reader = tf.WholeFileReader()
    key,value = reader.read(filename_queue)
    image = tf.image.decode_jpeg(value)
    return image

def inputs():
    filenames = ['img1.jpg', 'img2.jpg' ]
    filename_queue = tf.train.string_input_producer(filenames,num_epochs=2)
    read_input = read_image(filename_queue)
    reshaped_image = modify_image(read_input)
    return reshaped_image

with tf.Graph().as_default():
    image = inputs()
    init = tf.initialize_all_variables()
    sess = tf.Session()
    sess.run(init)
    tf.train.start_queue_runners(sess=sess)
    for i in xrange(2):
        img = sess.run(image)
        img = Image.fromarray(img, "RGB")
        img.save(os.path.join(cur_dir,"foo"+str(i)+".jpeg"))

In the code above, if you explicitly put num_epochs=2, then as the API suggests, string_input_producer cycles through the strings in string_tensor 2 times. Since the string_tensor has only 2 filenames, the queue is filled with 4 filenames. If I change the for loop to be:

for i in xrange(5)

then this will bug out. However, if I leave it at 4, then it will be fine. Take yet another example. If I do not put num_epochs, then as suggested, it can cycle through unlimited number of times. Putting:

for i in xrange(100)

thus does not bug out. I hope this answers your question.

EDIT: I realized you have more questions.

Also, if I want to preserve the original file names all the way from the filename_queue through to the end so I can save them with the original names, how do I do that? And how do I know how many files I need to save? Let's say I'm making the list of file names by reading a directory. I tried many different things but I could never find out how I know when I reach the end.

If you want to preserve the original file names, then your method needs to return the file name. Here's the code below.

import tensorflow as tf
import numpy as np
import os
from PIL import Image

cur_dir = os.getcwd()
print("resizing images")
print("current directory:",cur_dir)

def modify_image(image):
    resized = tf.image.resize_images(image, 180, 180, 1)
    resized.set_shape([180,180,3])
    flipped_images = tf.image.flip_up_down(resized)
    return flipped_images

def read_image(filename_queue):
    reader = tf.WholeFileReader()
    key,value = reader.read(filename_queue)
    image = tf.image.decode_jpeg(value)
    return key,image

def inputs():
    filenames = ['img1.jpg', 'img2.jpg' ]
    filename_queue = tf.train.string_input_producer(filenames)
    filename,read_input = read_image(filename_queue)
    reshaped_image = modify_image(read_input)
    return filename,reshaped_image

with tf.Graph().as_default():
    image = inputs()
    init = tf.initialize_all_variables()
    sess = tf.Session()
    sess.run(init)
    tf.train.start_queue_runners(sess=sess)
    for i in xrange(10):
        filename,img = sess.run(image)
        print (filename)
        img = Image.fromarray(img, "RGB")
        img.save(os.path.join(cur_dir,"foo"+str(i)+".jpeg"))

To know how many files you need to save, you could just call something along the lines of:

os.listdir(os.getcwd())

This lists all files in the directory. Check the API of os.listdir to filter specifically JPG, PNG file types. Once you get this, you can call a simple length operation and do:

for i in xrange(len(number_of_elements))