What is the equivalent of the following code in tensorflow?

Question

I have the following function:

import random

lst = []
for i in range(100):
    lst.append(random.randint(1, 10))

print(lst)

buffer = []

# This is the peace of code which I am interested to convert into tensorflow.
for a in lst:
    buffer.append(a)

    if len(buffer) > 5:
        buffer.pop(0)

    if len(buffer) == 5:
        print(buffer)

So, from the code, I need to create a buffer (that could be a variable in tensorflow). This buffer should hold the extracted features from the last conv layer. The variable will be an input to an RNN in my case.

The advantage of this approach is that when we have large images, and when we need to feed a RNN with a (batch of images) * (sequence length) * (size of 1 image), that will require a very big batch of images to be loaded into the main memory. On the other hand, according to the code above, we will be feeding 1 image at a time using the Datasets from tensorflow, or an input queue or any other alternative. As a result, we will be storing in memory the feature of size: batch_size * sequence_length * feature space .In addition, we can say:

if len(buffer) == n:
    # empty out the buffer after using its elements
    buffer = [] # Or any other alternative way

I am aware that I can feed my network batches of images, but I need to accomplish the mentioned code based on some literature.

Any help is much appreciated!!

Answer 1

I try to regenerate your output using tf.FIFOQueue (https://www.tensorflow.org/api_docs/python/tf/FIFOQueue). I have given my code below with the comments where necessary.

BATCH_SIZE = 20

lst = []
for i in range(BATCH_SIZE):
    lst.append(random.randint(1, 10))
print(lst)

curr_data = np.reshape(lst, (BATCH_SIZE, 1)) # reshape the tensor so that [BATCH_SIZE 1]

# queue starts here
queue_input_data = tf.placeholder(tf.int32, shape=[1]) # Placeholder for feed the data

queue = tf.FIFOQueue(capacity=50, dtypes=[tf.int32], shapes=[1]) # Queue define here

enqueue_op = queue.enqueue([queue_input_data])  # enqueue operation
len_op = queue.size()  # chek the queue size

#check the length of the queue and dequeue one if greater than 5
dequeue_one = tf.cond(tf.greater(len_op, 5), lambda: queue.dequeue(), lambda: 0)
#check the length of the queue and dequeue five elemts if equals to 5
dequeue_many = tf.cond(tf.equal(len_op, 5), lambda:queue.dequeue_many(5), lambda: 0)

with tf.Session() as session:
    for i in range(BATCH_SIZE):
        _ = session.run(enqueue_op, feed_dict={queue_input_data: curr_data[i]}) # enqueue one element each ietaration
        len = session.run(len_op)  # check the legth of the queue
        print(len)

        element = session.run(dequeue_one)  # dequeue the first element
        print(element)

However, following two problems are associated with the above code,

1. Only the dequeue one and dequeue many operations are available and you cannot see the elements inside the queue (I don't think you will need this since you are looking something like a pipeline).

2. I think that tf.cond is the only way to implement a conditional operation (I couldn't find any other suitable function similar to that). However, since its similar to the if-then-else statement, its mandatory to define an operation when the statement is false also (not just having only if statement without the else). Since Tensorflow is all about building a graph I think its necessary to include the two branches (when the condition is true and false).

Moreover, a good explanation for Tensorflow input pipelines can be found here (http://ischlag.github.io/2016/11/07/tensorflow-input-pipeline-for-large-datasets/).

Hope this helps.