conditional graph in tensorflow and for loop that accesses tensor size

Question

First the broad questions:

• Is it possible to build a conditional graph with tensorflow?
• If yes, do the automated gradient-calcuation and the implemented optimizers work with it?
• Can I access the shape of a tensor and turn it into an integer for using it with an "if" condition and a "for i in range()" loop?

My actual use case is that I want to do a 1D convolution with variable tensor length. For this I first need an if statement that only executes the convolution if the length is greater then one. Then I have a for loop that goes through the tensor for the convolution. The problem is that this code:

for i in range(tf.shape(tensor)[0]): 

doesn't work because the range operator needs an integer. Can I turn this somehow into an integer?

In the end I want to train this model with adagrad, either with the automatic differentiation or the already implemented optimiser

---

Edit:

this is the 1D convolution which will later be the first of two Layers in my Model. The type errors are behind each version of the for loop that triggers one

import tensorflow as tf 
import numpy as np 

def convolve(s, Tl, Tr, b):

    if (tf.shape(s)[0] == 1):
        return s

    sum = 0

    # for i in range(tf.shape(s)[0] - 1): # error: TypeError: range() integer end argument expected, got Tensor
    # for i in range(s._shape._dims[0]._value - 1): # error: TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
    for i in range(s.get_shape().as_list()[0] - 1): # error: TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

        sum += tf.tanh(tf.matmul(Tl,s[i]) + tf.matmul(Tr, s[i+1]) + b)

    return sum

ModelSize = 3

# tensor to be convolved
s = tf.placeholder("float", shape = [None, ModelSize])

# initialise weights
Tl = tf.Variable(tf.random_normal(shape=[ModelSize,ModelSize], stddev = 0.1 ))
Tr = tf.Variable(tf.random_normal(shape=[ModelSize,ModelSize], stddev = 0.1 ))
b = tf.Variable(tf.random_normal(shape=[ModelSize], stddev = 0.1 ))

#convolution
s_convolved = convolve(s, Tl, Tr, b)

# initialise variables.
init = tf.initialize_all_variables()

# run graph
sess = tf.Session()
sess.run(init)

# test data
s_dataLong = np.random.random((2,5,ModelSize))
s_dataShort = np.random.random((2,1,ModelSize))

for s_dataPart in s_dataLong:
    print sess.run(s_convolved, feed_dict = {s : s_dataPart})

for s_dataPart in s_dataShort:
    print sess.run(s_convolved, feed_dict = {s : s_dataPart})

Answer 1

I recommend you to write each question differently. Otherwise it will be closed as too broad.

I can answer only your 3-rd question. How to programmatically get a shape of the tensor. You are correctly using shape to get the shape of the tensor, but you still can not get the results before you run the graph (look at my explanation here).

a = tf.truncated_normal([2, 3], mean=0.0, stddev=0.02, dtype=tf.float32, seed=1)
b = tf.shape(a)
sess = tf.Session()
print sess.run(b) # will give you [2 3]

The ugly way that I have found to get the shape from constants, without running the graph is to do something like (do not really know why would you need it):

print a._shape._dims[0]._value
print a._shape._dims[1]._value

To get the shape from a variable, you can do this:

weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35))
print weights.get_shape().as_list()

Another way to access a shape of a Tensor before the evaluation is: tf.Tensor.get_shape()