tf.assign to variable slice doesn't work inside tf.while_loop

Question

What is wrong with the following code? The tf.assign op works just fine when applied to a slice of a tf.Variable if it happens outside of a loop. But, in this context, it gives the error below.

import tensorflow as tf

v = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
n = len(v)
a = tf.Variable(v, name = 'a')

def cond(i, a):
    return i < n 

def body(i, a):
    tf.assign(a[i], a[i-1] + a[i-2])
    return i + 1, a

i, b = tf.while_loop(cond, body, [2, a]) 

results in:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hrbigelow/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3210, in while_loop
    result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
  File "/home/hrbigelow/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2942, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/home/hrbigelow/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2879, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/home/hrbigelow/ai/lb-wavenet/while_var_test.py", line 11, in body
    tf.assign(a[i], a[i-1] + a[i-2])
  File "/home/hrbigelow/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/state_ops.py", line 220, in assign
    return ref.assign(value, name=name)
  File "/home/hrbigelow/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 697, in assign
    raise ValueError("Sliced assignment is only supported for variables")
ValueError: Sliced assignment is only supported for variables

Answer 1

Your variable is not an output of the operations run inside your loop, it is an external entity living outside the loop. So you do not have to provide it as an argument.

Also, you need to enforce the update to take place, for example using tf.control_dependencies in body.

import tensorflow as tf

v = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
n = len(v)
a = tf.Variable(v, name = 'a')

def cond(i):
    return i < n 

def body(i):
    op = tf.assign(a[i], a[i-1] + a[i-2])
    with tf.control_dependencies([op]):
      return i + 1

i = tf.while_loop(cond, body, [2])

sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
i.eval()
print(a.eval())
# [ 1  1  2  3  5  8 13 21 34 55 89]

Possibly you may want to be cautious and set parallel_iterations=1 to enforce the loop to run sequentially.

Answer 2

It makes sense from a CUDA perspective to disallow assignment of individual indices as it negates all performance benefits of heterogeneous parallel computing.

I know this adds a bit of computational overhead but it works.

import tensorflow as tf

v = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
n = len(v)
a = tf.Variable(v, name = 'a',dtype=tf.float32)

def cond(i, a):
    return i < n 

def body(i, a1):
    e = tf.eye(n,n)[i]
    a1 = a1 + e *(a1[i-1] + a1[i-2])
    return i + 1, a1

i, b = tf.while_loop(cond, body, [2, a]) 

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print('i: ',sess.run(i))
    print('b: ',sess.run(b))