Are the operations in tf.group executed in order?

Question

Are the operations in tf.group() executed in order?

If they are not executed in order, is there a similar operation that makes them run in order? Or is there a clean way to run them in order?

My aim is to run operations A and B many times over and over again, i.e.

sess.run(A)
sess.run(B)
sess.run(A)
sess.run(B)
sess.run(A)
sess.run(B)
sess.run(A)
sess.run(B)
...

Answer 1

The operations do not necessarily execute in order.

Here is a test that proves this:

import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.Variable(1.0)
b = tf.Variable(10.0)
c = tf.Variable(0.0)
grp = tf.group(tf.assign(c, a), tf.assign(c, b)) # this is the group op
for i in range(100):
    sess.run(tf.global_variables_initializer()) # initialize c each time
    sess.run(grp) # run the group op
    print(sess.run(c)) # observe results

When I run this on a cpu, I get that some iterations produce 1.0 and some 10.0.

tf.group does not require the operations to be on the same device, which means that they could not be expected to follow an order.

If you want the operations to execute in order, make sure to build them with control_dependencies

import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.Variable(1.0)
b = tf.Variable(10.0)
c = tf.Variable(0.0)
op1  = tf.assign(c, a)
with tf.get_default_graph().control_dependencies([op1]):
    op2 = tf.assign(c, b) # op2 will execute only after op1
grp = tf.group(op1,op2)
for i in range(100):
    sess.run(tf.global_variables_initializer())
    sess.run(grp)
    print(sess.run(c))