How to understand sess.as_default() and sess.graph.as_default()?

Question

I read the docs of sess.as_default()

N.B. The default session is a property of the current thread. If you create a new thread, and wish to use the default session in that thread, you must explicitly add a with sess.as_default(): in that thread's function.

My understanding is that if there are two more sessions when a new thread is created, we must set a session to run TensorFlow code in it. So, to do this, a session is chosen and as_default() is called.

N.B. Entering a with sess.as_default(): block does not affect the current default graph. If you are using multiple graphs, and sess.graph is different from the value of tf.get_default_graph, you must explicitly enter a with sess.graph.as_default(): block to make sess.graph the default graph.

In sess.as_default() block, to call a specific graph, one must call sess.graph.as_default() to run the graph?

Answer 1

The tf.Session API mentions that a graph is launched in a session. The following code illustrates this:

import tensorflow as tf

graph1 = tf.Graph()
graph2 = tf.Graph()

with graph1.as_default() as graph:
  a = tf.constant(0, name='a')
  graph1_init_op = tf.global_variables_initializer()

with graph2.as_default() as graph:
  a = tf.constant(1, name='a')
  graph2_init_op = tf.global_variables_initializer()

sess1 = tf.Session(graph=graph1)
sess2 = tf.Session(graph=graph2)
sess1.run(graph1_init_op)
sess2.run(graph2_init_op)

# Both tensor names are a!
print(sess1.run(graph1.get_tensor_by_name('a:0'))) # prints 0
print(sess2.run(graph2.get_tensor_by_name('a:0'))) # prints 1

with sess1.as_default() as sess:
  print(sess.run(sess.graph.get_tensor_by_name('a:0'))) # prints 0

with sess2.as_default() as sess:
  print(sess.run(sess.graph.get_tensor_by_name('a:0'))) # prints 1

with graph2.as_default() as g:
  with sess1.as_default() as sess:
    print(tf.get_default_graph() == graph2) # prints True
    print(tf.get_default_session() == sess1) # prints True

    # This is the interesting line
    print(sess.run(sess.graph.get_tensor_by_name('a:0'))) # prints 0
    print(sess.run(g.get_tensor_by_name('a:0'))) # fails

print(tf.get_default_graph() == graph2) # prints False
print(tf.get_default_session() == sess1) # prints False

You don't need to call sess.graph.as_default() to run the graph, but you need to get the correct tensors or operations in the graph to run it. The context allows you to get the graph or session using tf.get_default_graph or tf.get_default_session.

In the interesting line above, the default session is sess1 and it is implicitly calling sess1.graph, which is the graph in sess1, which is graph1, and hence it prints 0.

In the line following that, it fails because it is trying to run an operation in graph2 with sess1.