Why does defining tf.Session with and without context manager in Tensorflow result in different behaviour?

Question

I noticed that there is a difference when you define session with and without context manager. Here there is an example:

With context manager:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    x = tf.Variable(0)
    tf.summary.scalar("x", x)

with tf.Session(graph=graph) as sess:
    summaries = tf.summary.merge_all()
    print("Operations:", sess.graph.get_operations())
    print("\nSummaries:", summaries)

Results in:

Operations: [<tf.Operation 'Variable/initial_value' type=Const>, <tf.Operation 'Variable' type=VariableV2>, <tf.Operation 'Variable/Assign' type=Assign>, <tf.Operation 'Variable/read' type=Identity>, <tf.Operation 'x/tags' type=Const>, <tf.Operation 'x' type=ScalarSummary>, <tf.Operation 'Merge/MergeSummary' type=MergeSummary>]

Summaries: Tensor("Merge/MergeSummary:0", shape=(), dtype=string)

Without context manager:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    x = tf.Variable(0)
    tf.summary.scalar("x", x)

sess = tf.Session(graph=graph)
summaries = tf.summary.merge_all()
print("Operations:", sess.graph.get_operations())
print("Summaries:", summaries)
sess.close()

Results in:

Operations: [<tf.Operation 'Variable/initial_value' type=Const>, <tf.Operation 'Variable' type=VariableV2>, <tf.Operation 'Variable/Assign' type=Assign>, <tf.Operation 'Variable/read' type=Identity>, <tf.Operation 'x/tags' type=Const>, <tf.Operation 'x' type=ScalarSummary>]
Summaries: None

Why tf.summary.merge_all() does not find the summary?

Answer 1

You can find the implementation of tf.summary.merge_all() here. It works by calling this function, which gets the collection from the graph returned by get_default_graph(). The documentation for that function is as follows:

"""Returns the default graph for the current thread.

The returned graph will be the innermost graph on which a
`Graph.as_default()` context has been entered, or a global default
graph if none has been explicitly created.

NOTE: The default graph is a property of the current thread. If you
create a new thread, and wish to use the default graph in that
thread, you must explicitly add a `with g.as_default():` in that
thread's function.

Returns:
    The default `Graph` being used in the current thread.
"""

So, in your code without the session context manager, the problem is not necessarily the fact that you're not in a session; the problem is that the graph with the summary is not the default graph, and you have not entered a context (like the session) with that graph.

There are a few different ways to "solve" this without using the with tf.Session(graph=graph) as sess: context manager:

---

One option is to merge the summaries together while you still have graph as the default graph:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    x = tf.Variable(0)
    tf.summary.scalar("x", x)
    summaries = tf.summary.merge_all()

with tf.Session(graph=graph) as sess:
    print("Operations:", sess.graph.get_operations())
    print("\nSummaries:", summaries)

---

Another option is to explicitly __enter__() the session before merging the summaries (this is pretty much identical to what happens internally in python in the with tf.Session(graph=graph) as sess: statement):

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    x = tf.Variable(0)
    tf.summary.scalar("x", x)

sess = tf.Session(graph=graph)
sess.__enter__()
summaries = tf.summary.merge_all()
print("Operations:", sess.graph.get_operations())
print("Summaries:", summaries)
sess.close()