Cannot use the given session to evaluate tensor: the tensor's graph is different from the session's graph

Question

I just don't know what's the problem...I previously tried InteractiveSession() and passing an explicit session , but this error is just not getting resolved ... I am new to tensorflow ... please help.

cost=-tf.reduce_sum(y*tf.log(y_))
train_step=tf.train.AdamOptimizer(LEARNING_RATE).minimize(cost)
correct_pred=tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, 'float'))
predict=tf.argmax(y,1)

And here is my session

train_accuracies = []
validation_accuracies = []
x_range = []

num_examples=train_images.shape[0]
init=tf.global_variables_initializer()
minibatches=random_mini_batches(train_images,train_labels,
                            mini_batch_size = BATCH_SIZE)
display_step=1
init = tf.initialize_all_variables()
with tf.Session().as_default() as sess:
sess.run(init)
for epoch in range(TRAINING_ITERATIONS):
    for minibatch in minibatches:
        (minibatch_X,minibatch_Y)=minibatch
        if epoch%display_step == 0 or (epoch+1) == TRAINING_ITERATIONS:

            train_accuracy = accuracy.eval(session=sess,feed_dict={x:minibatch_X, 
                                                      y: minibatch_Y, 
                                                      keep_prob: 1.0})       
        if(VALIDATION_SIZE):
            validation_accuracy = accuracy.eval(session=sess,feed_dict={ x: validation_images[0:BATCH_SIZE], 
                                                            y: validation_labels[0:BATCH_SIZE], 
                                                            keep_prob: 1.0})                                  
            print('training_accuracy / validation_accuracy => %.2f / %.2f for step %d'%(train_accuracy, validation_accuracy, epoch))

            validation_accuracies.append(validation_accuracy)

        else:
             print('training_accuracy => %.4f for step %d'%(train_accuracy, epoch))
        train_accuracies.append(train_accuracy)
        x_range.append(epoch)

        # increase display_step
        if epoch%(display_step*10) == 0 and epoch:
            display_step *= 10
    # train on batch
    sess.run(train_step, feed_dict={x: minibatch_X, y:minibatch_Y, keep_prob: DROPOUT})

And following error is getting generated

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-63-910bbc0840b2> in <module>
      18                 train_accuracy = accuracy.eval(session=sess,feed_dict={x:minibatch_X, 
      19                                                           y: minibatch_Y,
 ---> 20                                                           keep_prob: 1.0})       
      21             if(VALIDATION_SIZE):
      22                 validation_accuracy = accuracy.eval(session=sess,feed_dict={ x: 
      validation_images[0:BATCH_SIZE], 

      /opt/conda/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py in eval(self, 
     feed_dict, session)
      788 
      789     """
  --> 790     return _eval_using_default_session(self, feed_dict, self.graph, session)
      791 
      792   def experimental_ref(self):

      /opt/conda/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py in 
     _eval_using_default_session(tensors, feed_dict, graph, session)
      5307   else:
      5308     if session.graph is not graph:
   -> 5309       raise ValueError("Cannot use the given session to evaluate tensor: "
      5310                        "the tensor's graph is different from the session's "
      5311                        "graph.")

      ValueError: Cannot use the given session to evaluate tensor: the tensor's graph is different 
      from the session's graph.

Please suggest how to work with two sessions and how to resolve this issue. And major issue is that I tried passing the session as eval(session=sess) but it is not working. It is saying that the computational graph that I am using is different from the accuracy tensor's graph

Answer 1

I have recreated the error caused because of possible ways and also have provided the fix.

Have provided more comments in the code to be more clear about the error and fix.

Note - I have used same code with little tweaks to recreate the possibility of cause of the errors and fix for the same.

The best fix code is present at the end of this answer.

Error Code 1 - Error with default session and using variable created in another graph

%tensorflow_version 1.x
import tensorflow as tf

g = tf.Graph()
with g.as_default():
  x = tf.constant(1.0)  # x is created in graph g

with tf.Session().as_default() as sess:
  y = tf.constant(2.0) # y is created in TensorFlow's default graph!!!
  print(y.eval(session=sess)) # y was created in TF's default graph, and is evaluated in
                  # default session, so everything is ok.  
  print(x.eval(session=sess)) # x was created in graph g and it is evaluated in session s
                  # which is tied to graph g, but it is evaluated in
                  # session s which is tied to graph g => ERROR

Output -

2.0
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-f35cb204cf59> in <module>()
     10   print(y.eval(session=sess)) # y was created in TF's default graph, and is evaluated in
     11                   # default session, so everything is ok.
---> 12   print(x.eval(session=sess)) # x was created in graph g and it is evaluated in session s
     13                   # which is tied to graph g, but it is evaluated in
     14                   # session s which is tied to graph g => ERROR

1 frames
/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py in _eval_using_default_session(tensors, feed_dict, graph, session)
   5402   else:
   5403     if session.graph is not graph:
-> 5404       raise ValueError("Cannot use the given session to evaluate tensor: "
   5405                        "the tensor's graph is different from the session's "
   5406                        "graph.")

ValueError: Cannot use the given session to evaluate tensor: the tensor's graph is different from the session's graph.

Error Code 2 - Error with graph session as default and using variable created in default graph

%tensorflow_version 1.x
import tensorflow as tf

g = tf.Graph()
with g.as_default():
  x = tf.constant(1.0)  # x is created in graph g

with tf.Session(graph=g).as_default() as sess:
  print(x.eval(session=sess)) # x was created in graph g and it is evaluated in session s
                         # which is tied to graph g, so everything is ok.
  y = tf.constant(2.0) # y is created in TensorFlow's default graph!!!
  print(y.eval()) # y was created in TF's default graph, but it is evaluated in
                  # session s which is tied to graph g => ERROR

Output -

1.0
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-6b8b687c5178> in <module>()
     10                          # which is tied to graph g, so everything is ok.
     11   y = tf.constant(2.0) # y is created in TensorFlow's default graph!!!
---> 12   print(y.eval()) # y was created in TF's default graph, but it is evaluated in
     13                   # session s which is tied to graph g => ERROR

1 frames
/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py in _eval_using_default_session(tensors, feed_dict, graph, session)
   5396                        "`eval(session=sess)`")
   5397     if session.graph is not graph:
-> 5398       raise ValueError("Cannot use the default session to evaluate tensor: "
   5399                        "the tensor's graph is different from the session's "
   5400                        "graph. Pass an explicit session to "

ValueError: Cannot use the default session to evaluate tensor: the tensor's graph is different from the session's graph. Pass an explicit session to `eval(session=sess)`.

Error Code 3 - As suggested in Error Code 2 - output, to Pass an explicit session to eval(session=sess). Lets try this.

%tensorflow_version 1.x
import tensorflow as tf

g = tf.Graph()
with g.as_default():
  x = tf.constant(1.0)  # x is created in graph g

with tf.Session(graph=g).as_default() as sess:
  print(x.eval(session=sess)) # x was created in graph g and it is evaluated in session s
                         # which is tied to graph g, so everything is ok.
  y = tf.constant(2.0) # y is created in TensorFlow's default graph!!!
  print(y.eval(session=sess)) # y was created in TF's default graph, but it is evaluated in
                  # session s which is tied to graph g => ERROR

Output -

1.0
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-83809aa4e485> in <module>()
     10                          # which is tied to graph g, so everything is ok.
     11   y = tf.constant(2.0) # y is created in TensorFlow's default graph!!!
---> 12   print(y.eval(session=sess)) # y was created in TF's default graph, but it is evaluated in
     13                   # session s which is tied to graph g => ERROR

1 frames
/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py in _eval_using_default_session(tensors, feed_dict, graph, session)
   5402   else:
   5403     if session.graph is not graph:
-> 5404       raise ValueError("Cannot use the given session to evaluate tensor: "
   5405                        "the tensor's graph is different from the session's "
   5406                        "graph.")

ValueError: Cannot use the given session to evaluate tensor: the tensor's graph is different from the session's graph.

Fix 1 - Fix with default session and variable not assigned to any graph

%tensorflow_version 1.x
import tensorflow as tf

x = tf.constant(1.0)  # x is in not assigned to any graph

with tf.Session().as_default() as sess:
  y = tf.constant(2.0) # y is created in TensorFlow's default graph!!!
  print(y.eval(session=sess)) # y was created in TF's default graph, and is evaluated in
                  # default session, so everything is ok.  
  print(x.eval(session=sess)) # x not assigned to any graph, and is evaluated in
                  # default session, so everything is ok.  

Output -

2.0
1.0

Fix 2 - The best fix is to cleanly separate the construction phase and the execution phase.

import tensorflow as tf

g = tf.Graph()
with g.as_default():
  x = tf.constant(1.0)  # x is created in graph g
  y = tf.constant(2.0) # y is created in graph g

with tf.Session(graph=g).as_default() as sess:
  print(x.eval()) # x was created in graph g and it is evaluated in session s
                         # which is tied to graph g, so everything is ok.
  print(y.eval()) # y was created in graph g and it is evaluated in session s
                         # which is tied to graph g, so everything is ok.

Output -

1.0
2.0