Is it necessary to close session after tensorflow InteractiveSession()

Question

I have a question about InteractiveSession in Tensorflow

I know tf.InteractiveSession() is just convenient syntactic sugar for keeping a default session open and basically work the same like below:

with tf.Session() as sess:
    # Do something

However, I have seen some examples online, they did't call close() at the end of the code after using InteractiveSession.

Question:
1. Would it caused any problem without closing the session like session leak?
2. How the GC work for the InteractiveSession if we don't close it?

Answer 1

Yes, tf.InteractiveSession is just convenient syntactic sugar for keeping a default session open.

The Session implementation has a comment

Calling this method frees all resources associated with the session.

A quick test

#! /usr/bin/env python
# -*- coding: utf-8 -*-


import argparse
import tensorflow as tf
import numpy as np


def open_interactive_session():
    A = tf.Variable(np.random.randn(16, 255, 255, 3).astype(np.float32))
    sess = tf.InteractiveSession()
    sess.run(tf.global_variables_initializer())


def open_and_close_interactive_session():
    A = tf.Variable(np.random.randn(16, 255, 255, 3).astype(np.float32))
    sess = tf.InteractiveSession()
    sess.run(tf.global_variables_initializer())
    sess.close()


def open_and_close_session():
    A = tf.Variable(np.random.randn(16, 255, 255, 3).astype(np.float32))
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--num', help='repeat', type=int, default=5)
    parser.add_argument('type', choices=['interactive', 'interactive_close', 'normal'])
    args = parser.parse_args()

    sess_func = open_and_close_session

    if args.type == 'interactive':
        sess_func = open_interactive_session
    elif args.type == 'interactive_close':
        sess_func = open_and_close_interactive_session

    for _ in range(args.num):
        sess_func()
    with tf.Session() as sess:
        print("bytes used=", sess.run(tf.contrib.memory_stats.BytesInUse()))

gives

"""
python example_session2.py interactive
('bytes used=', 405776640)
python example_session2.py interactive_close
('bytes used=', 7680)
python example_session2.py
('bytes used=', 7680)
"""

This provokes a session-leak, when not closing the session.Note, even when closing the session, there is currently bug in TensorFlow which keep 1280 bytes per session see Tensorflow leaks 1280 bytes with each session opened and closed?. (This has been fixed now).

Further, there is some logic in the __del__ trying to start the GC.

Interestingly, I never saw the warning

An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s)

which seems to be implemented. It guess the only raison d'être of the InteractiveSession is its usage in Jupyter Notebookfiles or inactive shells in combination with .eval(). But I advised against using eval (see Official ZeroOut gradient example error: AttributeError: 'list' object has no attribute 'eval')

However, I have seen some examples online, they did't call close() at the end of the code after using InteractiveSession.

And I am not surprised by that. Guess how many code snippets are the without a free or delete after some malloc. Bless the OS that it frees up the memory.