Why is __del__ called at the end of a with block?

Question

The scope of the variables created in a with statement is outside the with block (refer: Variable defined with with-statement available outside of with-block?). But when I run the following code:

class Foo:
    def __init__(self):
        print "__int__() called."

    def __del__(self):
        print "__del__() called."

    def __enter__(self):
        print "__enter__() called."
        return "returned_test_str"

    def __exit__(self, exc, value, tb):
        print "__exit__() called."

    def close(self):
        print "close() called."

    def test(self):
        print "test() called."

if __name__ == "__main__":
    with Foo() as foo:
        print "with block begin???"
        print "with block end???"

    print "foo:", foo  # line 1

    print "-------- Testing MySQLdb -----------------------"
    with MySQLdb.Connect(host="xxxx", port=0, user="xxx", passwd="xxx", db="test") as my_curs2:
        print "(1)my_curs2:", my_curs2
        print "(1)my_curs2.connection:", my_curs2.connection
    print "(2)my_curs2.connection:", my_curs2.connection
    print "(2)my_curs2.connection.open:", my_curs2.connection.open  # line 2

The output shows that Foo.__del__ is called before printing foo (at # line 1 above):

__int__() called.
__enter__() called.
with block begin???
with block end???
__exit__() called.
__del__() called.
foo: returned_test_str
-------- Testing MySQLdb -----------------------
(1)my_curs2: <MySQLdb.cursors.Cursor object at 0x7f16dc95b290>
(1)my_curs2.connection: <_mysql.connection open to 'xxx' at 2609870>
(2)my_curs2.connection: <_mysql.connection open to 'xxx' at 2609870>
(2)my_curs2.connection.open: 1

My question is, why is Foo.__del__ called here, if the with statement does not create a new execution scope?

Also, if the connection's __del__ method is called in the second with block, I don't understand why my_curs1.connection is still open afterward (see # line 2 above).

Answer 1

It's important to note that foo is not an object of type Foo. You do create a Foo and need to keep it around because it might contain state information needed to call __exit__. But once that's done, the object is unneeded and Python's free to throw it away.

Put another way, this:

with Foo() as foo:
    print ('Hello World!')

Is the same as this:

_bar = Foo()
foo = _bar.__enter__()
print ('Hello World!')
_bar.__exit__()
del _bar # This will call __del__ because _bar is the only reference

The behavior you are expecting would happen if foo were a reference to the with block's foo. For example...

class Foo:
    def __init__(self):
        print ("__int__() called.")

    def __del__(self):
        print ("__del__() called.")

    def __enter__(self):
        print ("__enter__() called.")
        return self # foo now stores the Foo() object

    def __str__(self):
        return 'returned_test_str'

    def __exit__(self, exc, value, tb):
        print ("__exit__() called.")

    def close(self):
        print ("close() called.")

    def test(self):
        print ("test() called.")

if __name__ == "__main__":
    with Foo() as foo:
        print ("with block begin???")
        print ("with block end???")

    print ("foo:", foo)  # line 1

Prints

__int__() called.
__enter__() called.
with block begin???
with block end???
__exit__() called.
foo: returned_test_str
__del__() called.

---

I have no idea why Connection.__exit__ would leave its cursors open however.