Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it worth closing files in small functions?

Say you have:

def my_func():
    fh = open(...)
    try:
        print fh.read()
    finally:
        fh.close()

My first question is: Is it worth having the try/finally (or with) statement? Isn't the file closed anyway when the function terminates (via garbage collection)?

I came across this after reading a recipe form Martelli's "python cookbook" where

all_the_text = open('thefile.txt').read()

comes with the comment: "When you do so, you no longer have a reference to the file object as soon as the reading operation finishes. In practice, Python notices the lack of a reference at once, and immediately closes the file."

My function example is almost the same. You do have a reference, it's just that the reference has a very short life.

My second question is: What does "immediately" in Martelli's statement mean? Even though you don't have a reference at all, doesn't the file closing happen at garbage collection time anyway?

like image 292
Ioan Alexandru Cucu Avatar asked Jul 10 '12 08:07

Ioan Alexandru Cucu


People also ask

Why should you close files in your program?

You've learned why it's important to close files in Python. Because files are limited resources managed by the operating system, making sure files are closed after use will protect against hard-to-debug issues like running out of file handles or experiencing corrupted data.

Do I need to close file descriptors?

Yes, close your file descriptors and free all heap memory, even if you know that the OS will clean it up - that way, when you run valgrind or some similar tool, you don't get a lot of noise in the results, and you can easily recognize "legit" fd leaks.

What happens if you don't close file?

As long as your program is running, if you keep opening files without closing them, the most likely result is that you will run out of file descriptors/handles available for your process, and attempting to open more files will fail eventually.

Why is it important to close files and streams?

When you open a file using any programming language , that is actually a request to operating system to access the file. When such a request is made , resources are allocated by the OS. When you gracefully close those files, those resources are set free.


1 Answers

It is good practice to close the file yourself. Using the with statement leads to clean code and it automatically closes the file (which is a Good Thing).

Even though Python is a high-level programming language, you still need to be in control of what you're doing. As a rule of thumb: if you open a file, it also needs to be closed. There's never a good reason to be sloppy in your code :-)

Regarding your second question: it won't run immediately, it'll run when the garbage collector decides it is time to run. When the file object is deallocated Python will close the file. Here are some articles on garbage collection in Python (also see the gc module), it's an interesting read.

It also shows that Python's garbage collection uses a threshold based on the number of allocated and deallocated objects before it decides to garbage collect. If your file is big then Python might hold the file open longer than necessary because the garbage collection code might not have run yet.

like image 122
Simeon Visser Avatar answered Oct 14 '22 15:10

Simeon Visser