Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Explicit way to close file in Python

Tags:

python

file

Please look at the following code:

for i in xrange(1,5000):
    with open(r'test\test%s.txt' % i, 'w') as qq:
        qq.write('aa'*3000)

It seems to be written according to all Python rules; files are closing after using. Seems to. But in fact it seems to recommend(!) system to close file, not to close it explicitly because when I'm looking on Resource monitor it shows a lot of open files . It gives me a lot of problems because in my script I use a lot of files and after a long time I got "Too many open files" error despite of 'closing' it from source code.

Is there some way to explicitly close file in Python? Or how can I check whether the file was really(!) closed or not?

Update: I've just tried with another monitoring tool - Handle from Sysinternals and it shows all correct and I trust it. So, it may be problem in Resource monitor itself.

Screenshot which shows files opened:

Resource Monitor with the script running

like image 509
The Godfather Avatar asked Aug 07 '14 07:08

The Godfather


People also ask

How do you close a file in Python?

The close() method of a file object flushes any unwritten information and closes the file object, after which no more writing can be done. Python automatically closes a file when the reference object of a file is reassigned to another file. It is a good practice to use the close() method to close a file.

Which file is used to close a file in Python?

Python File close() Method Python file method close() closes the opened file. A closed file cannot be read or written any more. Any operation, which requires that the file be opened will raise a ValueError after the file has been closed.

Do I need to close file in Python?

You've learned why it's important to close files in Python. Because files are limited resources managed by the operating system, making sure files are closed after use will protect against hard-to-debug issues like running out of file handles or experiencing corrupted data.

What is the syntax to close a file?

Syntax : int fclose( FILE * stream ); Return valu e : Here fclose() function closes the file and returns zero on success, or EOF if there is an error in closing the file.


1 Answers

Your code

for i in xrange(1, 5000):
    with open(r'test\test%s.txt' % i, 'w') as qq:
        qq.write('aa' * 3000)

is semantically exactly equivalent to

for i in xrange(1, 5000):
    qq = open(r'test\test%s.txt' % i, 'w')
    try:
        qq.write('aa' * 3000)
    finally:
        qq.close()

as using with with files is a way to ensure that the file is closed immediately after the with block is left.

So your problem must be somewhere else.

Maybe the version of the Python environment in use has a bug where fclose() isn't called due to some reason.

But you might try something like

try:
    qq.write('aa' * 3000)
finally:
    # qq.close()
    os.close(qq.fileno())

which does the system call directly.

like image 141
glglgl Avatar answered Sep 29 '22 03:09

glglgl