Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Check if a file is not open nor being used by another process

Tags:

python

file

I my application, i have below requests: 1. There has one thread will regularly record some logs in file. The log file will be rollovered in certain interval. for keeping the log files small. 2. There has another thread also will regularly to process these log files. ex: Move the log files to other place, parse the log's content to generate some log reports.

But, there has a condition is the second thread can not process the log file that's using to record the log. in code side, the pseudocode similars like below:

#code in second thread to process the log files for logFile in os.listdir(logFolder):      if not file_is_open(logFile) or file_is_use(logFile):           ProcessLogFile(logFile) # move log file to other place, and generate log report.... 

So, how do i check is a file is already open or is used by other process? I did some research in internet. And have some results:

try:    myfile = open(filename, "r+") # or "a+", whatever you need except IOError:     print "Could not open file! Please close Excel!" 

I tried this code, but it doesn't work, no matter i use "r+" or "a+" flag

try:    os.remove(filename) # try to remove it directly except OSError as e:     if e.errno == errno.ENOENT: # file doesn't exist         break 

This code can work, but it can not reach my request, since i don't want to delete the file to check if it is open.

like image 299
zengwke Avatar asked Jun 20 '12 07:06

zengwke


People also ask

How do you check if a file is open by another process in Python?

The open() function is used in Python to open a file. Using the open() function is one way to check a particular file is opened or closed. If the open() function opens a previously opened file, then an IOError will be generated.

How can you check that whether a file is opened successfully or not explain?

Check to make sure the file was successfully opened by checking to see if the variable == NULL. If it does, an error has occured. Use the fprintf or fscanf functions to write/read from the file. Usually these function calls are placed in a loop.


1 Answers

An issue with trying to find out if a file is being used by another process is the possibility of a race condition. You could check a file, decide that it is not in use, then just before you open it another process (or thread) leaps in and grabs it (or even deletes it).

Ok, let's say you decide to live with that possibility and hope it does not occur. To check files in use by other processes is operating system dependant.

On Linux it is fairly easy, just iterate through the PIDs in /proc. Here is a generator that iterates over files in use for a specific PID:

def iterate_fds(pid):     dir = '/proc/'+str(pid)+'/fd'     if not os.access(dir,os.R_OK|os.X_OK): return      for fds in os.listdir(dir):         for fd in fds:             full_name = os.path.join(dir, fd)             try:                 file = os.readlink(full_name)                 if file == '/dev/null' or \                   re.match(r'pipe:\[\d+\]',file) or \                   re.match(r'socket:\[\d+\]',file):                     file = None             except OSError as err:                 if err.errno == 2:                          file = None                 else:                     raise(err)              yield (fd,file) 

On Windows it is not quite so straightforward, the APIs are not published. There is a sysinternals tool (handle.exe) that can be used, but I recommend the PyPi module psutil, which is portable (i.e., it runs on Linux as well, and probably on other OS):

import psutil  for proc in psutil.process_iter():     try:         # this returns the list of opened files by the current process         flist = proc.open_files()         if flist:             print(proc.pid,proc.name)             for nt in flist:                 print("\t",nt.path)      # This catches a race condition where a process ends     # before we can examine its files         except psutil.NoSuchProcess as err:         print("****",err)  
like image 89
cdarke Avatar answered Oct 16 '22 05:10

cdarke