Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is python's shutil.copyfile() atomic?

I'm writing a python script that copies a file using shutil.copyfile() on Linux. During the copying, other processes may be trying to read the file. Is the following sufficient to ensure that an external process doesn't get a corrupted view of the file?

os.unlink(dest)
shutil.copyfile(src, dest)

That is, is shutil.copyfile() atomic such that other processes cannot read the destination file until after the copy operation is complete?

like image 522
dinosaur Avatar asked Jan 01 '14 21:01

dinosaur


1 Answers

No, shutil.copyfile is not atomic. This is part of the definition of shutil.copyfile:

def copyfile(src, dst, *, follow_symlinks=True):    
    ...
    with open(src, 'rb') as fsrc:
        with open(dst, 'wb') as fdst:
            copyfileobj(fsrc, fdst)

where copyfileobj is defined like this:

def copyfileobj(fsrc, fdst, length=16*1024):
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)

The thread calling copyfile could be stopped inside this while-loop at which point some other process could try to open the file to be read. It would get a corrupted view of the file.

like image 103
unutbu Avatar answered Sep 24 '22 12:09

unutbu