Is running a bash command to copy paste a file in Python3 a reliable option?

Question

Copying a file in Python3 using the following code takes a lot of time:

shutil.copy(self.file, self.working_dir)

However, the cp command of Linux is pretty fast. If I try to execute the bash command from Python3 for copying files with sizes greater than 100GB, will that be a reliable option for production servers?

I have seen this answer but its suggestions are not fast.

Answer 1

If you are running on Windows, Python's copy buffer size may be too small: https://stackoverflow.com/a/28584857/679240

You would need to implement something similar to this (warning: untested):

def copyfile_largebuffer(src, dst, length=16*1024*1024):
    with open(newfile, 'wb') as outfile, open(oldfile, 'rb') as infile:
        copyfileobj_largebuffer(infile, outfile, length=length)

def copyfileobj_largebuffer(fsrc, fdst, length=16*1024*1024):
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)