Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Deleting large numbers of files in python

I'm trying to remove all files found in a directory. The accepted answer to Delete Folder Contents in Python suggests getting a list of all files and calling "unlink" on them in a loop.

Suppose I have thousands of files on a network share, and want to tie up the directory for as short a time as possible.

Is it more efficient to delete them all using a shell command like rm -f /path/* or by using shutils.rmtree or some such?

like image 613
Dave Avatar asked Aug 04 '11 17:08

Dave


People also ask

How do you delete multiple files in Python?

To delete multiple files, just loop over your list of files and use the above os. rmdir() function. To delete a folder containing all files you want to remove have to import shutil package. Then you can remove the folder as follows.

How do I delete mass amounts of files?

Navigate to the folder that you want to delete (with all its files and subfolders). Use cd *path*, for example, cd C:\Trash\Files\ to do so. Use cd .. to navigate to the parent folder and run the command RMDIR /Q/S *foldername* to delete the folder and all of its subfolders.


1 Answers

If you actually want to delete the whole directory tree, shutils.rmtree should be faster than os.remove (which is the same as os.unlink). It also allows you to specify a callback function to handle errors.

The suggestion in the comment by @nmichaels is also good, you can os.rename the directory then make a new one in its place and use shutils.rmtree on the original, renamed directory.

like image 116
agf Avatar answered Sep 20 '22 02:09

agf