What's the best strategy to delete a very huge folder using Perl?

Question

I need to delete all content (files and folders) under a given folder. The problems is the folder has millions of files and folders inside it. So I don't want to load all the file names in one go.

Logic should be like this:

• iterate a folder without load everything
• get a file or folder
• delete it (verbose that the file or folder "X" was deleted)
• go to the next one

I'm trying something like this:

sub main(){
  my ($rc, $help, $debug, $root)   = ();
  $rc = GetOptions ( "HELP"           => \$help,
                     "DEBUG"          => \$debug,
                     "ROOT=s"         => \$root);

  die "Bad command line options\n$usage\n" unless ($rc);
  if ($help) { print $usage; exit (0); }

  if ($debug) {
      warn "\nProceeding to execution with following parameters: \n";
      warn "===============================================================\n";
      warn "ROOT = $root\n";

  } # write debug information to STDERR

  print "\n Starting to delete...\n";  

  die "usage: $0 dir ..\n" unless $root;
  *name = *File::Find::name;
  find \&verbose, @ARGV;

}

sub verbose {
    if (!-l && -d _) {
        print "rmdir $name\n";
    } else {
        print "unlink $name\n";
    }
}

main();

It's working fine, but whenever "find" reads the huge folder, the application gets stuck and I can see the system memory for Perl increasing until timeout. Why? Is it trying to load all the files in one go?

Thanks for your help.

Answer 1

The remove_tree function from File::Path can portably and verbosely remove a directory hierarchy, keeping the top directory, if desired.

use strict;
use warnings;
use File::Path qw(remove_tree);

my $dir = '/tmp/dir';
remove_tree($dir, {verbose => 1, keep_root => 1});

Pre-5.10, use the rmtree function from File::Path. If you still want the top directory, you could just mkdir it again.

use File::Path;

my $dir = '/tmp/dir';
rmtree($dir, 1);  # 1 means verbose
mkdir $dir;