Loop Find Command's Output

Question

I'm wanting to issue the find command in Perl and loop through the resulting file paths. I'm trying it like so (but not having any luck):

my $cmd;

open($cmd, '-|', 'find $input_dir -name "*.fastq.gz" -print') or die $!;

while ($line = <$cmd>) {
   print $line;
}

close $cmd;

Any ideas?

Thanks

Answer 1

If you were to do

print 'find $input_dir -name "*.fastq.gz" -print';

The problem should become obvious: Single-quotes don't interpolate. You probably meant to do

open(my $cmd_fh, '-|', qq{find $input_dir -name "*.fastq.gz" -print}) or die $!;

but that's buggy too. You don't convert $input_dir into a shell literal. Two solutions present themselves.

use String::ShellQuote qw( shell_quote );

my $cmd = shell_quote("find", $input_dir, "-name", "*.fastq.gz", "-print");
open(my $cmd_fh, '-|', $cmd) or die $!;

Or

my @cmd = ("find", $input_dir, "-name", "*.fastq.gz", "-print");
open(my $cmd_fh, '-|', @cmd) or die $!;

Answer 2

You're not applying enough escaping to the * character. Prepending a \ should fix it.

It's better not to invoke the shell in the first place, by separating the arguments:

use warnings;
use strict;

open(my $cmd, '-|', 'find', $input_dir, '-name' ,'*.fastq.gz', '-print') or die $!;

while (my $line = <$cmd>) {
   print $line;
}

close $cmd;

Answer 3

Your problem seems to be using single quotes. Your variable will not be interpolated, but the variable name will be fed to find as-is.

But why not use File::Find?

> perl -MFile::Find -lwe '
    $foo = "perl"; 
    find ( sub { /\.pl$/i or return; print $File::Find::name }, $foo);'
perl/foo.pl
perl/parsewords.pl
perl/yada.pl

Here, the wanted subroutine is simply a pattern match against the file name. We exit (return from) the subroutine unless the extension is .pl, else we print the file name with the relative path.

Answer 4

To read the output of a command, use the backtick operator.

my $command = "find $inputdir ...";  # interpolate the input directory
my $output  = `$command`;            # be careful here
my @lines   = split /\n/ => $output; # split in single lines

for my $line (@lines) {              # iterate
    # do something with $line
}

I think it's much better readable than piping. The downside is that it blocks, so if you want to process huge output strings with lots of lines, the pipe approach may be better.

But you may want to use an appropriate module. File::Find (core module) should fit your needs.