Asynchronously running perl's backticks

Question

Right now I have a perl script that at a certain point, gathers and then processes the output of several bash commands, right now here is how I've done it:

if ($condition) {
  @output = `$bashcommand`;
  @output1 = `$bashcommand1`;
  @output2 = `$bashcommand2`;
  @output3 = `$bashcommand3`;
}

The problem is, each of these commands take a fairly amount of time, therefore, I'd like to know if I could run them all at the same time.

Answer 1

On a Unix system, you should be able to open multiple command pipes, then run a loop calling IO::Select to wait for any of them to be ready to read; keep reading and slurping their output (with sysread) until they all reach end of file.

Unfortunately, apparently Win32 emulation of Unix select can't handle file I/O, so to pull it off on Windows you'll also have to add a layer of socket I/O, for which select works, see perlmonks.

Answer 2

This sounds like a good use case for Forks::Super::bg_qx.

use Forks::Super 'bg_qx';
$output = bg_qx $bashcommand;
$output1 = bg_qx $bashcommand1;
$output2 = bg_qx $bashcommand2;
$output3 = bg_qx $bashcommand3;

will run these four commands in the background. The variables used for return values ($output, $output1, etc.) are overloaded objects. Your program will retrieve the output from these commands (waiting for the commands to complete, if necessary) the next time those variables are referenced in the program.

... more stuff happens ...
# if $bashcommand is done, this next line will execute right away
# otherwise, it will wait until $bashcommand finishes ...
print "Output of first command was ", $output;

&do_something_with_command_output( $output1 );
@output2 = split /\n/, $output2;
...

Update 2012-03-01: v0.60 of Forks::Super has some new constructions that let you retrieve results in list context:

if ($condition) {
    tie @output, 'Forks::Super::bg_qx', $bashcommand;
    tie @output1, 'Forks::Super::bg_qx', $bashcommand1;
    tie @output2, 'Forks::Super::bg_qx', $bashcommand2;
    tie @output3, 'Forks::Super::bg_qx', $bashcommand3;
}
...