How do I serve a large file for download with Perl?

Question

I need to serve a large file (500+ MB) for download from a location that is not accessible to the web server. I found the question Serving large files with PHP, which is identical to my situation, but I'm using Perl instead of PHP.

I tried simply printing the file line by line, but this does not cause the browser to prompt for download before grabbing the entire file:

use Tie::File;

open my $fh, '<', '/path/to/file.txt';
tie my @file, 'Tie::File', $fh
    or die 'Could not open file: $!';
my $size_in_bytes = -s $fh;
print "Content-type: text/plain\n";
print "Content-Length: $size_in_bytes\n";
print "Content-Disposition: attachment; filename=file.txt\n\n";
for my $line (@file) {
    print $line;
}
untie @file;
close $fh;
exit;

Does Perl have an equivalent to PHP's readfile() function (as suggested with PHP) or is there a way to accomplish what I'm trying to do here?

Answer 1

If you just want to slurp input to output, this should do the trick.

use Carp ();

{ #Lexical For FileHandle and $/ 
  open my $fh, '<' , '/path/to/file.txt' or Carp::croak("File Open Failed");
  local $/ = undef; 
  print scalar <$fh>; 
  close $fh or Carp::carp("File Close Failed");
}

I guess in response to the "Does Perl have a PHP ReadFile Equivelant" , and I guess my answer would be "But it doesn't really need one".

I've used PHP's manual File IO controls and they're a pain, Perls are just so easy to use by comparison that shelling out for a one-size-fits-all function seems over-kill.

Also, you might want to look at X-SendFile support, and basically send a header to your webserver to tell it what file to send: http://john.guen.in/past/2007/4/17/send_files_faster_with_xsendfile/ ( assuming of course it has permissions enough to access the file, but the file is just NOT normally accessible via a standard URI )

Edit Noted, it is better to do it in a loop, I tested the above code with a hard-drive and it does implicitly try store the whole thing in an invisible temporary variable and eat all your ram.

Alternative using blocks

The following improved code reads the given file in blocks of 8192 chars, which is much more memory efficient, and gets a throughput respectably comparable with my disk raw read rate. ( I also pointed it at /dev/full for fits and giggles and got a healthy 500mb/s throughput, and it didn't eat all my rams, so that must be good )

{ 
    open my $fh , '<', '/dev/sda' ; 
    local $/ = \8192; # this tells IO to use 8192 char chunks. 
    print $_ while defined ( $_ = scalar <$fh> ); 
    close $fh; 
}

Applying jrockways suggestions

{ 
    open my $fh , '<', '/dev/sda5' ; 
    print $_ while ( sysread $fh, $_ , 8192 ); 
    close $fh; 
}

This literally doubles performance, ... and in some cases, gets me better throughput than DD does O_o.

Answer 2

The readline function is called readline (and can also be written as <>).

I'm not sure what problem you're having. Perhaps that for loops aren't lazily evaluated (which they're not). Or, perhaps Tie::File is screwing something up? Anyway, the idiomatic Perl for reading a file a line at a time is:

open my $fh, '<', $filename or die ...;
while(my $line = <$fh>){
   # process $line
}

No need to use Tie::File.

Finally, you should not be handling this sort of thing yourself. This is a job for a web framework. If you were using Catalyst (or HTTP::Engine), you would just say:

open my $fh, '<', $filename ...
$c->res->body( $fh );

and the framework would automatically serve the data in the file efficiently. (Using stdio via readline is not a good idea here, it's better to read the file in blocks from the disk. But who cares, it's abstracted!)