Get MD5 Checksum for Very Large Files

Question

I've written a script that reads through all files in a directory and returns md5 hash for each file. However, it renders nothing for a rather large file. I assume that the interpreter has some value set for maximum processing time, and since it takes too long to get this value, it just skips along to other files. Is there anyway to get an md5 checksum for large files through PHP? If not, could it be done through a chron job with cpanel? I gave it a shot there but it doesn't seem that my md5sum command has ever been processed: I never get an email with the hash. Here's the PHP I've already written. It's a very simple code and works file for files of a reasonable size:

function md5_dir($dir) {
    if (is_dir($dir)) {
        if ($dh = opendir($dir)) {
            while (($file = readdir($dh)) !== false) {
                echo nl2br($file . "\n" . md5_file($file) . "\n\n");
            }
            closedir($dh);
        }
    }
}

Answer 1

Make sure to use escapeshellarg ( http://us3.php.net/manual/en/function.escapeshellarg.php ) if you decide to use a shell_exec() or system() call. I.e.,

shell_exec('md5sum -b ' . escapeshellarg($filename));

Answer 2

While i couldn't reproduce it with PHP 5.2 or 5.3 with a 2GB file the issue seems to come up on 32bit PHP builds.

Even so it's not a really nice solution you could try to let the system to the hasing

echo system("md5sum test.txt");

46d6a7bcbcf7ae0501da341cb3bae27c test.txt