Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unzipping larger files with PHP

Tags:

php

unzip

I'm trying to unzip a 14MB archive with PHP with code like this:

    $zip = zip_open("c:\kosmas.zip");
    while ($zip_entry = zip_read($zip)) {
    $fp = fopen("c:/unzip/import.xml", "w");
    if (zip_entry_open($zip, $zip_entry, "r")) {
     $buf = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
     fwrite($fp,"$buf");
     zip_entry_close($zip_entry);
     fclose($fp);
     break;
    }
   zip_close($zip);
  }

It fails on my localhost with 128MB memory limit with the classic "Allowed memory size of blablabla bytes exhausted". On the server, I've got 16MB limit, is there a better way to do this so that I could fit into this limit? I don't see why this has to allocate more than 128MB of memory. Thanks in advance.

Solution: I started reading the files in 10Kb chunks, problem solved with peak memory usage arnoud 1.5MB.

        $filename = 'c:\kosmas.zip';
        $archive = zip_open($filename);
        while($entry = zip_read($archive)){
            $size = zip_entry_filesize($entry);
            $name = zip_entry_name($entry);
            $unzipped = fopen('c:/unzip/'.$name,'wb');
            while($size > 0){
                $chunkSize = ($size > 10240) ? 10240 : $size;
                $size -= $chunkSize;
                $chunk = zip_entry_read($entry, $chunkSize);
                if($chunk !== false) fwrite($unzipped, $chunk);
            }

            fclose($unzipped);
        }
like image 907
cypher Avatar asked Jul 16 '10 08:07

cypher


2 Answers

Why do you read the whole file at once?

 $buf = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
 fwrite($fp,"$buf");

Try to read small chunks of it and writing them to a file.

like image 61
Quamis Avatar answered Oct 04 '22 16:10

Quamis


Just because a zip is less than PHP's memory limit & perhaps the unzipped is as well, doesn't take account of PHP's overhead generally and more importantly the memory needed to actually unzip the file, which whilst I'm not expert with compression I'd expect may well be a lot more than the final unzipped size.

like image 27
46bit Avatar answered Oct 04 '22 17:10

46bit