How to partially download a remote file with cURL?

Question

Is it possible to partially download a remote file with cURL? Let's say, the actual filesize of the remote file is 1000 KB. How can I download only first 500 KB of it?

Answer 1

You can also set the range header parameter with the php-curl extension.

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.spiegel.de/'); curl_setopt($ch, CURLOPT_RANGE, '0-500'); curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($ch); curl_close($ch); echo $result; 

But as noted before if the server doesn't honor this header but sends the whole file curl will download all of it. E.g. http://www.php.net ignores the header. But you can (in addition) set a write function callback and abort the request when more data is received, e.g.

// php 5.3+ only // use function writefn($ch, $chunk) { ... } for earlier versions $writefn = function($ch, $chunk) {    static $data='';   static $limit = 500; // 500 bytes, it's only a test    $len = strlen($data) + strlen($chunk);   if ($len >= $limit ) {     $data .= substr($chunk, 0, $limit-strlen($data));     echo strlen($data) , ' ', $data;     return -1;   }    $data .= $chunk;   return strlen($chunk); };  $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.php.net/'); curl_setopt($ch, CURLOPT_RANGE, '0-500'); curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1); curl_setopt($ch, CURLOPT_WRITEFUNCTION, $writefn); $result = curl_exec($ch); curl_close($ch); 

Answer 2

Get the first 100 bytes of a document:

curl -r 0-99 http://www.get.this 

from the manual

make sure you have a modern curl