Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Simple PHP / cURL caching

Tags:

php

curl

I'm getting content of remote page through cURL, I want to cache that page so that next time this page should not request cURL, and load from cache.

Cache should expire in one week.

like image 293
Nandla Avatar asked Jun 09 '26 05:06

Nandla


1 Answers

Some pseudo code.

make a temp folder 


before each request check if page exists(with name==hash of URL) in the temp folder,  
if not,  
  fetch the page, 
  hash the URL,  
  save page in temp with the Hashed url file name.
else if exists
  get the temp file content
return contents

Update: Code::

$curl_defaults = array(
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_HEADER => 0,
    CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11',
    CURLOPT_FOLLOWLOCATION => 1,
    CURLOPT_AUTOREFERER => 1,
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_CONNECTTIMEOUT => 5,
    CURLOPT_TIMEOUT => 20,
    CURLOPT_VERBOSE => 0,
    CURLOPT_SSL_VERIFYHOST => 0,
    CURLOPT_SSL_VERIFYPEER => 0
);

$curl_headers = array();
$curl_headers[] = 'Accept: image/gif, image/x-bitmap, image/jpeg, image/pjpeg';
$curl_headers[] = 'Connection: Keep-Alive';
$curl_headers[] = 'Content-type: application/x-www-form-urlencoded;charset=UTF-8';

function get($url, $data = "") {

    if (strlen($url) < 7) return;
    //echo "\n<br> Sending GET :: $url\n<br>";
    if (is_array($data)) $data = implode("&", $data);
    if (strpos($url, "?") > 5) $url .= "&$data";
    else
        $url .= "?$data";

    global $curl_defaults;

    $rep = array(":", "/", ".", "?", "&", "+", "=");
    $fn = dirname(__FILE__) . "/cache/" . md5(str_replace($rep, "", $url)) . ".txt";

    if (file_exists($fn) && @filesize($fn)>1){   //Add a file time check Here
        //echo "\n<br>From Cache FileSize :: "  . filesize($fn);
        return file_get_contents($fn);
    }


    $ch = curl_init();
    curl_setopt_array($ch, $curl_defaults);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    $html = curl_exec($ch);

    @unlink($fn);
    if(strlen($html) > 100)
        write2file($fn, $html);

    return $html;
}

You will probably need to make some changes.

Cache Dir : ./cache/

like image 90
Pheonix Avatar answered Jun 10 '26 20:06

Pheonix