Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scrape a site content With a Secure Login

I am trying to scrape the contents of the a site with login secured but unable to do it The site's login has three options username,password,passcode here is the code I am using

<?php

// HTTP authentication

$url = "http://aftabcurrency.com/login_script.php";

$ch = curl_init();    

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 

curl_setopt($ch, CURLOPT_URL, $url); 
$cookie = 'cookies.txt';
$timeout = 30;
curl_setopt($curl, CURLOPT_TIMEOUT,         10); 
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT,  $timeout );
curl_setopt($curl, CURLOPT_COOKIEJAR,       $cookie);
curl_setopt($curl, CURLOPT_COOKIEFILE,      $cookie);

curl_setopt ($ch, CURLOPT_POST, 1); 
curl_setopt ($ch,CURLOPT_POSTFIELDS,"user_name=user&user_password=pass&passcode=code");             

$result = curl_exec($ch); 

curl_close($ch); 

echo $result;

?>
like image 923
user1447187 Avatar asked Jun 23 '12 17:06

user1447187


1 Answers

you need to do a POST to http://aftabcurrency.com/login_script.php your curl needs also to accept cookies.
After the authentification the script will redirect you, so you need also to add CURLOPT_FOLLOWACTION.

here is a edited version of your script, I can't test it on http://aftabcurrency.com/ hope it works:

$url = "http://aftabcurrency.com/login_script.php";

$ch = curl_init();    
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 

curl_setopt($ch, CURLOPT_URL, $url); 
$cookie = 'cookies.txt';
$timeout = 30;

curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_TIMEOUT,         10); 
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,  $timeout );
curl_setopt($ch, CURLOPT_COOKIEJAR,       $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE,      $cookie);

curl_setopt ($ch, CURLOPT_POST, 1); 
curl_setopt ($ch,CURLOPT_POSTFIELDS,"user_name=user&user_password=pass&passcode=code");     

$result = curl_exec($ch);

/* //OPTIONAL - Redirect to another page after login
$url = "http://aftabcurrency.com/some_other_page";
curl_setopt ($ch, CURLOPT_POST, 0); 
curl_setopt($ch, CURLOPT_URL, $url);
$result = curl_exec($ch);
 */ //end OPTIONAL 

curl_close($ch); 
echo $result;
like image 69
MilMike Avatar answered Sep 28 '22 14:09

MilMike