Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to pass Age Verification with DOM

I'm attempting to pull some image URLs from Steam store pages, such as: http://store.steampowered.com/app/35700/
http://store.steampowered.com/app/252490/

Here's the code I'm using:

$url = 'http://store.steampowered.com/app/35700/';
$html = file_get_contents($url);
$dom = new domDocument;
$dom->loadHTML($html);
$dom->preserveWhiteSpace = false;
$images = $dom->getElementsByTagName('img');
foreach ($images as $image) {
  echo $image->getAttribute('src');
}

It works fine with the first store page, but the second one redirects to an age verification page, and the script returns the images from there. I need a way for the script to get past the age verification and access the actual store page.

Any help would be appreciated.

Edit:

This is what's passed to the server when the age form is submitted:

snr=1_agecheck_agecheck__age-gate&ageDay=1&ageMonth=January&ageYear=1979

and the cookies that it sets:

lastagecheckage=1-January-1979; expires=Tue, 03 Mar 2015 19:53:42 GMT; path=/; domain=store.steampowered.com
birthtime=662716801; path=/; domain=store.steampowered.com

Edit2:

I can set the cookies using cURL but they aren't used by DOM loadHTML, so I get the same result as before. I need either a way for loadHTML to use specific cookies that I set, or another method of grabbing the image URLs that will use cookies set by cURL.

like image 517
Martok Avatar asked Mar 03 '14 06:03

Martok


1 Answers

Solved! Here's the working code:

$url = 'http://store.steampowered.com/app/35700/';

$ch = curl_init();

curl_setopt($ch, CURLOPT_COOKIE, "birthtime=28801; path=/; domain=store.steampowered.com");
curl_setopt($ch, CURLOPT_TIMEOUT, 5); 
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

$result = curl_exec($ch);

$dom = new domDocument;
libxml_use_internal_errors(true);
$dom->loadHTML($result);
$dom->preserveWhiteSpace = false;

$images = $dom->getElementsByTagName('img');
foreach ($images as $image) {
  $src = $image->getAttribute('src');
  echo $src.PHP_EOL;
}

curl_close($ch);
like image 72
Martok Avatar answered Oct 06 '22 06:10

Martok