file_get_contents failed to open stream: HTTP request failed! HTTP/1.1 500 Internal > Server Error in

Question

I see many questions matching to my issue. But none of them is providing any workable solution. so instead of making complex functionality i need simple solution. Please I dont' want cURL. I've already done a lot with this code.

I'm working on a scraping project and need to load the contents of product revisions from ebay.

I am using PHP function

file_get_contents();

Here is the URL i'm troubling with.

https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&item=272908801183

this file_get_contents(); is working perfectly fine on different URLs but don't know why not on this.

This is the code i'm using.

<?php
$html = file_get_contents('https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&item=272908801183');
echo $html;
?>

but i'm getting this error..

Warning: file_get_contents(https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&&item=272908801183): failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in /Users/samz/Documents/projects/scraping/test.php on line 3

Answer 1

file_get_contents (read the tip on blue rectangle) can be easily blocked on server side through php.ini avoid using it. When you want to get data from an other site use curl instead. http://php.net/manual/en/book.curl.php, there are plenty of options that you can use with curl, by playing a bit the following code can work with your url.

$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch,CURLOPT_URL,"https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&item=272908801183");
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13");
$data = curl_exec($ch);
curl_close($ch);

and by echoing the $data varible you can see the whole page.

echo $data;

you can try parsing the data from the page by utilizing php DOM Methods and convert them to the data type you want (object class, array etc).