Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

php: Get html source code with cURL

Tags:

html

php

curl

How can I get the html source code of http://www.example-webpage.com/file.html without using file_get_contents()?

I need to know this because on some webhosts allow_url_fopen is disabled so you can't use file_get_contents(). Is it possible to get the html file's source with cURL (if cURL support is enabled)? If so, how? Thanks.

like image 964
John Paneth Avatar asked Aug 28 '10 20:08

John Paneth


People also ask

How can I get HTML code from a website using php?

Simple way: Use file_get_contents() : $page = file_get_contents('http://stackoverflow.com/questions/ask'); Please note that allow_url_fopen must be true in you php. ini to be able to use URL-aware fopen wrappers.

Can I use cURL in php?

Uses of cURL in PHPcURL is a PHP extension that allows you to use the URL syntax to receive and submit data. cURL makes it simple to connect between various websites and domains. Obtaining a copy of a website's material. Submission of forms automatically, authentication and cookie use.

How do I make my html page cURL?

Sending GET Request with Curl. To make a GET request using Curl, run the curl command followed by the target URL. Curl automatically selects the HTTP GET request method unless you use the -X, --request, or -d command-line option. The target URL is passed as the first command-line option.


2 Answers

Try the following:

$ch = curl_init("http://www.example-webpage.com/file.html");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$content = curl_exec($ch);
curl_close($ch);

I would only recommend this for small files. Big files are read as a whole and are likely to produce a memory error.


EDIT: after some discussion in the comments we found out that the problem was that the server couldn't resolve the host name and the page was in addition a HTTPS resource so here comes your temporary solution (until your server admin fixes the name resolving).

what i did is just pinging graph.facebook.com to see the IP address, replace the host name with the IP address and instead specify the header manually. This however renders the SSL certificate invalid so we have to suppress peer verification.

//$url = "https://graph.facebook.com/19165649929?fields=name";
$url = "https://66.220.146.224/19165649929?fields=name";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Host: graph.facebook.com'));
$output = curl_exec($ch);
curl_close($ch); 

Keep in mind that the IP address might change and this is an error source. you should also do some error handling using curl_error();.

like image 191
The Surrican Avatar answered Nov 09 '22 16:11

The Surrican


$curl = curl_init($url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($curl);
curl_close($curl);

Source: http://www.christianschenk.org/blog/php-curl-allow-url-fopen/

like image 3
Brad Avatar answered Nov 09 '22 15:11

Brad