cURL returns 302, whereas a browser returns 200

Question

I have a script that uses a load of cURLs to log into a site and submit a series of forms, however this has recently stopped working due to the cURL requests returning 302s and redirecting to a block/endpoint page. If I do the same actions with a browser, I there are no redirects, just a 200 OK.

My cURL is using the cookie returned by the login process, so I don't think that the session is being dropped.

I'd originally thought that a CSRF Token (of some kind) was missing, given the redirect location, and that later forms in the process (using the browser) contain a hidden CSRF Token field, but the URL requires no posted data.

The cURL and response are as follows:

curl_setopt($ch, CURLOPT_URL, 'https://*******.********.co.uk/Dispatcher?menuid=pos_home'); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_HEADER, 0);
$content = curl_exec ($ch);
echo "CURL INFO : <BR/><pre>" ;
print_r(curl_getinfo($ch));

which returns:

Array
(
[url] => https://*******.********.co.uk/Dispatcher?menuid=pos_home
[content_type] => text/html
[http_code] => 302
[header_size] => 253
[request_size] => 332
[filetime] => -1
[ssl_verify_result] => 20
[redirect_count] => 0
[total_time] => 0.142718
[namelookup_time] => 2.4E-5
[connect_time] => 2.4E-5
[pretransfer_time] => 9.0E-5
[size_upload] => 43
[size_download] => 327
[speed_download] => 2291
[speed_upload] => 301
[download_content_length] => -1
[upload_content_length] => 43
[starttransfer_time] => 0.142659
[redirect_time] => 0
[certinfo] => Array
    (
    )

[primary_ip] => nnn.nnn.nnn.nn
[primary_port] => 443
[local_ip] => nnn.nnn.nnn.nn
[local_port] => 53154
[redirect_url] => https://*******.********.co.uk/Dispatcher?menuid=badorMissingCSRFT

)

If anyone has any ideas why or how a server can give a different response for a cURL to a browser request, I'd be very grateful - thanks.of

Answer 1

You need to follow the redirection:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

Other causes for not following a redirection:

4.14 Redirects work in browser but not with curl!

curl supports HTTP redirects fine (see item 3.8). Browsers generally support at least two other ways to perform redirects that curl does not:

Meta tags. You can write a HTML tag that will cause the browser to redirect to another given URL after a certain time.

Javascript. You can write a Javascript program embedded in a HTML page that redirects the browser to another given URL.

There is no way to make curl follow these redirects. You must either manually figure out what the page is set to do, or you write a script that parses the results and fetches the new URL.

source: http://curl.haxx.se/docs/faq.html#Redirects_work_in_browser_but_no