Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use curl to get a GET request exactly same as using Chrome?

I have a web api http://something.com/api and I want to use GET to get the response body.

This is my command:

curl "http://something.com/api" 

Of course, it fails and gives an error message.

When I use Chrome and input the above url, everythings correct. However I do the same things with Firefox, the url gives me the same error message. I try to repeat the action with Chrome extension DHC, the request gives correct response again. After some searching, I believe that the curl option --user-agent makes a difference. What is the correct way to set the user agent to Chrome? Or this is not the point, the problem comes from other fields? Thank you very much.

like image 416
wdetac Avatar asked Feb 27 '15 08:02

wdetac


People also ask

How do you curl a GET request?

To make a GET request using Curl, run the curl command followed by the target URL. Curl automatically selects the HTTP GET request method unless you use the -X, --request, or -d command-line option.

Does curl use GET?

curl knows the HTTP methodIf you just pass in a HTTP URL like curl http://example.com , curl will use GET. If you use -d or -F curl will use POST, -I will cause a HEAD and -T will make it a PUT.

How do I copy a browser request as a curl?

From Chrome On the line of the specific resource you're interested in, you right-click with the mouse and you select “Copy as cURL” and it'll generate a command line for you in your clipboard. Paste that in a shell to get a curl command line that makes the transfer.


1 Answers

If you need to set the user header string in the curl request, you can use the -H option to set user agent like:

curl -H "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36" http://stackoverflow.com/questions/28760694/how-to-use-curl-to-get-a-get-request-exactly-same-as-using-chrome 

Updated user-agent form newest Chrome at 02-22-2021


Using a proxy tool like Charles Proxy really helps make short work of something like what you are asking. Here is what I do, using this SO page as an example (as of July 2015 using Charles version 3.10):

  1. Get Charles Proxy running
  2. Make web request using browser
  3. Find desired request in Charles Proxy
  4. Right click on request in Charles Proxy
  5. Select 'Copy cURL Request'

Copy cURL Request example in Charles 3.10.2

You now have a cURL request you can run in a terminal that will mirror the request your browser made. Here is what my request to this page looked like (with the cookie header removed):

curl -H "Host: stackoverflow.com" -H "Cache-Control: max-age=0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36" -H "HTTPS: 1" -H "DNT: 1" -H "Referer: https://www.google.com/" -H "Accept-Language: en-US,en;q=0.8,en-GB;q=0.6,es;q=0.4" -H "If-Modified-Since: Thu, 23 Jul 2015 20:31:28 GMT" --compressed http://stackoverflow.com/questions/28760694/how-to-use-curl-to-get-a-get-request-exactly-same-as-using-chrome 
like image 177
Mike Grace Avatar answered Sep 28 '22 23:09

Mike Grace