Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using CURL to download file and view headers and status code

Tags:

bash

curl

I'm writing a Bash script to download image files from Snapito's web page snapshot API. The API can return a variety of responses indicated by different HTTP response codes and/or some custom headers. My script is intended to be run as an automated Cron job that pulls URLs from a MySQL database and saves the screenshots to local disk.

I am using curl. I'd like to do these 3 things using a single CURL command:

  1. Extract the HTTP response code
  2. Extract the headers
  3. Save the file locally (if the request was successful)

I could do this using multiple curl requests, but I want to minimize the number of times I hit Snapito's servers. Any curl experts out there?

Or if someone has a Bash script that can respond to the full documented set of Snapito API responses, that'd be awesome. Here's their API documentation.

Thanks!

like image 366
curtisdf Avatar asked Aug 06 '12 21:08

curtisdf


People also ask

How do I use curl command to download a file?

How to download a file with curl command. The basic syntax: Grab file with curl run: $ curl https://your-domain/file.pdf. Get file using ftp or sftp protocol: $ curl ftp://ftp-your-domain-name/file.tar.gz.

How do I show headers in curl?

We can use curl -v or curl -verbose to display the request headers and response headers in the cURL command. The > lines are request headers . The < lines are response headers .

How do I use curl command to download Windows?

To download a file with Curl, use the --output or -o command-line option. This option allows you to save the downloaded file to a local drive under the specified name. If you want the uploaded file to be saved under the same name as in the URL, use the --remote-name or -O command line option.

How do I download a folder using curl?

To download you just need to use the basic curl command but add your username and password like this curl --user username:password -o filename. tar. gz ftp://domain.com/directory/filename.tar.gz . To upload you need to use both the –user option and the -T option as follows.


1 Answers

Use the dump headers option: curl -D /tmp/headers.txt http://server.com

like image 74
WA Hunt Avatar answered Oct 19 '22 06:10

WA Hunt