Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to skip already existing files when downloading with curl?

I want to curl to download a link, but I want it to skip files that already exist. Right now, the line of code I have will continue to overwrite it no mater what:

curl '$url' -o /home/$outputfile &>/dev/null & 

How this can be achieved?

like image 556
thevoipman Avatar asked Aug 08 '12 01:08

thevoipman


People also ask

Does curl overwrite files?

curl attempts to cut off the directory parts from any given file name in the header to only store files in the current directory. It will overwrite a local file using the same name as the header specifies.

How do I download multiple files with curl?

Download multiple files simultaneously Instead of downloading multiple files one by one, you can download all of them simultaneously by running a single command. To download multiple files at the same time, use –O followed by the URL to the file that you wish to download. The above command will download both files.

Where does curl save downloaded files?

Consequentially, the file will be saved in the current working directory. If you want the file saved in a different directory, make sure you change current working directory before you invoke curl with the -O, --remote-name flag!

How do I transfer files using curl?

How to send a file using Curl? To upload a file, use the -d command-line option and begin data with the @ symbol. If you start the data with @, the rest should be the file's name from which Curl will read the data and send it to the server. Curl will use the file extension to send the correct MIME data type.


1 Answers

You could just put your call to curl inside an if block:

if ! [ -f /home/$outputfile ]; then   curl -o /home/$outputfile "url" fi 

Also note that in your example, you've got $url inside single quotes, which won't do what you want. Compare:

echo '$HOME' 

To:

echo "$HOME" 

Also, curl has a --silent option that can be useful in scripts.

like image 128
Bob the Angry Coder Avatar answered Oct 02 '22 17:10

Bob the Angry Coder