Skip successfully downloaded files using Wget

Question

So I have this Bash subroutine to download files using wget and my problem now is how to skip successfully downloaded files. The script downloads a lot of files and once the download fails, it re-downloads all files overwriting those successful downloads from the start (which may be incomplete because of the re-download).

So how do I skip those files downloaded successfully?

DownloadFile() {
  paramURL=$1
  paramFilename=$2

  if [ $flag_archive_fetch = "false"  ];
  then
      wget "--timeout=180" "--tries=5" "$paramURL" "-O" "${scratch_dir}$paramFilename"
  else
      unzip -o "$archive_file" "$paramFilename" -d "${scratch_dir}"
  fi

  touch "${scratch_dir}$paramFilename"
}

Answer 1

You can take advantage of Wget --continue (to resume broken downloads) and --timestamping (to overwrite successfully downloaded files only when Last-modified attribute has changed, otherwise skips the download)

wget "--continue ‐‐timestamping --timeout=180" "--tries=5" "$paramURL" "-O" "${scratch_dir}$paramFilename"

another option is to use --no-clobber instead of --timestamping, it skips the already downloaded files without checking the Last-modified attribute,

 wget "--continue ‐‐no-clobber --timeout=180" "--tries=5" "$paramURL" "-O" "${scratch_dir}$paramFilename"

Answer 2

you could check the WGET exit status code by checking the $?

wget .....
# store the error
error=$?
if (( $error != 0 )) 
then
   #handle error
else
   #handle success
fi