Simple bash and curl check that a file exists on a web server?

Question

I am trying to improve a WebDAV bash uploader to bypass uploading files when they already exist on the web server.

What should work with a HEAD request and parsing this after this.

Extra bonus if the solution would also include size verification.

Answer 1

Quick and dirty: Just check if a Content-Length header is in the response?

Something like:

curl -I http://google.com | grep Content-Length

Then potentially parse the length using awk:

curl -I http://google.com | grep Content-Length | awk -F ': ' '{print $2}'

This give you the length of the content in bytes (if the file exists) or otherwise an empty string.

Answer 2

Here is some bash code that checks if the files exists on the webserver and if it does have the same size as the one one disk.

The file is not downloaded, only checked using a HEAD request.

TMPFILE=$(tempfile)
FILE=location/some_file_to_upload.txt
MYVAR=http://example.com/$FILE

response=$(curl -n --silent -X HEAD $MYVAR --write-out "\n%{http_code}\n" -D $TMPFILE)
status_code=$(echo "$response" | sed -n '$p')

LOCAL_SIZE=$(stat -c%s "$FILE")
SERVER_SIZE=$(cat $TMPFILE|grep -i "Content-Length" | awk 'BEGIN {RS=""}{gsub(/\r/,"",$0); print $0}' | awk -F ': ' '{print $2}')
if [ -z "$VAR" ]; then
       SERVER_SIZE=0
fi

if [ "$status_code" -lt "200" -o "$status_code" -gt "299" -o "$LOCAL_SIZE" -ne "$SERVER_SIZE" ]; then
        echo 'file does not exist or is of different size
fi

Answer 3

status=$(curl --head --silent http://domain/object | head -n 1)
if echo "$status" | grep -q 404
  echo "Uploading"
else
  echo "Not uploading: $status"
fi