Get the latest download link programmatically

Question

I want to be able to download the latest version of a software automatically with my bash script. Unfortunately not ever website has the latest release link just like github. In my case I need to download the latest stable version of Nginx

Currently I use this http://nginx.org/download/nginx-1.4.7.tar.gz and then I compile from source

The problem is that I will need to manually check for updates from time to time and update the link.

Is there any way that I can use to make my script grab the latest stable version of Nginx automatically

P.S installing via yum is not an option

Answer 1

Update:
Solution that determines and downloads the latest stable version via http://nginx.org/en/download.html (http://nginx.org/download/, used in the original solution below, does not distinguish between stable and mainline versions) - works on both Linux and OSX:

# Determine the latest stable version's download URL, assumed to be 
# the first `/download/nginx-*.tar.gz`-like link following the header 
# "Stable version".
latestVer=$(curl -s 'http://nginx.org/en/download.html' | 
   sed 's/</\'$'\n''</g' | sed -n '/>Stable version$/,$ p' | 
   egrep -m1 -o '/download/nginx-.+\.tar\.gz')

# Download.
curl "http://nginx.org${latestVer}" > nginx-latest.tar.gz

Note: This relies on specifics of the HTML structure of page http://nginx.org/en/download.html, which is not the most robust solution.

---

Original solution that determines the latest version via http://nginx.org/download/, where no distinction is made between stable and mainline versions:

On Linux, try:

 # Determine latest version:
 latestVer=$(curl 'http://nginx.org/download/' | 
   grep -oP 'href="nginx-\K[0-9]+\.[0-9]+\.[0-9]+' | 
   sort -t. -rn -k1,1 -k2,2 -k3,3 | head -1)

 # Download latest version:
 curl "http://nginx.org/download/nginx-${latestVer}.tar.gz" > nginx-latest.tar.gz

This does NOT rely on a specific listing order at http://nginx.org/download/; instead, version numbers are extracted and sorted appropriately.

---

On OSX - where grep doesn't support -P and \K for dropping the left part of a match is not available, try:

# Determine latest version:
latestVer=$(curl 'http://nginx.org/download/' | 
 egrep -o 'href="nginx-[0-9]+\.[0-9]+\.[0-9]+' | sed 's/^href="nginx-//' |
 sort -t. -rn -k1,1 -k2,2 -k3,3 | head -1)

# Download latest version:
curl "http://nginx.org/download/nginx-${latestVer}.tar.gz" > nginx-latest.tar.gz