Parsing a curl json response and using response to build another request

Question

I am trying to build an shell script that will fetch the image url for a random XKCD comic, so that I can display it using Übersicht. Since you can only ask the XKCD API for the latest comic or a specific comic I need to:

1. Send a GET request to http://xkcd.com/info.0.json, fetch the value of the num element.
2. Send another request to http://xkcd.com/XXX/info.0.json where XXXis the value of num.

My current command looks like this and successfully returns the comic number:

curl -s 'http://xkcd.com/1510/info.0.json' | grep -Eo '"num": \d+' | grep -Eo '\d+'

1. I haven't been able to figure out how to use capture groups with grep, so I need to grep the json twice. The common advice is to use -P, which is not supported in Mac OS X 10.10.
2. I have no idea how to read the output of grep (as XXX) into the second curl -s 'http://xkcd.com/XXX/info.0.json' command.

Answer 1

1. On OS X you can use the system Perl:

    curl -sS http://xkcd.com/info.0.json | /usr/bin/perl -pe 's/.*"num": ([0-9]+).*/\1/'
   

2. You can save the output to a variable with command substitution:

   num=$(curl -sS http://xkcd.com/info.0.json | /usr/bin/perl -pe 's/.*"num": ([0-9]+).*/\1/')
   curl -sS "http://xkcd.com/${num}/info.0.json"
   

   or more concisely, two-in-one, albeit not very readable:

   curl -sS "http://xkcd.com/$(curl -sS http://xkcd.com/info.0.json | /usr/bin/perl -pe 's/.*"num": ([0-9]+).*/\1/')/info.0.json"
   

---

By the way, I highly recommend jq as the command line JSON processor. To extract num with jq, it's as simple as

curl -sS http://xkcd.com/info.0.json | jq '.num'

and although you didn't ask for it, here's a simple one-liner with jq that extracts the URI of the latest image:

curl -sS "http://xkcd.com/$(curl -sS http://xkcd.com/info.0.json | jq '.num')/info.0.json" | jq -r '.img'

Example output:

http://imgs.xkcd.com/comics/spice_girl.png