Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting the URLs for the first Google search results in a shell script

Tags:

bash

It's relatively easy to parse the output of the AJAX API using a scripting language:

#!/usr/bin/env python

import urllib
import json

base = 'http://ajax.googleapis.com/ajax/services/search/web?v=1.0&'
query = urllib.urlencode({'q' : "something"})
response = urllib.urlopen(base + query).read()
data = json.loads(response)
print data['responseData']['results'][0]['url']

But are there any better ways to do something similar with just basic shell scripting? If you just curled the API page, how should you encode the URL parameters or parse JSON?

like image 827
Lri Avatar asked Mar 31 '11 21:03

Lri


2 Answers

many years later, you can install googler

googler -n 1 -c in -l en search something here --json

you can control the number of output page using the n flag.

To get only the url, simply pipe it to:

grep "\"url\""|tr -s ' ' |cut -d ' ' -f3|tr -d "\""
like image 158
once Avatar answered Sep 20 '22 13:09

once


Untested approach as I don't have access to a unix box currently ...

Assuming "test" is the query string, you could use a simple wget on the following url http://www.google.co.in/#hl=en&source=hp&biw=1280&bih=705&q=test&btnI=Google+Search&aq=f&aqi=g10&aql=&oq=test&fp=3cc29334ffc8c2c

This would leverage Google's "I'm feeling lucky" functionality and wget the first url for you. You may be able to clean up the above url a bit too.

like image 38
qwerty Avatar answered Sep 20 '22 13:09

qwerty