python asynchronous httprequest

Question

I am trying to use twitter search web service in python. I want to call a web service like:

http://search.twitter.com/search.json?q=blue%20angels&rpp=5&include_entities=true&result_type=mixed

from my python program.

Can anybody tell me

1. how to use xmlhttprequst object in python

2. how to pass parameters to it, and

3. how to get the data in dictionary.

Here is my try:

import urllib
import sys
url = "http://search.twitter.com/search.json?q=blue%20angels&rpp=5&include_entities=true&result_type=mixed"
urlobj = urllib.urlopen(url)
data = urlobj.read()
print data

Thanks.

Answer 1

You don't need "asynchronous httprequest" to use twitter search api:

import json
import urllib
import urllib2

# make query
query = urllib.urlencode(dict(q="blue angel", rpp=5, include_entities=1,
                              result_type="mixed"))  
# make request
resp = urllib2.urlopen("http://search.twitter.com/search.json?" + query)

# make dictionary (parse json response)
d = json.load(resp)

There are probably several libraries that provide a nice OO interface around these http requests.

To make multiple requests concurrently you could use gevent:

import gevent
import gevent.monkey; gevent.monkey.patch_all() # patch stdlib

import json
import urllib
import urllib2

def f(querystr):
    query = urllib.urlencode(dict(q=querystr, rpp=5, include_entities=1,
                                  result_type="mixed"))
    resp = urllib2.urlopen("http://search.twitter.com/search.json?" + query)
    d = json.load(resp)
    print('number of results %d' % (len(d['results']),))

jobs = [gevent.spawn(f, q) for q in ['blue angel', 'another query']]
gevent.joinall(jobs) # wait for completion