How do you test a function using which retrieves data by urllib2?

Question

I am getting into testing in python and I asked myself how to test this method.

def get_response(self, url, params):
    encoded_params = urllib.urlencode(params)
    request = urllib2.Request(BASE_URL, headers=HEADERS)
    response = urllib2.urlopen(request, encoded_params)
    return response

Using doctest or unittest, how is this best achieved? I thought of passing get_response() a test url and some test parameters, which exists in real world and to check if response.read() returns the expected data. But somehow I feel, this is not the way it should be done. Any suggestions? I would appreciate suggestions how to handle such cases in context of texting.

Answer 1

This is a good case for using fake objects:

# my_module
get_url = urllib2.urlopen

def get_response(self, url, params):
    encoded_params = urllib.urlencode(params)
    request = urllib2.Request(BASE_URL, headers=HEADERS)
    response = get_url(request, encoded_params)
    return response

# test_my_module    
def fake_get_url(request, params):
    assert request == "the url I expect"
    assert params == ['the', 'params', 'i', 'expect']
    return SomeFakeResponse("OK")

my_module.get_url = fake_get_url
assert my_module.get_response("blah", "blah").content == "OK"

This is just the sketchiest sketch of how you could override the real urllib2.urlopen function with your own fake implementation to test your code without really hitting the web.