How to unit test/mock requests.get(url) & its response

Question

I am having trouble unit testing a function that uses Kenneth Reitz's requests library:

The following function is to be unit tested:

def getPage(url):
    r = requests.get(url)
    r.raise_for_status()    # raises HTTPError if necessary
    dom = fromstring(r.text)
    dom.make_links_absolute(url)
    return dom

My unit test is at the moment as follows. (Though clearly this does not work.)

@patch('requests.Response')
@patch('requests.Response.raise_for_status', return_value=None)
@patch('requests.get', return_value=requests.Response)
def test_getPage(mock_requests_get, mock_RFS, mock_response):
    with open("domtest.htm", "r") as testfile:
        mock_response.text = testfile.read()    

    dom = getPage('http://www.test.com')
    eq_(dom, dom_test)

The resulting traceback is:

Traceback (most recent call last):
  File "C:\Anaconda\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "C:\Anaconda\lib\site-packages\mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "C:\...\test_myfile.py", line 79, in test_getPage
    dom = getPage('http://www.test.com')
  File "C:\...\myfile.py", line 67, in getPage
    r.raise_for_status()    # raises HTTPError if necessary
TypeError: unbound method raise_for_status() must be called with Response instance as first argument (got nothing instead)

How does one unit test a function like this using mocking rather than a temporary webserver? I tried pickling the response and setting it instead as return value for requests.get() but it cannot be pickled.

Answer 1

Very late, but I really like the HTTPretty library for this exact problem:

http://falcao.it/HTTPretty/