How can one mock/stub python module like urllib

Question

I need to test a function that needs to query a page on an external server using urllib.urlopen (it also uses urllib.urlencode). The server could be down, the page could change; I can't rely on it for a test.

What is the best way to control what urllib.urlopen returns?

Answer 1

Another simple approach is to have your test override urllib's urlopen() function. For example, if your module has

import urllib

def some_function_that_uses_urllib():
    ...
    urllib.urlopen()
    ...

You could define your test like this:

import mymodule

def dummy_urlopen(url):
    ...

mymodule.urllib.urlopen = dummy_urlopen

Then, when your tests invoke functions in mymodule, dummy_urlopen() will be called instead of the real urlopen(). Dynamic languages like Python make it super easy to stub out methods and classes for testing.

See my blog posts at http://softwarecorner.wordpress.com/ for more information about stubbing out dependencies for tests.

Answer 2

I am using Mock's patch decorator:

from mock import patch

[...]

@patch('urllib.urlopen')
def test_foo(self, urlopen_mock):
    urlopen_mock.return_value = MyUrlOpenMock()