Not able to mock urllib2.urlopen using Python's mock.patch

Question

Below is a code snippet of my api.py module

# -*- coding: utf-8 -*-

from urllib2 import urlopen
from urllib2 import Request

class API:

    def call_api(self, url, post_data=None, header=None):
        is_post_request = True if (post_data and header) else False
        response = None
        try:
            if is_post_request:
                url = Request(url = url, data = post_data, headers = header)
            # Calling api
            api_response = urlopen(url)
            response = api_response.read()
        except Exception as err:
            response = err

        return response

I am trying to mock urllib2.urlopen in unittest of above module. I have written

# -*- coding: utf-8 -*-
# test_api.py

from unittest import TestCase
import mock

from api import API

class TestAPI(TestCase):

    @mock.patch('urllib2.Request')
    @mock.patch('urllib2.urlopen')
    def test_call_api(self, urlopen, Request):
        urlopen.read.return_value = 'mocked'
        Request.get_host.return_value = 'google.com'
        Request.type.return_value = 'https'
        Request.data = {}
        _api = API()
        assert _api.call_api('https://google.com') == 'mocked'

After I run the unittest, I get an exception

<urlopen error unknown url type: <MagicMock name='Request().get_type()' id='159846220'>>

What am I missing? Please help me out.

Answer 1

You are patching the wrong things: take a look to Where to patch.

In api.py by

from urllib2 import urlopen
from urllib2 import Request

you create a local reference to urlopen and Request in your file. By mock.patch('urllib2.urlopen') you are patching the original reference and leave the api.py's one untouched.

So, replace your patches by

@mock.patch('api.Request')
@mock.patch('api.urlopen')

should fix your issue.... but is not enough.

In your test case api.Request are not used but urllib2.urlopen() create a Request by use the patched version: that is why Request().get_type() is a MagicMock.

For a complete fix you should change your test at all. First the code:

@mock.patch('api.urlopen', autospec=True)
def test_call_api(self, urlopen):
    urlopen.return_value.read.return_value = 'mocked'
    _api = API()
    self.assertEqual(_api.call_api('https://google.com'), 'mocked')
    urlopen.assert_called_with('https://google.com')

Now the clarification... In your test you don't call Request() because you pass just the first parameter, so I've removed useless patch. Moreover you are patching urlopen function and not urlopen object, that means the read() method you want mocked is a method of the object return by urlopen() call.

Finally I add a check on urlopen call and autospec=True that is always a good practice.