How to mock the service layer in a python (flask) webapp for unit testing?

Question

I am working on a webapp in flask and using a services layer to abstract database querying and manipulation away from the views and api routes. Its been suggested that this makes testing easier because you can mock out the services layer, but I am having trouble figuring out a good way to do this. As a simple example, imagine that I have three SQLAlchemy models:

models.py

class User(db.Model):
    id = db.Column(db.Integer, primary_key = True)
    email = db.Column(db.String)

class Group(db.Model):
    id = db.Column(db.Integer, primary_key = True)
    name = db.Column

class Transaction(db.Model):
    id = db.Column(db.Integer, primary_key = True)
    from_id = db.Column(db.Integer, db.ForeignKey('user.id'))
    to_id = db.Column(db.Integer, db.ForeignKey('user.id'))
    group_id = db.Column(db.Integer, db.ForeignKey('group.id'))
    amount = db.Column(db.Numeric(precision = 2))

There are users and groups, and transactions (which represent money changing hands) between users. Now I have a services.py that has a bunch of functions for things like checking if certain users or groups exist, checking if a user is a member of a particular group, etc. I use these services in an api route which is sent JSON in a request and uses it to add transactions to the db, something similar to this:

routes.py

import services

@app.route("/addtrans")
def addtrans():
    # get the values out of the json in the request
    args = request.get_json()
    group_id = args['group_id']
    from_id = args['from']
    to_id = args['to'] 
    amount = args['amount']

    # check that both users exist
    if not services.user_exists(to_id) or not services.user_exists(from_id):
        return "no such users"

    # check that the group exists
    if not services.group_exists(to_id):
        return "no such group"

    # add the transaction to the db
    services.add_transaction(from_id,to_id,group_id,amount)
    return "success"

The problem comes when I try to mock out these services for testing. I've been using the mock library, and I'm having to patch the functions from the services module in order to get them to be redirected to mocks, something like this:

mock = Mock()
mock.user_exists.return_value = True
mock.group_exists.return_value = True

@patch("services.user_exists",mock.user_exists)
@patch("services.group_exists",mock.group_exists)
def test_addtrans_route(self):
    assert "success" in routes.addtrans()

This feels bad for any number of reasons. One, patching feels dirty; two, I don't like having to patch every service method I'm using individually (as far as I can tell there's no way to patch out a whole module).

I've thought of a few ways around this.

1. Reassign routes.services so that it refers to my mock rather than the actual services module, something like: routes.services = mymock
2. Have the services be methods of a class which is passed as a keyword argument to each route and simply pass in my mock in the test.
3. Same as (2), but with a singleton object.

I'm having trouble evaluating these options and thinking of others. How do people who do python web development usually mock services when testing routes that make use of them?

Answer 1

You can use dependency injection or inversion of control to achieve a code much simpler to test.

replace this:

def addtrans():
    ...
    # check that both users exist
    if not services.user_exists(to_id) or not services.user_exists(from_id):
        return "no such users"
    ...

with:

def addtrans(services=services):
    ...
    # check that both users exist
    if not services.user_exists(to_id) or not services.user_exists(from_id):
        return "no such users"
    ...

what's happening:

• you are aliasing a global as a local (that's not the important point)
• you are decoupling your code from services while expecting the same interface.
• mocking the things you need is much easier

e.g.:

class MockServices:
    def user_exists(id):
        return True

Some resources:

• https://github.com/ivan-korobkov/python-inject
• http://code.activestate.com/recipes/413268/
• http://www.ninthtest.net/aglyph-python-dependency-injection/

Answer 2

You can patch out the entire services module at the class level of your tests. The mock will then be passed into every method for you to modify.

@patch('routes.services')
class MyTestCase(unittest.TestCase):

    def test_my_code_when_services_returns_true(self, mock_services):
        mock_services.user_exists.return_value = True

        self.assertIn('success', routes.addtrans())


    def test_my_code_when_services_returns_false(self, mock_services):
        mock_services.user_exists.return_value = False

        self.assertNotIn('success', routes.addtrans())

Any access of an attribute on a mock gives you a mock object. You can do things like assert that a function was called with the mock_services.return_value.some_method.return_value. It can get kind of ugly so use with caution.