Testing for MongoDB Functionality using Motor AsyncIO and Pytest

Question

So I am trying to write several tests to test my functions that use an async MongoDB connection. To connect to MongoDB I use Motor with asyncio. I need help with mocking the Motor connection.

My Code:

commons.py

mongo = None

blacklist.py

import commons

class Blacklist(object):
    async def check_if_blacklisted(self, word: str):
        blacklisted = False
        if await commons.mongo.dbtest.blacklist.find_one({'word': word}):
            blacklisted = True
        return blacklisted

main.py

import asyncio
from blacklist import Blacklist
from motor.motor_asyncio import AsyncIOMotorClient
import commons

async def run():
    commons.mongo = AsyncIOMotorClient("mongodb://localhost", io_loop=asyncio.get_event_loop())
    blacklist_checker = Blacklist()
    result = await blacklist_checker.check_if_blacklisted(word="should_be_false")
    print(result)
    # > False

    result = await blacklist_checker.check_if_blacklisted(word="should_be_true")
    print(result)
    # > True

loop = asyncio.get_event_loop()
loop.run_until_complete(run())
loop.close()

I now want to test blacklist.py by mocking the Motor Connection but I cannot seem to get the test running properly. Here are the codes that I've tried:

test_blacklist.py

import pytest
from blacklist import Blacklist

class TestBlacklist(object):

@pytest.fixture
async def motor(self, event_loop):
    # I know I'm not mocking the Motor Connection here, 
    # but just wanted to show you the output using this fixture.
    commons.mongo = motor.motor_asyncio.AsyncIOMotorClient(io_loop=event_loop)
    yield commons.mongo
    commons.mongo.close()

@pytest.mark.asyncio
async def test_check_if_blacklisted(self):
    blacklist_checker = Blacklist()
    blacklisted = await blacklist_checker.check_if_blacklisted(word="should_be_false")
    assert blacklisted == False
    # > AttributeError: 'NoneType' object has no attribute 'blacklist'

pytest-mongodb:

import pytest
from unittest.mock import patch
from blacklist import Blacklist

class TestBlacklist(object):

    @pytest.mark.asyncio
    async def test_check_if_blacklisted(self, mongodb):
        with patch("blacklist.commons.mongo") as db:
            db = mongodb
            blacklist_checker = Blacklist()
            blacklisted = await blacklist_checker.check_if_blacklisted(word="should_be_false")
        assert blacklisted == False
        # > TypeError: object MagicMock can't be used in 'await' expression

I tried searching online but I could not find a proper thread which would help me to perform the test while mocking the Motor connection which is async. Moreover, if you think that the direction I'm heading into for testing isn't right, kindly let me know since I am new to writing tests, especially with async db connections.

Note: blacklist.py has various functions that require MongoDB functionality so it would be great if in my test_blacklist.py I could just initialize commons.mongo once and all the subsequent tests use that.

Answer 1

You can mock the async MongoDB database with pytest-async-mongodb but have in mind that it's outdated and has dependency errors so you have to fix the dependencies versions as followings:

mongomock==3.12.0
pyyaml==3.13
pytest-asyncio==0.10.0
pytest==3.6.4

With pytest-async-mongodb you can get the mocked DB in the test by adding an argument called async_mongodb. I'm going to let you the code and the structure.

project
  -app
    __init__.py
    blacklist.py
    commons.py
  -test
    -fixtures
      blacklist.json
    __init__.py
    test_blacklist.py
  main.py
  pytest.ini

main.py

import asyncio
from app.blacklist import Blacklist
from app.commons import get_database, set_client
from motor.motor_asyncio import AsyncIOMotorClient


async def run():
    set_client(
        AsyncIOMotorClient("mongodb://localhost", io_loop=asyncio.get_event_loop())
    )
    db = await get_database()
    blacklist_checker = Blacklist()
    result = await blacklist_checker.check_if_blacklisted(db, word="should_be_false")
    print(result)
    # > False

    result = await blacklist_checker.check_if_blacklisted(db, word="should_be_true")
    print(result)
    # > True


loop = asyncio.get_event_loop()
loop.run_until_complete(run())
loop.close()

blacklist.py

from motor.motor_asyncio import AsyncIOMotorDatabase


class Blacklist(object):
    async def check_if_blacklisted(self, db: AsyncIOMotorDatabase, word: str):
        blacklisted = False
        if await db.blacklist.find_one({"word": word}):
            blacklisted = True
        return blacklisted

commons.py

from motor.motor_asyncio import AsyncIOMotorClient, AsyncIOMotorDatabase


class DataBase:
    client: AsyncIOMotorClient = None


db = DataBase()


async def get_database() -> AsyncIOMotorDatabase:
    return db.client["dbtest"]


def set_client(client):
    db.client = client

test_blacklist.py

import pytest
from app import blacklist


@pytest.mark.asyncio
async def test_should_be_false(async_mongodb):
    blacklist_checker = blacklist.Blacklist()
    blacklisted = await blacklist_checker.check_if_blacklisted(
        async_mongodb, word="should_be_false"
    )
    assert blacklisted == False


@pytest.mark.asyncio
async def test_should_be_true(async_mongodb):
    blacklist_checker = blacklist.Blacklist()
    blacklisted = await blacklist_checker.check_if_blacklisted(
        async_mongodb, word="should_be_true"
    )
    assert blacklisted == True

pytest.ini

[pytest]
async_mongodb_fixture_dir =
  test/fixtures

async_mongodb_fixtures =
  blacklist

blacklist.json

[
    {
      "_id": {"$oid": "60511d158f80a8d34986e2b0"},
      "word" : "should_be_true"
    }
]

The fixture can be a .yaml too and can define the amount that you want. Read the package documentation for more information.

Since it is outdated, I created a fork to update it and improve it with new features. You are invited to take a look at it and use it if you wish.