How do I mock the filesystem in Python unit tests?

Question

Is there a standard way (without installing third party libraries) to do cross platform filesystem mocking in Python? If I have to go with a third party library, which library is the standard?

Answer 1

pyfakefs (homepage) does what you want – a fake filesystem; it’s third-party, though that party is Google. See How to replace file-access references for a module under test for discussion of use.

For mocking, unittest.mock is the standard library for Python 3.3+ (PEP 0417); for earlier version see PyPI: mock (for Python 2.5+) (homepage).

Terminology in testing and mocking is inconsistent; using the Test Double terminology of Gerard Meszaros, you’re asking for a “fake”: something that behaves like a filesystem (you can create, open, and delete files), but isn’t the actual file system (in this case it’s in-memory), so you don’t need to have test files or a temporary directory.

In classic mocking, you would instead mock out the system calls (in Python, mock out functions in the os module, like os.rm and os.listdir), but that’s much more fiddly.

Answer 2

pytest is gaining a lot of traction, and it can do all of this using tmpdir and monkeypatching (mocking).

You can use the tmpdir function argument which will provide a temporary directory unique to the test invocation, created in the base temporary directory (which are by default created as sub-directories of the system temporary directory).

import os
def test_create_file(tmpdir):
    p = tmpdir.mkdir("sub").join("hello.txt")
    p.write("content")
    assert p.read() == "content"
    assert len(tmpdir.listdir()) == 1

The monkeypatch function argument helps you to safely set/delete an attribute, dictionary item or environment variable or to modify sys.path for importing.

import os
def test_some_interaction(monkeypatch):
    monkeypatch.setattr(os, "getcwd", lambda: "/")

You can also pass it a function instead of using lambda.

import os.path
def getssh(): # pseudo application code
    return os.path.join(os.path.expanduser("~admin"), '.ssh')

def test_mytest(monkeypatch):
    def mockreturn(path):
        return '/abc'
    monkeypatch.setattr(os.path, 'expanduser', mockreturn)
    x = getssh()
    assert x == '/abc/.ssh'

# You can still use lambda when passing arguments, e.g.
# monkeypatch.setattr(os.path, 'expanduser', lambda x: '/abc')

If your application has a lot of interaction with the file system, then it might be easier to use something like pyfakefs, as mocking would become tedious and repetitive.