Faking an IO Error on Linux

Question

I have a Python and C application on Linux that's supposed to properly handle IO errors whilst reading files from disk. The bulk of the application is written in Python, with a C extension that does the IO. It's within this extension that the IO errors are detected.

There are two cases that the errors appear to occur for me.

1. A file is missing.
2. A file appears larger on disk (using stat) than can be read using fread.

I can test and handle case number 1 rather easily. However, I'd also like to write a unit test for case 2. However, I have no idea how to trigger a "fake" IO error for the test. Is this even possible? Is there a better approach to testing this kind of error?

Answer 1

errno(3) is set to EIO only for

   EIO    Input/output error (POSIX.1)

also, according to read(2) for:

   EIO    I/O error.  This will happen for example when the process is
          in a background process group, tries to read from its
          controlling terminal, and either it is ignoring or blocking
          SIGTTIN or its process group is orphaned.  It may also occur
          when there is a low-level I/O error while reading from a 
          disk or tape.

and according to write(2) for:

   EIO    A low-level I/O error occurred while modifying the inode.

So simulating that particular error code could be difficult; notice that there are other syscalls for I/O, notably writev(2) and (indirectly) mmap(2), but read(2) and write(2) are the most common ones.

Notice also that file systems and the Linux kernel (e.g. its VFS layer) are caching data. You could get EIO much later or never. See sync(2) and fsync(2)

However, generally, most software does not handle EIO specially w.r.t. other error codes; you probably are testing enough by getting another error code, like e.g.

  EDQUOT The user's quota of disk blocks on the filesystem containing
          the file referred to by fd has been exhausted.

So you'll probably test enough by limiting disk quotas (see quotactl(2), setquota(8) etc...) and file space (see setrlimit(2) with RLIMIT_FSIZE, prlimit(1), ulimit builtin of bash(1) etc...)

If you really want to fake specifically EIO you could physically damage a device (or perhaps just unplug an USB disk at the wrong moment) or write your own Filesystem in User Space (FUSE) simulating it. I don't think it is worth the effort (because when something gets EIO the entire computer becomes very quickly unusable, and the user will notice that anyway.... and because most software handle all error codes likewise -except for EINTR)

In the C part of your code you may want to use strerror(3) (with syslog(3) perhaps) and/or perror(3). I am not sure it is worth the effort to handle EIO very differently of most other errors.

NB: many critical domains have standards defining how errors should be handled and code should be developed and tested, e.g. ISO26262 in automotive or DO-178B in avionics. Follow the standards of your domain.

Answer 2

As far as I understand the matter, classics of TDD warn us against writing mocks/stubs for 3rd-party interfaces (including standard library), see e.g. here. The major issue is that there is usually a gap between the application code and generic-purpose 3rd-party library which is hard to tie with mock-objects. Also, that prevents you from using tests to derive the design issues.

(Even though in your case the C library is not exactly 3rd party, unit-testing means that you test the entities in isolation).

The idea is that instead you write an adaptor class that encapsulates all the low-level logic and exposes an interface close to what your application needs (and, for example, raises more meaningful exceptions, like FileIsTooBig). Then you write mock-objects in terms of your domain. As for testing the adaptor itself, it's tested with few simple system tests.

Answer 3

Use fusepy.

fusepy is a python layer on top of FUSE, which allows file systems to be implemented in the Linux user space. fusepy is a Python module that provides a simple interface to FUSE and MacFUSE. It's just one file and is implemented using ctypes. With fusepy, you can modify the behavior of the write function implementation and throw an EIO if you want to. I'd use the memory.py example as a base.

Answer 4

libfiu (as mentioned in the "How can I simulate a failed disk during testing?" answer) is a structured approach to using interposing to perform fault injection of POSIX calls and would be ideal for use in test suites.

A more general list of techniques (such as using a FUSE filesystem) are mentioned in the list of Linux disk fault injection mechanisms answer to the "Special File that causes I/O error" question.