Python failure injection

Question

Is there a neat way to inject failures in a Python script? I'd like to avoid sprinkling the source code with stuff like:

failure_ABC = True
failure_XYZ = True
def inject_failure_ABC():
    raise Exception('ha! a fake error')
def inject_failure_XYZ():
    # delete some critical file
    pass

# some real code
if failure_ABC:
    inject_failure_ABC()
# some more real code
if failure_XYZ:
    inject_failure_XYZ()
# even more real code

Edit: I have the following idea: insert "failure points" as specially-crafted comments. The write a simple parser that will be called before the Python interpreter, and will produce the actual instrumented Python script with the actual failure code. E.g:

#!/usr/bin/parser_script_producing_actual_code_and_calls python
# some real code
# FAIL_123
if foo():
    # FAIL_ABC
    execute_some_real_code()
else:
    # FAIL_XYZ
    execute_some_other_real_code()

Anything starting with FAIL_ is considered as a failure point by the script, and depending on a configuration file the failure is enabled/disabled. What do you think?

Answer 1

You could use mocking libraries, for example unittest.mock, there also exist many third party ones as well. You can then mock some object used by your code such that it throws your exception or behaves in whatever way you want it to.

Answer 2

When testing error handling, the best approach is to isolate the code that can throw errors in a new method which you can override in a test:

class ToTest:
    def foo(...):
        try:
            self.bar() # We want to test the error handling in foo()
        except:
            ....

    def bar(self):
        ... production code ...

In your test case, you can extend ToTest and override bar() with code that throws the exceptions that you want to test.

EDIT You should really consider splitting large methods into smaller ones. It will make the code easier to test, to understand and to maintain. Have a look at Test Driven Development for some ideas how to change your development process.

Regarding your idea to use "Failure Comments". This looks like a good solution. There is one small problem: You will have to write your own Python parser because Python doesn't keep comments when it produces bytecode.

So you can either spend a couple of weeks to write this or a couple of weeks to make your code easier to test.

There is one difference, though: If you don't go all the way, the parser will be useless. Also, the time spent won't have improved one bit of your code. Most of the effort will go into the parser and tools. So after all that time, you will still have to improve the code, add failure comments and write the tests.

With refactoring the code, you can stop whenever you want but the time spent so far will be meaningful and not wasted. Your code will start to get better with the first change you make and it will keep improving.

Writing a complex tool takes time and it will have it's own bugs which need to fix or work around. None of this will improve your situation in the short term and you don't have a guarantee that it will improve the long term.