How to properly deal with optional features in python

Question

I'm working on python packages that implement scientific models and I'm wondering what is the best way to handle optional features. Here's the behavior I'd like: If some optional dependencies can't be imported (plotting module on a headless machine for example), I'd like to disable the functions using these modules in my classes, warn the user if he tries to use them and all that without breaking the execution. so the following script would work in any cases:

mymodel.dostuff()
mymodel.plot() <= only plots if possible, else display log an error 
mymodel.domorestuff() <= get executed regardless of the result of the previous statement

So far the options I see are the following:

• check in the __init __.py for available modules and keep a list of them (but how to properly use it in the rest of the package?)
• for each function relying on optional dependencies have a try import ... except ... statement
• putting functions depending on a particular module in a separated file

These options should work, but they all seem to be rather hacky and hard to maintain. what if we want to drop a dependency completely? or make it mandatory?

Answer 1

The easiest solution, of course, is to simply import the optional dependencies in the body of the function that requires them. But the always-right PEP 8 says:

Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants.

Not wanting to go against the best wishes of the python masters, I take the following approach, which has several benefits...

First, import with an try-except

Say one of my functions foo needs numpy, and I want to make it an optional dependency. At the top of the module, I put:

try:
    import numpy as _numpy
except ImportError:
    _has_numpy = False
else:
    _has_numpy = True

Here (in the except block) would be the place to print a warning, preferably using the warnings module.

Then throw the exception in the function

What if the user calls foo and doesn't have numpy? I throw the exception there and document this behaviour.

def foo(x):
    """Requires numpy."""
    if not _has_numpy:
        raise ImportError("numpy is required to do this.")
    ...

Alternatively you can use a decorator and apply it to any function requiring that dependency:

@requires_numpy
def foo(x):
    ...

This has the benefit of preventing code duplication.

And add it as an optional dependency to your install script

If you're distributing code, look up how to add the extra dependency to the setup configuration. For example, with setuptools, I can write:

install_requires = ["networkx"],

extras_require = {
    "numpy": ["numpy"],
    "sklearn": ["scikit-learn"]}

This specifies that networkx is absolutely required at install time, but that the extra functionality of my module requires numpy and sklearn, which are optional.

---

Using this approach, here are the answers to your specific questions:

• What if we want to make a dependency mandatory?

We can simply add our optional dependency to our setup tool's list of required dependencies. In the example above, we move numpy to install_requires. All of the code checking for the existence of numpy can then be removed, but leaving it in won't cause your program to break.

• What if we want to drop a dependency completely?

Simply remove the check for the dependency in any function that previously required it. If you implemented the dependency check with a decorator, you could just change it so that it simply passes the original function through unchanged.

This approach has the benefit of placing all of the imports at the top of the module so that I can see at a glance what is required and what is optional.