How to define a `from ... import *' api seperate from __all__ in a module?

Question

I have a module which is split roughly in half between utility type functions and core api functions. I need to have all of them in __all__ in order to have help() be useful, and also to specify which of the many functions/classes/etc in the module are for external use, but I also want to support from mymodule import * as a way to get just the core functionality into other modules. Is there a way to do that?

Answer 1

Almost. While you can't have __all__ do double duty in this way, you can add your own virtual api module which can then be imported...

class fake_module(object):
    def __init__(self, name, *args):
        self.name = name
        self.__all__ = []
        all_objects = globals()
        for name in args:
            self.__dict__[name] = all_objects[name]
            self.__all__.append(name)
    def register(self):
        sys.modules["%s.%s" % (__name__, self.name)] = self

    fake_module('api', 'class1', 'class2', 'func3', 'exception4').register()

Then in the other modules instead of from mymodule import * you can do from mymodule.api import * to get just the desired subset while still keeping everything in a single module.

Note: from ... import * is not usually good practice, and should be used with great care and only with modules/packages that explicity state that they have been designed with such usage in mind.

Answer 2

Ethan, your fake_module is likely better accomplished with plain old python. Just move your code into a directory setup:

mymodule
    __init__.py
    api.py
    util.py
    _mymodule.py

Where _mymodule.py contains your current code.

In your __init__.py:

from _mymodule import * 

In you're api.py

# explicity import only those api classes you want to expose
# this will allow you to call `from mymodule.api import *`
import from _mymodule ApiClass1
import from _mymodule ApiClass2
# etc  

Do the same for your util.py

# explicity import only those util classes you want to expose
# this will allow you to call `from mymodule.util import *`
import from _mymodule UtilClass1
import from _mymodule UtilClass2
# etc

So now you can:

# import everything willy nilly for your help()
from mymodule import *

# or import only the "public" values from api and util
from mymodule.api import *
from mymodule.util import *

Breaking up your code like this can be useful for organizing a largish module that you still want to be able to access from a top level namespace. But I would have to agree with brandizzi that using from mymodule import * in code that is using this module is generally a bad idea. What you gain in ease of importing you lose in transparency of the code that is using those imports.