Properly importing modules in Python

Question

How do I set up module imports so that each module can access the objects of all the others?

I have a medium size Python application with modules files in various subdirectories. I have created modules that append these subdirectories to sys.path and imports a group of modules, using import thisModule as tm. Module objects are referred to with that qualification. I then import that module into the others with from moduleImports import *. The code is sloppy right now and has several of these things, which are often duplicative.

First, the application is failing because some module references aren't assigned. This same code does run when unit tested.

Second, I'm worried that I'm causing a problem with recursive module imports. Importing moduleImports imports thisModule, which imports moduleImports . . . .

What is the right way to do this?

Answer 1

"I have a medium size Python application with modules files in various subdirectories."

Good. Make absolutely sure that each directory include a __init__.py file, so that it's a package.

"I have created modules that append these subdirectories to sys.path"

Bad. Use PYTHONPATH or install the whole structure Lib/site-packages. Don't update sys.path dynamically. It's a bad thing. Hard to manage and maintain.

"imports a group of modules, using import thisModule as tm."

Doesn't make sense. Perhaps you have one import thisModule as tm for each module in your structure. This is typical, standard practice: import just the modules you need, no others.

"I then import that module into the others with from moduleImports import *"

Bad. Don't blanket import a bunch of random stuff.

Each module should have a longish list of the specific things it needs.

import this
import that
import package.module

Explicit list. No magic. No dynamic change to sys.path.

My current project has 100's of modules, a dozen or so packages. Each module imports just what it needs. No magic.

Answer 2

Few pointers

1. You may have already split functionality in various module. If correctly done most of the time you will not fall into circular import problems (e.g. if module a depends on b and b on a you can make a third module c to remove such circular dependency). As last resort, in a import b but in b import a at the point where a is needed e.g. inside function.

2. Once functionality is properly in modules group them in packages under a subdir and add a __init__.py file to it so that you can import the package. Keep such pakages in a folder e.g. lib and then either add to sys.path or set PYTHONPATH env variable

3. from module import * may not be good idea. Instead, import whatever is needed. It may be fully qualified. It doesn't hurt to be verbose. e.g. from pakageA.moduleB import CoolClass.

Answer 3

The way to do this is to avoid magic. In other words, if your module requires something from another module, it should import it explicitly. You shouldn't rely on things being imported automatically.

As the Zen of Python (import this) has it, explicit is better than implicit.