Python and conflicting module names

Question

It seems that if a file is called io.py and it imports scipy.ndimage, the latter somehow ends up failing to find its own submodule, also called io:

$ echo "import scipy.ndimage" > io.py
$ python io.py 
Traceback (most recent call last):
  File "io.py", line 1, in <module>
    import scipy.ndimage
  File "/usr/lib/python2.7/dist-packages/scipy/__init__.py", line 70, in <module>
    from numpy import show_config as show_numpy_config
  File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 153, in <module>
    from . import add_newdocs
  File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 22, in <module>
    from .npyio import *
  File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 4, in <module>
    from . import format
  File "/usr/lib/python2.7/dist-packages/numpy/lib/format.py", line 141, in <module>
    import io
  File "/tmp/rm_me/io.py", line 1, in <module>
    import scipy.ndimage
  File "/usr/lib/python2.7/dist-packages/scipy/ndimage/__init__.py", line 172, in <module>
    from .filters import *
  File "/usr/lib/python2.7/dist-packages/scipy/ndimage/filters.py", line 37, in <module>
    from scipy.misc import doccer
  File "/usr/lib/python2.7/dist-packages/scipy/misc/__init__.py", line 45, in <module>
    from .common import *
  File "/usr/lib/python2.7/dist-packages/scipy/misc/common.py", line 10, in <module>
    from numpy import exp, log, asarray, arange, newaxis, hstack, product, array, \
ImportError: cannot import name exp

Is this a bug in SciPy, or am I using Python wrong?

Update: I think less surprising behavior would be if import mod2 in mod1 resolved paths relative to mod1 rather than relative to whomever imported mod1.

Answer 1

The simple fix is to avoid naming your module io, because it's conflicting with a core library module name.

It's not really a bug in numpy, but user error: just as we shouldn't use list as a variable name because it's shadowing the builtin list name, we shouldn't use io as a module name because it's shadowing the core library io module name.

See this part:

  File "/usr/lib/python2.7/dist-packages/numpy/lib/format.py", line 141, in <module>
    import io

Here numpy tries to import the io module. This is intended to find the stdlib io because of the absolute import - a numpy submodule would have been loaded with relative import like you can see in the traceback where there is another line beginning from .npyio import *.

Of course your own io.py module is found first, because in the current working directory is generally the first entry in sys.path. Whoops!

Answer 2

You're using python wrong.

Before you create any top-level python module or package, you should make sure there isn't already a module or package by that name.

The best solution here is to not use top-level modules, and instead put everything in a single top-level package (i.e. directory with an __init__.py file) named after your project.

To check if a top-level module or package exists, you could try importing it in the interpreter, or running pydoc name from the shell.

---

It is worth noting that there is a very similar error that can happen that is not your fault, if a package uses absolute-style import syntax to perform a relative import. This "feature" is removed in Python3.