Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Imports in __init__.py and 'import as' statement

I've run into a problem with having imports in __init__.py and using import as with absolute imports in modules of the package.

My project has a subpackage and in its __init__.py I "lift" one of the classes from a module to the subpackage level with from import as statement. The module imports other modules from that subpackage with absolute imports. I get this error AttributeError: 'module' object has no attribute 'subpkg'.

Example

Structure:

pkg/
├── __init__.py
├── subpkg
│   ├── __init__.py
│   ├── one.py
│   └── two_longname.py
└── tst.py

pkg/init.py is empty.

pkg/subpkg/init.py:

from pkg.subpkg.one import One

pkg/subpkg/one.py:

import pkg.subpkg.two_longname as two

class One(two.Two):
    pass

pkg/subpkg/two_longname.py:

class Two:
    pass

pkg/tst.py:

from pkg.subpkg import One

print(One)

Output:

$ python3.4 -m pkg.tst
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/and/dev/test/python/imptest2/pkg/tst.py", line 1, in <module>
    from pkg.subpkg import One
  File "/home/and/dev/test/python/imptest2/pkg/subpkg/__init__.py", line 1, in <module>
    from pkg.subpkg.one import One
  File "/home/and/dev/test/python/imptest2/pkg/subpkg/one.py", line 1, in <module>
    import pkg.subpkg.two_longname as two
AttributeError: 'module' object has no attribute 'subpkg'

Workarounds

There are changes that make it work:

  1. Empty pkg/subpkg/__init__.py and importing directly from pkg.subpkg.one.

    I don't consider this as an option because AFAIK "lifting" things to the package level is ok. Here is quote from an article:

    One common thing to do in your __init__.py is to import selected Classes, functions, etc into the package level so they can be conveniently imported from the package.

  2. Changing import as to from import in one.py:

     from pkg.subpkg import two_longname
    
     class One(two_longname.Two):
         pass
    

    The only con here is that I can't create a short alias for module. I got that idea from @begueradj's answer.

It is also possible to use a relative import in one.py to fix the problem. But I think it's just a variation of workaround #2.

Questions

  1. Can someone explain what is actually going on here? Why a combination of imports in __init__.py and usage of import as leads to such problems?

  2. Are there any better workarounds?


Original example

This is my original example. It's not very realistic but I'm not deleting it so @begueradj's answer still makes sense.

pkg/init.py is empty.

pkg/subpkg/init.py:

from pkg.subpkg.one import ONE

pkg/subpkg/one.py:

import pkg.subpkg.two
ONE = pkg.subpkg.two.TWO

pkg/subpkg/two.py:

TWO = 2

pkg/tst.py:

from pkg.subpkg import ONE

Output:

$ python3.4 -m pkg.tst
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/and/dev/test/python/imptest/pkg/tst.py", line 1, in <module>
    from pkg.subpkg import ONE
  File "/home/and/dev/test/python/imptest/pkg/subpkg/__init__.py", line 2, in <module>
    from pkg.subpkg.one import ONE
  File "/home/and/dev/test/python/imptest/pkg/subpkg/one.py", line 6, in <module>
    ONE = pkg.subpkg.two.TWO
AttributeError: 'module' object has no attribute 'subpkg'

Initially I had this in one.py:

import pkg.subpkg.two as two
ONE = two.TWO

In that case I get error on import (just like in my original project which uses import as too).

like image 217
and Avatar asked Jul 17 '14 15:07

and


People also ask

What does adding __ init __ py do?

In addition to labeling a directory as a Python package and defining __all__ , __init__.py allows you to define any variable at the package level. Doing so is often convenient if a package defines something that will be imported frequently, in an API-like fashion.

What is import as in Python?

What is Importing? Importing refers to allowing a Python file or a Python module to access the script from another Python file or module. You can only use functions and properties your program can access. For instance, if you want to use mathematical functionalities, you must import the math package first.

What are the two types of import in Python?

There are generally three groups: standard library imports (Python's built-in modules) related third party imports (modules that are installed and do not belong to the current application) local application imports (modules that belong to the current application)

What is import statement in Python with example?

The import statement syntax is: import modulename. Python is accompanied by a number of built-in modules that allow you to perform common operations in your code. int(), for example, converts a value to an integer. sum() calculates the sum of all items in a list.


Video Answer


3 Answers

You incorrectly assume that one cannot have an alias with from ... import, as from ... import ... as has been there since Python 2.0. The import ... as is the obscure syntax that not many know about, but which you use by accident in your code.

PEP 0221 claims that the following 2 are "effectively" the same:

  1. import foo.bar.bazaar as baz
  2. from foo.bar import bazaar as baz

The statement is not quite true in Python versions up to and including 3.6.x as evidenced by the corner case you met, namely if the required modules already exist in sys.modules but are yet uninitialized. The import ... as requires that the module foo.bar is injected in foo namespace as the attribute bar, in addition to being in sys.modules, whereas the from ... import ... as looks for foo.bar in sys.modules.

(Do note also that import foo.bar only ensures that the module foo.bar is in sys.modules and accessible as foo.bar, but might not be fully initialized yet.)

Changing the code as follows did the trick for me:

# import pkg.subpkg.two_longname as two from pkg.subpkg import two_longname as two 

And code runs perfectly on both Python 2 and Python 3.

Also, in one.py you cannot do from pkg import subpkg, for the same reason.


To demonstrate this bug further, fix your one.py as above, and add the following code in tst.py:

import pkg import pkg.subpkg.two_longname as two  del pkg.subpkg  from pkg.subpkg import two_longname as two import pkg.subpkg.two_longname as two 

Only the last line crashes, because from ... import consults the sys.modules for pkg.subpkg and finds it there, whereas import ... as consults sys.modules for pkg and tries to find subpkg as an attribute in the pkg module. As we just had deleted that attribute, the last line fails with AttributeError: 'module' object has no attribute 'subpkg'.


As the import foo.bar as baz syntax is a bit obscure and adds more corner cases, and I have rarely if ever seen it being used, I would recommend avoiding it completely and favouring from .. import ... as.

like image 124

Here is a theory on what's going on.

When you use the as reserved word, for instance:

import pkg.subpkg.two_longname as two 

Python must to completely initialize and resolve all dependences that has to do with pkg.subpkg. But there is a problem, to completely load subpkg you need to completely load one.py as well right? wich at the same time imports two_longname.py using the as keyword ... Can you see the recursion here? That's why at the moment of doing:

import pkg.subpkg.two_longname as two 

you get an error claiming subpkg does not exist.

To perform a test, go to one.py and change it to this:

#import pkg.subpkg.two_longname as two from pkg.subpkg import two_longname  #class One(two.Two): class One(two_longname.Two):     pass 

I suppose this is all about performance, Python loads a module partially whenever is possible. And the as keyword is one of the exceptions. I don't know if there are others, but it would be interesting know about them.

like image 35
Raydel Miranda Avatar answered Sep 19 '22 08:09

Raydel Miranda


As the accepted answer states this is an issue with Python's behavior.

I've filed a bug: http://bugs.python.org/issue30024

The fix by Serhiy Storchaka was merged and expected in Python 3.7

like image 40
warvariuc Avatar answered Sep 19 '22 08:09

warvariuc