Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Import structure that works both in packages and out, in both Python 2 and 3?

When I developed a package purely for Python 2, I could use the plain import b syntax to import a relative path without caring about whether the importing file was in a package or not. This had the advantage that I could run an if __name__ == "__main__": block of any file simply by executing the file, and all imports would work fine.

After adding Python 3 support, I had to move to the new relative import syntax, which is also supported by 2.7: from . import b. However, this syntax only works inside packages. Directly executing the file directly no longer works:

Traceback (most recent call last):
  File "./a.py", line 2, in <module>
    from . import b
ValueError: Attempted relative import in non-package

A workaround is to call the file by importing it as a module from an upper directory:

python -m foo.a

However, this places a requirement on the work directory, which prevents you from piping the output to some other program that also cares about the working directory.

Is there a way to have your cake and eat it too? I.e. support both running as a script and importing as part of a package, while working in both Python 2 and 3?


Example package structure:

foo/
foo/__init__.py
foo/a.py (imports b)
foo/b.py (imports c)
foo/c.py

I would like both of the following to work for x in (a, b, c):

import foo.x (in some file when foo/ is in path)

python[23] path/to/foo/x.py

The comment below mentions setting __package__ according to PEP 366, but "if the script is moved to a different package or subpackage, the boilerplate will need to be updated manually."

Update: I tried to make the PEP 366 solution work, but couldn't figure it out. It says:

Additional code that manipulates sys.path would be needed in order for direct execution to work without the top level package already being importable.

That's the case when executing a file from a non-imported package. What would that additional code look like?

like image 212
otus Avatar asked May 30 '14 10:05

otus


People also ask

What are the two types of import in Python?

There are generally three groups: standard library imports (Python's built-in modules) related third party imports (modules that are installed and do not belong to the current application) local application imports (modules that belong to the current application)

What is __ import __ in Python?

__import__() . This means all semantics of the function are derived from importlib. __import__() . The most important difference between these two functions is that import_module() returns the specified package or module (e.g. pkg. mod ), while __import__() returns the top-level package or module (e.g. pkg ).

Which operator is used in the Python to import all modules from packages?

We can import modules from packages using the dot (.) operator.

How do module imports work in Python?

In Python, you use the import keyword to make code in one module available in another. Imports in Python are important for structuring your code effectively. Using imports properly will make you more productive, allowing you to reuse code while keeping your projects maintainable.


1 Answers

Is there a way to have your cake and eat it too? I.e. support both running as a script and importing as part of a package, while working in both Python 2 and 3?

No..... Maybe... But from what I can gather of what you're trying to do, you're making a problem even more complicated that it should be. I would create a package like normal where you can support both Python 2 & 3. Then install using a setup script and import the package into the script without having to use relative paths. This gives you the freedom of script execution anywhere, and Python 2 & 3 compatibility of your package.


I still hold to my original statement above in that I believe you're making this more complicated than it really needs to be OR you're not giving us all the information as to why this HAS to be done this way. None the less, If you follow what PEP 366 states this should work. In your modules where the script resides (i.e. contains if __name__ == "__main__":) then add the following lines to the beginning (or before your main if __name__ == "__main__":) of the file:

if __name__ == "__main__" and __package__ == None:
        __package__ == "expected.package.name"
        sys.path.append(<path to root package 'expected'>)

This of course means that you'll need to manually update these if you ever move the script around, or if the package gets moved, or anything related to that path gets moved around (hence why I still think an installation is by far the better option here).

like image 69
James Mertz Avatar answered Sep 27 '22 17:09

James Mertz