Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python module with methods imported from a sub-module into root namespace

Tags:

python

module

I want to create a Python module that works like NumPy. The methods are not only sub-modules in the leaves of the tree from the module source. There is a root module containing many methods that I can call directly, and there are also sub-modules. The problem is the root methods must be defined somewhere. I was thinking to have a directory structure:

module/
  __init__.py
  core.py
  stuff1.py
  submodule/
    __init__.py
    stuff2.py
    stuff3.py

Now that I want is for everything inside "core" to be imported into the "module" namespace, as if it were a module.py file, and the contents of core.py were inside this module.py. The problem is that module is a directory instead of a file, so how do I define these methods that should sit in the root of the module?

I tried putting "from core import *" inside init.py, but that didn't work. (EDIT: Actually it does.)

Should I have the core methods inside a "module.py" file, and also a module directory? I don't know if that works, but it looks pretty awkward.

like image 520
dividebyzero Avatar asked Mar 11 '13 20:03

dividebyzero


People also ask

What is sub module in Python?

How is this done? In my limited experience, modules with submodules are simply folders with a __init__.py file, while modules with functions/classes are actual python files.

Can a Python module import another module?

Import in python is similar to #include header_file in C/C++. Python modules can get access to code from another module by importing the file/function using import. The import statement is the most common way of invoking the import machinery, but it is not the only way.

Can two modules import each other?

Modules can import other modules. It is customary but not required to place all import statements at the beginning of a module (or script, for that matter). The imported module names, if placed at the top level of a module (outside any functions or classes), are added to the module's global namespace.

What are the different methods of importing the Python module?

So there's four different ways to import: Import the whole module using its original name: pycon import random. Import specific things from the module: pycon from random import choice, randint. Import the whole module and rename it, usually using a shorter variable name: pycon import pandas as pd.


2 Answers

What I think you want is to be able to do this:

# some_other_script.py

import module
# Do things using routines defined in module.core

What happens when you ask Python to import module is (in a very basic sense), module/__init__.py is run, and a module object is created and imported into your namespace. This object (again, very basically) encompasses the things that happened when __init__.py was run: name definitions and so on. These can be accessed through module.something.

Now, if your setup looks like this:

# module/__init__.py

from module.core import Clazz
c = Clazz()
print c # Note: demo only! Module-level side-effects are usually a bad idea!

When you import module, you'll see a print statement like this:

<module.core.Clazz object at 0x00BBAA90>

Great. But if you then try to access c, you'll get a NameError:

# some_other_script.py
import module # prints "<module.core.Clazz object at 0x00BBAA90>"
print c # NameError (c is not defined)

This is because you haven't imported c; you've imported module. If instead your entry-point script looks like this:

# some_other_script.py
import module # prints "<module.core.Clazz object at 0x00BBAA90>"
print module.c  # Access c *within module*

Everything will run fine. This will also work fine with from core import * and/or from module import *, but I (and PEP8) advise against that just because it's not very clear what's going on in the script when you start mucking around with wild imports. For clarity:

# module/core.py

def core_func():
    return 1


# module/__init__.py

from core import *
def mod_func():
    return 2

The above is really pretty much fine, although you might as well make core "private" (rename to _core) to indicate that there's no reason to touch it from outside the package anymore.

# some_other_script.py

from module import *

print core_func() # Prints 1
print mod_func() # Prints 2
like image 187
Henry Keiter Avatar answered Sep 22 '22 12:09

Henry Keiter


Check out information about the __all__ list. It allows you to define what names are exported.

Tag it as such and you can setup a function to determine what to pull in from your submodules:

@property
all(self):
    #Whatever introspective code you may want for your modules
    __all__ += submodule.__all__

If you just want the whole damn shabang in module space, here's a way:

$ ipython
In [1]: from foomod import *

In [2]: printbar()
Out[2]: 'Imported from a foreign land'

In [3]: ^D
Do you really want to exit ([y]/n)?
$ ls foomod/
__init__.py __init__.pyc    core.py         core.pyc        submodule
$ grep . foomod/*.py 
foomod/__init__.py:from foomod.core import *
foomod/core.py:def printbar():
foomod/core.py:     return "Imported from a foreign land"

... and if we make __init__.py empty:

$ echo > foomod/__init__.py
$ ipython

In [1]: from foomod import *

In [2]: printbar()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-ba5b6693441e> in <module>()
----> 1 printbar()

NameError: name 'printbar' is not defined
like image 44
Jeff Ferland Avatar answered Sep 21 '22 12:09

Jeff Ferland