Importing dynamically all modules from a folder

Question

I know there is this question, but not only they're not working, it's not exactly what I want. I'm developing a racing game and want to load all tracks from a folder dynamically (They're stored as .py instead of .json). I don't want to know the names of the tracks, since users can mod/add them at will. I just want to import their data. So, for example:

>tracks 
>>track0.py
>>track1.py
>>track2.py
>>track3.py
>>track4.py

Inside each track, I have data like this:

track_ground_data = [
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
]

I need to import each track module like this:

loaded_tracks = [t for t in tracks] # Where tracks is the folder.

And then access a given track_ground_data like this:

loaded_tracks[0].track_ground_data

If I knew Python was going to be so harsh with its imports, I'd have used json instead .py.

Answer 1

Python does not automatically import submodules contained in a package. Hence import tracks only loads tracks/__init__.py.

However you can put code inside the __init__.py file that imports all the modules it finds in that directory.

For example putting something like this in the __init__.py:

import os
import importlib

for file in os.listdir(os.path.dirname(__file__)):
    mod_name = file.removesuffix(".py")
    if mod_name in ("__init__", "__pycache__"): continue
    importlib.import_module('.' + mod_name, package=__name__)

Should make your submodules available as tracks.trackX when importing only tracks.

Or you could use exec:

import os
import importlib

for file in os.listdir(os.path.dirname(__file__)):
    mod_name = file.removesuffix(".py")
    if mod_name in ("__init__", "__pycache__"): continue
    exec('import .' + mod_name)

---

A cleaner approach would be to use import hooks or implement your own custom module importer. There are multiple ways to do this using importlib see also sys.path_hooks

Answer 2

Just for the sake of the good wellfare of future pythoners, I'm posting how I've solved. A friend helped me through it. Coudln't make Bakuriu's solution work, because the modules came empty. Inside __init__.py I've put:

import os

dir = os.path.dirname(os.path.abspath(__file__))
modules = [os.path.splitext(_file)[0] for _file in os.listdir(dir) if not _file.startswith('__')]

tracks = []
for mod in modules:
    exec('from tracks import {}; tracks.append({})'.format(mod, mod))

And then, on the main file, I've loaded it as:

dir = os.path.dirname(os.path.abspath(__file__))
sys.path.append(dir)

from tracks import tracks

And then:

loaded_tracks = [t for t in tracks]

That actually solved it quite well. I was almost switching to JSON / giving up.