Alembic. ModuleNotFoundError in env.py

Question

I have the following project structure

 --some db:
        --some db:
            --alchemy:
                -- __init__.py
            --alembic:
                -- versions
                -- env.py
                -- README.py
                -- script.py
            --migrations:
                -- __init__.py
            --models:
                -- model_1
                -- model_2
                -- __init__.py

I try to autogenerate migrations by alembic.

I have Base in __init__.py in models folder

import sqlalchemy as sa

from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base, declared_attr


metadata = sa.MetaData()
Base = declarative_base(metadata=metadata)

And import this is env.py

from logging.config import fileConfig

from alembic import context
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from models import Base

config = context.config
fileConfig(config.config_file_name)
target_metadata = Base.metadata

So, when I import Base from models in env.py in alembic directory and try to generate automigration I have some error like

ModuleNotFoundError: No module named 'models'

How can I fix this error?

Answer 1

The problem is that when env.py is executed, models is not in your PYTHONPATH, so it can't be imported.

With the project structure you've outlined, it might be a little hard to address; the easiest solution might be to modify your PYTHONPATH inside env.py like this:

import sys

sys.path = ['', '..'] + sys.path[1:]

from models import Base

This will add the parent directory of alembic/ to your PYTHONPATH so that it can find the models module.

Alternatively, you add the directory containing your modules to your PYTHONPATH environment variable in your shell:

$ export PYTHONPATH='/path/to/some db/some db':$PYTHONPATH

This solution is a little more brittle because you have to remember to do this each session and it will be different for each machine you intend to run Alembic on.

When I ran into this issue, the SQLAlchemy developer suggested that if I was using pip and virtual environments, I could install my project in editable mode so that it's in the system PYTHONPATH and Alembic would be able to find it from anywhere. More details are available in the Python packaging guide. For this, you'd need a setup.py for your project and you may want to change your project structure so that there's a top-level module containing things like models and alchemy. For example:

myproj/
    setup.py
    alembic/
        env.py
        migrations/
    myapp/
        __init__.py
        alchemy/
            __init__.py
        models/
            __init__.py

If this is set up correctly, you can

$ pip install -e .

from inside the myproj/ directory. Then in env.py you would then import Base like this:

from myapp.models import Base

Answer 2

Now there is a beautiful way: https://github.com/sqlalchemy/alembic/commit/d6b0c1af3df98b50c6ec52781aa411592c4e0c32

"...the default "alembic.ini" file includes a directive prepend_sys_path = . so that the local path is also in sys.path.

Answer 3

When a module is loaded from a file in Python, __file__ is set to its path. You can then use that with other functions to find the directory that the file is located in.

Add the following line in alembic/env.py file to resolve the ModuleNotFoundError

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASE_DIR)