Importing from python modules inside parent directory into jupyter notebook files inside subdirectory

Question

I have a file structure like this:

project_folder/
     notebooks/
          notebook01.ipynb
          notebook02.ipynb
          ...
          notebookXY.ipynb
     module01.py
     module02.py
     module03.py

In .ipynb files inside notebook/ folder I want to import classes and functions from module01.py, module02.py and module03.py.

I have found answer in this question that it is possible using following lines of code inside every notebook and run those lines as first cell every time:

import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)

Is there please a better way for this? What if I have A LOT of .ipynb files inside notebooks/ folder, do I have to paste those lines of code at the beginning of every single one? Is there a better, more minimalist or cleaner way?

Answer 1

Another solution is to move all your Python modules (.py files) into a folder and make them an installable package. If you pip install it into your current environment, you can then import the package into any notebook in that environment, regardless of folder structure.

So in your situation you could have:

project_folder/
  notebooks/
    notebook01.ipynb
    notebook02.ipynb
    ...
    notebookXY.ipynb
  my_package/
    __init__.py
    module01.py
    module02.py
    module03.py
  setup.py

• __init__.py can just be an empty file, and tells Python "everything in this folder is part of a package"
• For an explanation of what goes in setup.py see here.

A basic setup.py can be as simple as this:

import setuptools

setuptools.setup(
    name="my_package",
    version="0.0.1",
    description="A small example package",
    packages=setuptools.find_packages(),
    python_requires='>=3.7',
)

Install it:

cd project_folder
pip install [-e] .

Including the optional -e flag will install my_package in "editable" mode, meaning that instead of copying the files into your virtual environment, a symlink will be created to the files where they are.

Now in any notebook you can do:

import my_package

Or

from my_package.module01 import <some object>

Answer 2

Try adding the project_folder to your PYTHONPATH environment variable. This will allow you to tell python to search that directory for imports.

You would do this in your user profile settings, or in your startup script - not in python. It's something that has to be set before python ever gets run.