Python submodule importing correctly in python 3.7 but not 3.6

Question

I have a directory structure as follows:

test/
  __init__.py
  m1/
    __init__.py
    f1.py
    f2.py

test/__init__.py is empty.

test/m1/__init__.py contains a single line import test.m1.f1.

test/m1/f1.py contains a single line import test.m1.f2 as f2.

In python 3.7.6, I can do import test.m1 and everything works as expected. However, in python 3.6.9 when I try this I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scratch/test/m1/__init__.py", line 2, in <module>
    import test.m1.f1
  File "/home/scratch/test/m1/f1.py", line 1, in <module>
    import test.m1.f2 as f2
AttributeError: module 'test' has no attribute 'm1'

This seems strange, because it does not error on the import test.m1.f1, which is the first thing it encounters. It errors on a subsequent import test.m1.f2 as f2 statement, claiming that test has no m1 submodule.

Answer 1

import test.m1.f2 as f2 tries to access the m1 attribute of the test module object, as part of the process of finding the object to bind to f2. The m1 attribute won't be set until the test.m1 subpackage finishes initializing, which won't happen until the __init__.py for test.m1 finishes executing.

On Python 3.7 and up, if the attribute lookup fails, the import falls back to a sys.modules['test.m1.f2'] lookup to find test.m1.f2. This fallback does not exist on 3.6, causing the observed discrepancy.

A similar fallback also exists for circular from imports on Python 3.5 and up, though from . import f2 or from test.m1 import f2 wouldn't need the fallback. The from import retrieves test.m1 straight from sys.modules, so it only looks for the f2 attribute, and that attribute is present.

Answer 2

This is not the answer explaining the how, but it explains how you can avoid the error. So useful for the ones wanting to write code that runs under 3.6 or earlier.

Replace

import test1.m1.f2 as f2

with

from test.m1 import f2

Or as @alaniwi pointed out

from . import f2