Resolving circular dependencies in a python/django application

Question

I have a model that calls a file parser (to parse a file) and that file parser calls the model to save the object. Currently, the code looks something like this:

models.py

class Source(models.Model):
    ...

    def parse_file(self):
        from ingest.parser import FileParser
        ...

ingest.py

class FileParser()

    def save(self):
        from models import Source
        ...

This 'works' fine, however, doing the import within the save method adds about 0.25s the first time I have to use it, as it's initializing the import. Is there a better way to do the above?

Answer 1

When a module is first loaded, a module object with an empty namespace is placed immediately into sys.modules. The namespace is filled in as the module code is executed. Any further references to the module just retrieve the reference in sys.modues, regardless of whether it is completely loaded or not. This yields two approaches to the problem.

Method 1

Since the imported names are not used outside methods, you only need to ensure that they exist by the time the methods are called, not when they are first created.

You can fix the import issue by placing the offending imports at the end of your respective files. That way, no matter which module gets loaded first, all the top level names in it will be initialized before the other module tries to access them:

models.py

class Source(models.Model):
    ...

    def parse_file(self):
        ...

from ingest.parser import FileParser

ingest.py

class FileParser()
    def save(self):
        ...

from models import Source

If models.py is loaded first, the line from ingest.parser import FileParser will trigger a load of ingest.py, but only after Source is defined in the module namespace. That means that from models import Source will be able to find the name. The same applies in the reverse order.

If you know which module will always be loaded first, then only one of the imports needs to be moved to the end of the file (the one in the file being loaded first).

Method 2

A simpler alternative might be to just import the modules, rather than trying to extract names from them. That would allow you to keep the imports at the top of your file, since an empty module object will be available to satisfy the circular import:

models.py

from ingest import parser

class Source(models.Model):
    ...

    def parse_file(self):
        # use parser.FileParser
        ...

ingest.py

import models

class FileParser()
    def save(self):
        # use models.Source
        ...