how to find list of modules which depend upon a specific module in python

Question

In order to reduce development time of my Python based web application, I am trying to use reload() for the modules I have recently modified. The reload() happens through a dedicated web page (part of the development version of the web app) which lists the modules which have been recently modified (and the modified time stamp of py file is later than the corresponding pyc file). The full list of modules is obtained from sys.modules (and I filter the list to focus on only those modules which are part of my package).

Reloading individual python files seems to work in some cases and not in other cases. I guess, all the modules which depend on a modified module should be reloaded and the reloading should happen in proper order.

I am looking for a way to get the list of modules imported by a specific module. Is there any way to do this kind of introspection in Python?

I understand that my approach might not be 100% guaranteed and the safest way would be to reload everything, but if a fast approach works for most cases, it would be good enough for development purposes.

Response to comments regarding DJango autoreloader

@Glenn Maynard, Thanx, I had read about DJango's autoreloader. My web app is based on Zope 3 and with the amount of packages and a lot of ZCML based initializations, the total restart takes about 10 seconds to 30 seconds or more if the database size is bigger. I am attempting to cut down on this amount of time spent during restart. When I feel I have done a lot of changes, I usually prefer to do full restart, but more often I am changing couple of lines here and there for which I do not wish to spend so much of time. The development setup is completely independent of production setup and usually if something is wrong in reload, it becomes obvious since the application pages start showing illogical information or throwing exceptions. Am very much interested in exploring whether selective reload would work or not.

Answer 1

Some introspection to the rescue:

from types import ModuleType

def find_modules(module, all_mods = None):
   if all_mods is None:
      all_mods = set([module])
   for item_name in dir(module):
       item = getattr(module, item_name)
       if isinstance(item, ModuleType) and not item in all_mods:
           all_mods.add(item)
           find_modules(item, all_mods)
   return all_mods

This gives you a set with all loaded modules - just call the function with your first module as a sole parameter. You can then iterate over the resulting set reloading it, as simply as: [reload (m) for m in find_modules(<module>)]

Answer 2

You might want to take a look at Ian Bicking's Paste reloader module, which does what you want already:

http://pythonpaste.org/modules/reloader?highlight=reloader

It doesn't give you specifically a list of dependent files (which is only technically possible if the packager has been diligent and properly specified dependencies), but looking at the code will give you an accurate list of modified files for restarting the process.