Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can an IPython/Jupyter notebook cell be imported as if it were a module?

Is there a means in IPython to import the contents of a notebook cell as if it were a separate module? Or alternatively get the contents of a cell to have its own namespace.

like image 482
Mike Avatar asked Jul 12 '16 19:07

Mike


1 Answers

@Mike, as mentioned in the comment you can follow the well documented steps in the following link to import a Jupyter Notebook as a module:

Importing Jupyter Notebooks as Modules

In the link they will mention the work done in Python to provide users with hooks (now superseded with importlib and import system) to provide better customization of the import mechanism.

As so the recipe they propose is the following:

  • load the notebook document into memory
  • create an empty Module
  • execute every cell in the Module namespace

, and they offer their own implementation for the Notebook Loader (unnecessary if the code is all pure python):

class NotebookLoader(object):
    """Module Loader for Jupyter Notebooks"""
    def __init__(self, path=None):
        self.shell = InteractiveShell.instance()
        self.path = path

    def load_module(self, fullname):
        """import a notebook as a module"""
        path = find_notebook(fullname, self.path)

        print ("importing Jupyter notebook from %s" % path)

        # load the notebook object
        with io.open(path, 'r', encoding='utf-8') as f:
            nb = read(f, 4)


        # create the module and add it to sys.modules
        # if name in sys.modules:
        #    return sys.modules[name]
        mod = types.ModuleType(fullname)
        mod.__file__ = path
        mod.__loader__ = self
        mod.__dict__['get_ipython'] = get_ipython
        sys.modules[fullname] = mod

        # extra work to ensure that magics that would affect the user_ns
        # actually affect the notebook module's ns
        save_user_ns = self.shell.user_ns
        self.shell.user_ns = mod.__dict__

        try:
          for cell in nb.cells:
            if cell.cell_type == 'code':
                # transform the input to executable Python
                code = self.shell.input_transformer_manager.transform_cell(cell.source)
                # run the code in themodule
                exec(code, mod.__dict__)
        finally:
            self.shell.user_ns = save_user_ns
        return mod

Also here is the implementation for the Notebook Finder:

class NotebookFinder(object):
    """Module finder that locates Jupyter Notebooks"""
    def __init__(self):
        self.loaders = {}

    def find_module(self, fullname, path=None):
        nb_path = find_notebook(fullname, path)
        if not nb_path:
            return

        key = path
        if path:
            # lists aren't hashable
            key = os.path.sep.join(path)

        if key not in self.loaders:
            self.loaders[key] = NotebookLoader(path)
        return self.loaders[key] 

And the final step being the registration of the new module:

sys.meta_path.append(NotebookFinder())

All of this is, however, a direct quote from the first link given in this answer. The document is well built and provides answers for other stuff such as displaying notebooks or dealing with packages.

like image 138
armatita Avatar answered Oct 07 '22 17:10

armatita