Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to run multiple notebooks in Google Colab

I have several different Notebooks in Google Colab that I use to automate some tasks. I want to create a single Notebook that runs all this tasks, so I can open a single tab with one single Notebook, run it and it will run all other tasks inside these different Notebooks.

I have two questions regarding this problem:

  1. The solution I found is not working (I will describe it below). How do I make it work?
  2. Is there a better solution than the one I found?

About the first question:

Image I have Notebook_1 and Notebook_2 each one with a bunch of functions that automate my tasks. What I am doing is, downloading them as Notebook_1.py and Notebook_2.py, saving these files in a Google Drive folder. Then in my Notebook_main, which is the notebook that should house all notebooks, I run:

# Mounts Google Drive
from google.colab import drive
drive.mount('/content/drive')

# Copies .py files into Google Colab
!cp /content/drive/MyDrive/my_modules/Notebook_1.py /content
!cp /content/drive/MyDrive/my_modules/Notebook_2.py /content

# Import Modules

import Notebook_1
import Notebook_2

If I want to run a simple function inside these modules I just do:

Notebook_1.run_simple_function()

and this works. My problem happens when the function I am trying to run from the Notebook_1, for example, uses another module. Then I get the following error:

name 'os' is not defined

I imagine it happens because inside Notebook_1.py I call:

import os
...
os.remove(os.path.join(dir_download, item))
...

And I also think this will happen with all the modules that I call inside Notebook_1.py.

I have tried importing theses modules in Notebook_main, but it did not work. I do not know how to fix this. I need help.

Another issue is that I use a lot of Selenium, which needs to be installed in Google Colab before being imported. So I need to install and import it in Notebook_main and when I run a function with Notebook_1.run_function_that_uses_selenium() it should use the import from Notebook_main.

The second question is simpler. I just want to know if there is a better way to achieve the same result, i.e. run different Notebooks in Google Colab from a single notebook.

My constriction is that I can only use Google Colab and other Google related Platforms, I can not run anything locally.

like image 999
user3347814 Avatar asked Sep 18 '25 08:09

user3347814


1 Answers

Simple.

from google.colab import drive
drive.mount('/content/gdrive')

then

libdir = "/content/gdrive/My\ Drive/Colab\ Notebooks/lib/"
%run {libdir}MyNotebook.ipynb
like image 152
Lex Avatar answered Sep 19 '25 21:09

Lex