Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to import one databricks notebook into another?

I have a python notebook A in Azure Databricks having import statement as below:

import xyz, datetime, ...

I have another notebook xyz being imported in notebook A as shown in above code. When I run notebook A, it throws the following error:

ImportError: No module named xyz  

Both notebooks are in the same workspace directory. Can anyone help in resolving this?

like image 824
user39602 Avatar asked Apr 15 '19 08:04

user39602


People also ask

How do I clone a notebook in Databricks?

Cloning a Notebook Click File > Clone in the notebook context bar above. Enter a new name and location for your notebook. If Access Control is enabled, you can only clone to folders that you have Manage permissions on.

How do I copy multiple cells from one Databricks notebook to another?

Select multiple cells or all cells You can select adjacent notebook cells using Shift + Up or Down for the previous and next cell respectively. When multiple cells are selected, you can copy, cut, delete, and paste them. To select all cells, select Edit > Select All Cells or use the command mode shortcut Cmd+A.

How do you call a notebook from another notebook?

Running a Jupyter Notebook from Another Jupyter Notebook From the left Sidebar, select and right-click on the Jupyter notebook that has to be run from another notebook. From the context menu, select Copy Path. Open the Jupyter notebook from which you want to run another notebook. Click Run.


2 Answers

The only way to import notebooks is by using the run command:

%run /Shared/MyNotebook

or relative path:

%run ./MyNotebook

More details: https://docs.azuredatabricks.net/user-guide/notebooks/notebook-workflows.html

like image 131
simon_dmorias Avatar answered Oct 13 '22 17:10

simon_dmorias


To get the result back as a DataFrame from different notebook in Databricks we can do as below.

noebook1

def func1(arg):

    df=df.transfomationlogic
    return df

notbook2

%run path-of-notebook1

df=func1(**dfinput**)

Here the dfinput is a df you are passing and you will get the transformed df back from func1.

like image 38
RaHuL VeNuGoPaL Avatar answered Oct 13 '22 17:10

RaHuL VeNuGoPaL