How can I reference the path of a notebook in Databricks/what is %run doing?

Question

I'm familiar with the %run magic commands in Databricks, but where do the notebooks actually live?

The rootdir using %sh pwd appears to be /databricks/driver. Making Python look for the notebook path (e.g., subprocess.call([the/notebook/path]) fails, since it looks for the notebook path in this rootdir and obviously comes up empty. The notebooks must live somewhere else, but where? What path is %run invoking to find the notebook?

For reference, I am trying to implement pytest within Databricks, and pytest.main() should allow me to run the tests within the notebook itself in lieu of the typical command line approach... if it could just find out where the notebook lives.

Answer 1

I was looking for this same thing and after some digging into dbutils, this is the solution I came up with:

import json
context = json.loads(dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson())
print(context['extraContext']['aclPathOfAclRoot']) # Base path
print(context['extraContext']['notebook_path']) # Notebook path