Can Jupyter run a separate R notebook from within a Python notebook?

Question

I have a Jupyter notebook (python3) which is a batch job -- it runs three separate python3 notebooks using %run. I want to invoke a fourth Jupyter R-kernel notebook from my batch.

Is there a way to execute an external R notebook from a Python notebook in Jupyter / iPython?

Current setup:

run_all.ipynb: (python3 kernel)

%run '1_py3.ipynb'
%run '2_py3.ipynb'
%run '3_py3.ipynb'
%run '4_R.ipynb'

The three python3 notebooks run correctly. The R notebook runs correctly when opened separately in Jupyter -- however it fails when called using %run from run_all.ipynb. It is interpreted as python, and the cell gives a python error on the first line:

cacheDir <- "caches"

TypeError: bad operand type for unary -: 'str'

I am interested in any solution for running a separate R notebook from a python notebook -- Jupyter magic, shell, python library, et cetera. I would also be interested in a workaround -- e.g. a method (like a shell script) that would run all four notebooks (both python3 and R) even if this can't be done from inside a python3 notebook.

(NOTE: I already understand how to embed %%R in a cell. This is not what I am trying to do. I want to call a complete separate R notebook.)

Answer 1

I don't think you can use the %run magic command that way as it executes the file in the current kernel.

Nbconvert has an execution API that allows you to execute notebooks. So you could create a shell script that executes all your notebooks like so:

#!/bin/bash
jupyter nbconvert --to notebook --execute 1_py3.ipynb
jupyter nbconvert --to notebook --execute 2_py3.ipynb
jupyter nbconvert --to notebook --execute 3_py3.ipynb
jupyter nbconvert --to notebook --execute 4_R.ipynb

Since your notebooks require no shared state this should be fine. Alternatively, if you really wanna do it in a notebook, you use the execute Python API to call nbconvert from your notebook.

import nbformat
from nbconvert.preprocessors import ExecutePreprocessor

with open("1_py3.ipynb") as f1, open("2_py3.ipynb") as f2, open("3_py3.ipynb") as f3, open("4_R.ipynb") as f4:
    nb1 = nbformat.read(f1, as_version=4)
    nb2 = nbformat.read(f2, as_version=4)
    nb3 = nbformat.read(f3, as_version=4)
    nb4 = nbformat.read(f4, as_version=4)

ep_python = ExecutePreprocessor(timeout=600, kernel_name='python3')
#Use jupyter kernelspec list to find out what the kernel is called on your system
ep_R = ExecutePreprocessor(timeout=600, kernel_name='ir')

# path specifies which folder to execute the notebooks in, so set it to the one that you need so your file path references are correct
ep_python.preprocess(nb1, {'metadata': {'path': 'notebooks/'}})
ep_python.preprocess(nb2, {'metadata': {'path': 'notebooks/'}})
ep_python.preprocess(nb3, {'metadata': {'path': 'notebooks/'}})
ep_R.preprocess(nb4, {'metadata': {'path': 'notebooks/'}})

with open("1_py3.ipynb", "wt") as f1, open("2_py3.ipynb", "wt") as f2, open("3_py3.ipynb", "wt") as f3, open("4_R.ipynb", "wt") as f4:
    nbformat.write(nb1, f1)
    nbformat.write(nb2, f2)
    nbformat.write(nb3, f3)
    nbformat.write(nb4, f4)

Note that this is pretty much just the example copied from the nbconvert execute API docs: link