Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to manage and communicate with multiple IPython/Jupyter kernels from a Python script?

I want to mimic the functionality of a notebook server, and instead coordinate the creation/management of different IPython/Jupyter kernels from a central body of logic (i.e. my own Python script).

For example, I want to:

  • Define an abstract command e.g. "add(x, y)"
  • Communicate the abstract command to multiple kernels e.g. an IPython kernel and Scala kernel
  • Have each kernel execute the command however they wish
  • Return the result from each kernel to the central body of logic

Can anyone point me in the direction of how to programmatically start/stop/communicate with multiple IPython/Jupyter kernels?

like image 980
trianta2 Avatar asked Apr 29 '15 16:04

trianta2


People also ask

Can you use multiple kernels within the same Jupyter Notebook?

SoS Notebook is an extension to Jupyter Notebook that allows the use of multiple kernels in one notebook. More importantly, it allows the exchange of data among subkernels so that you can, for example, preprocess data using Bash, analyze the processed data in Python, and plot the results in R.

What is the difference between IPython and Jupyter?

IPython continued to exist as a Python shell and kernel for Jupyter, but the notebook interface and other language-agnostic parts of IPython were moved under the Jupyter name. Jupyter is language agnostic and its name is a reference to core programming languages supported by Jupyter, which are Julia, Python, and R.

Is IPython tied to Jupyter?

In addition, IPython is closely tied with the Jupyter project, which provides a browser-based notebook that is useful for development, collaboration, sharing, and even publication of data science results.


1 Answers

A KernelManager deals with starting and stopping a single kernel, and there's a MultiKernelManager to co-ordinate more than one.

  • http://ipython.org/ipython-doc/3/api/generated/IPython.kernel.manager.html
  • http://ipython.org/ipython-doc/3/api/generated/IPython.kernel.multikernelmanager.html

Then you can use the .client() method to get a KernelClient instance which handles communications with a kernel:

  • http://ipython.org/ipython-doc/3/api/generated/IPython.kernel.client.html

For details of how you communicate with a kernel, see the message spec docs. Some of this is abstracted away by KernelClient, but you'll probably need to know some of it.

like image 82
Thomas K Avatar answered Oct 23 '22 09:10

Thomas K