How to manage and communicate with multiple IPython/Jupyter kernels from a Python script?

Question

I want to mimic the functionality of a notebook server, and instead coordinate the creation/management of different IPython/Jupyter kernels from a central body of logic (i.e. my own Python script).

For example, I want to:

• Define an abstract command e.g. "add(x, y)"
• Communicate the abstract command to multiple kernels e.g. an IPython kernel and Scala kernel
• Have each kernel execute the command however they wish
• Return the result from each kernel to the central body of logic

Can anyone point me in the direction of how to programmatically start/stop/communicate with multiple IPython/Jupyter kernels?

Answer 1

A KernelManager deals with starting and stopping a single kernel, and there's a MultiKernelManager to co-ordinate more than one.

• http://ipython.org/ipython-doc/3/api/generated/IPython.kernel.manager.html
• http://ipython.org/ipython-doc/3/api/generated/IPython.kernel.multikernelmanager.html

Then you can use the .client() method to get a KernelClient instance which handles communications with a kernel:

• http://ipython.org/ipython-doc/3/api/generated/IPython.kernel.client.html

For details of how you communicate with a kernel, see the message spec docs. Some of this is abstracted away by KernelClient, but you'll probably need to know some of it.