Setting priority/niceness within a jupyter/ipython notebook

Question

Is there any way to set the CPU priority of a jupyter notebook lower within the notebook? E.g. when running a days long parameter search on a compute server, setting priority within that notebook behind notebooks that others are working on in real-time.

Answer 1

Maybe setting the priority of your python process would be enough for you? As possible startet processes inherit the priority. If yes, the following code my help:

import psutil
psutil.Process().nice(19)# if on *ux
psutil.Process().nice(psutil.IDLE_PRIORITY_CLASS)# if on win

Answer 2

I guess you can renice the whole server (not aware of any way to renice a particular notebook). If the other notebooks are running on distinct notebook server process, you can do it from a subshell:

---------------------------------------------------------------
  !for pid in `pgrep -f jupyter`; do { renice -20 $pid; }; done
---------------------------------------------------------------

9721 (process ID) old priority 19, new priority -20
12449 (process ID) old priority 19, new priority -20
25502 (process ID) old priority 19, new priority -20

Or, if you want to do it in Python, first get the list of running servers:

>>> from notebook import notebookapp
>>> servers = list(notebookapp.list_running_servers())
>>> servers
[{'url': 'http://localhost:8888/', 
  'base_url': '/', 
  'token': '5ea29b3...7e1fba5331ae', 
  'secure': False, 
  'pid': 9721, 
  'hostname': 'localhost', 
  'password': False, 
  'port': 8888, 
  'notebook_dir': '/home/paulos/work'
}]

Filter the list for the ones you want if needed.

pids = [_['pid'] for _ in servers if meets_condition(_)]

Then call setpriority:

>>> from ctypes import cdll
>>> libc = cdll.LoadLibrary("libc.so.6")
>>> pids = [_['pid'] for _ in servers]
>>> for pid in pids:
        print("old priority for PID", pid, "is", libc.getpriority(0, pid))
        libc.setpriority(0, pid, 20)
        print("new priority for PID", pid, "is", libc.getpriority(0, pid))

old priority for PID 9721 is 0
new priority for PID 9721 is 19