Simple way to choose which cells to run in ipython notebook during run all

Question

I have an ipython notebook that runs several steps in a data processing routine and saves information in files along the way. This way, while developing my code (mostly in a separate .py module), I can skip to and run various steps. I'd like to set it up so that I can Cell->run all but only have it execute certain chosen steps that would be easily chosen. e.g., I'd envision defining the steps I want to run in a dict like so:

process = {
    'load files':False,
    'generate interactions list':False,
    'random walk':True,
    'dereference walk':True,
    'reduce walk':True,
    'generate output':True
}

then the steps would run based on this dict. BTW, each step comprises multiple cells.

I think %macro is not quite what I want since anytime I changed anything or restarted the kernel I'd have to redefine the macro, with changing cell numbers.

Is there like a %skip or %skipto magic or something along those lines? Or perhaps a clean way to put at the beginning of cells, if process[<current step>]: %dont_run_rest_of_cell?

Answer 1

You can create your own skip magic with the help of a custom kernel extension.

skip_kernel_extension.py

def skip(line, cell=None):
    '''Skips execution of the current line/cell if line evaluates to True.'''
    if eval(line):
        return

    get_ipython().ex(cell)

def load_ipython_extension(shell):
    '''Registers the skip magic when the extension loads.'''
    shell.register_magic_function(skip, 'line_cell')

def unload_ipython_extension(shell):
    '''Unregisters the skip magic when the extension unloads.'''
    del shell.magics_manager.magics['cell']['skip']

Load the extension in your notebook:

%load_ext skip_kernel_extension

Run the skip magic command in the cells you want to skip:

%%skip True  #skips cell
%%skip False #won't skip

You can use a variable to decide if a cell should be skipped by using $:

should_skip = True
%%skip $should_skip

Answer 2

Adding to what Robbe said above (I can't comment because I'm new), you could just do the following in your first cell if you don't want to create a custom extension that you might just forget about:

def skip(line, cell=None):
    '''Skips execution of the current line/cell if line evaluates to True.'''
    if eval(line):
        return

    get_ipython().ex(cell)

def load_ipython_extension(shell):
    '''Registers the skip magic when the extension loads.'''
    shell.register_magic_function(skip, 'line_cell')

def unload_ipython_extension(shell):
    '''Unregisters the skip magic when the extension unloads.'''
    del shell.magics_manager.magics['cell']['skip']
    
    
load_ipython_extension(get_ipython())