Logging while nbconvert execute

Question

I have a Jupyter notebook that needs to run from the command line. For this I have the following command:

jupyter nbconvert --execute my_jupyter_notebook.ipynb --to python

This command creates a python script and then executes it. However, I'm using the logging library in Python to log certain events. When it executes the script from the command above, nothing can be seen on the terminal.

However, when I execute manually the converted jupyter, like below, I can see all the logs on my terminal:

python3 my_jupyter_notebook.py

I've tried adding extra arguments like --debug and --stdout but those just output all the code, not just the logs. Is it possible to output on the terminal the results of logging while doing an nbconvert execute command?

Answer 1

Here is a code that catch warning and exception produced during execution of nbconvert and pass them to the logger. Jupyter and nbconvert use a different way of handling exceptions.

from logging import getLogger
import sys
import traceback
import warnings
import IPython
import logging

logger = getLogger(name)
logging.basicConfig(stream=sys.stdout, level=logging.WARNING)

# Catch Traceback 
def showtraceback(self):
    traceback_lines = traceback.format_exception(*sys.exc_info())
    del traceback_lines[1]
    message = ''.join(traceback_lines)
    logger.error(traceback_lines[-1] + str(message))
IPython.core.interactiveshell.InteractiveShell.showtraceback = showtraceback

# Catch Warning 
def warning_on_one_line(message, category, filename, lineno, file=None, line=None):
    logger.warning(str(message) + '\n' + str(filename) + ' : ' + str(lineno))
    return '%s:%s: %s:%s\n' % (filename, lineno, category.__name__, message)
warnings.formatwarning = warning_on_one_line