How to Convert a Jupyter Notebook to an Academic Paper Format?

Question

I have a Jupyter notebook containing all of the content I would like to put in an academic paper (text, equations, and plots), together with code which I would like hidden.

Does there exist a way to programmatically convert the notebook into a form acceptable for journal submission? Ideally I would like something like knitr's functionality for creating academic-style papers, but for Jupyter rather than R-focused tools.

I've found pandoc templates for several journals (including the one I'd like to submit to), but it's not clear how well this integrates with Jupyter.

I also found this description of using Jupyter notebooks in the paper-writing process, but the author appears to simply use it for data analysis and generating plots; it might as well be a standard script.

If this doesn't exist yet, what would be necessary to make a reusable tool for this task? Would an nbconvert custom exporter be sufficient, or would I need a full Jupyter extension in order to specify bibliographies and plot/table locations from within the notebook interface?

Answer 1

Convert your notebook to markdown then use pandoc with filters (pandoc crossref and pandoc citeproc) to convert to Word.

You need title, sections, cross-referencing, citations, numbering, figures and tables with captionings… Here is a workflow that works: You can first convert your notebook into markdown which can easily be done using nbconvert:

jupyter nbconvert --to markdown --no-prompt your_notebook.ipynb

--no-prompt: will help get rid of code cell if you don’t want in the manuscript. You can also use hidden cells among markdown cells that you don't want in the final output. Then you need pandoc and two filters pandoc-crossref, and pandoc-citeproc. I'm using windows and anaconda 3 to install them:

https://anaconda.org/conda-forge/pandoc

https://anaconda.org/conda-forge/pandoc-crossref

Fortunately, installing pandoc through anaconda will also bring pandoc-citeproc. You need a pandoc-crossref compatible with the version of pandoc (currently pandoc 2.5 ). So, when installing from conda, you can use:

conda install -c conda-forge pandoc=2.5

then you can simply run:

conda install -c conda-forge pandoc-crossref

The key thing in all will be the markdown syntax. You have to write your notebook using the markdown syntax which is the default anyway in Jupiter notebooks. Interesting examples are here, here and here also :) . You will also need a reference file in .bib format for managing your citations. Then by simply running the command:

pandoc -s your_notebook.md -o your_notebook.pdf -F pandoc-crossref -F pandoc-citeproc --bibliography=your_references.bib -f markdown 

you can turn your entire notebook into a nice, clean manuscript 😊

pandoc has many other options for customizations: colorlinks, papersize, geometry, number-sections… (see pandoc –help)

So writing a custom Jupiter exporter you can combine the two commands and export your notebook as a manuscript.

Another interesting approach here https://github.com/chrisjsewell/ipypublish