How to prevent Git from commiting Jupyter Notebook results?

Question

I m working on project in Jupyter Notebook.

Whenever I make a commit not only changed code and markdown columns get commited but also results from code columns.

That makes Git diffs unreadable and it is very hard to review pull requests and changes due to commiting of those code cell results.

Is there a way of preventing this?

Answer 1

I strongly recommend putting the following little script in .git/hooks/pre-commit. It uses nbconvert on all .ipynb files that are staged to be committed, and if after stripping all the output there's no changes to be committed it exits. That last part is important because otherwise you'll be making useless empty commits. Since it only runs on notebooks you have committed it won't remove all the output from other notebooks you're still working on.

#!/bin/bash
for f in $(git diff --name-only --cached); do
    if [[ $f == *.ipynb ]]; then
        jupyter nbconvert --clear-output --inplace $f
        git add $f
    fi
done

if git diff --name-only --cached --exit-code
then
    echo "No changes detected after removing notebook output"
    exit 1
fi

That script plus the appropriate .gitignore entries should ensure that your Git history is kept clear from unwanted Jupyter output.

---

Here's a Husky compatible variant, just save it in .husky/pre-commit.

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

for f in $(git diff --name-only --cached); do
    case "$f" in
        *".ipynb") jupyter nbconvert --clear-output --inplace $f && git add $f ;;
    esac
done

if git diff --name-only --cached --exit-code
then
    echo "No changes detected after removing notebook output"
    exit 1
fi

Answer 2

You have a few options:

Jupytext (https://github.com/mwouts/jupytext), will let you open .py files as Jupyter notebooks, and since they do not store the input, the diff will be as easy as any other source code diff.

If you want to keep the .ipynb format, you can use nbdime (https://github.com/jupyter/nbdime) which produces nicer notebook diffs (you can integrate it with git diff).