Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to run a Jupyter notebook with Python code automatically on a daily basis?

I have some Python code in a Jupyter notebook and I need to run it automatically every day, so I would like to know if there is a way to set this up. I really appreciate any advice on this.

like image 274
Betsy Curbelo Avatar asked Feb 12 '18 15:02

Betsy Curbelo


People also ask

How do you run a Jupyter Notebook everyday?

How do I schedule a Jupyter Notebook to run? When your Jupyter notebook is ready for scheduling, open the Schedule runs option inside the Computation tab or access it from the Run menu in Datalore. Then choose the run interval (hourly, daily, weekly, monthly) and the time zone.


2 Answers

Update
recently I came across papermill which is for executing and parameterizing notebooks.

https://github.com/nteract/papermill

papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1 

This seems better than nbconvert, because you can use parameters. You still have to trigger this command with a scheduler. Below is an example with cron on Ubuntu.


Old Answer

nbconvert --execute 

can execute a jupyter notebook, this embedded into a cronjob will do what you want.

Example setup on Ubuntu:

Create yourscript.sh with the following content:

/opt/anaconda/envs/yourenv/bin/jupyter nbconvert \                       --execute \                       --to notebook /path/to/yournotebook.ipynb \                       --output /path/to/yournotebook-output.ipynb 

You have more options except --to notebook. I like this option since you have a fully executable "log"-File afterwards.

I recommend using a virtual environment to run your notebook, to avoid that future updates mess with your script. Do not forget to install nbconvert into the environment.

Now create a cronjob, that runs every day e.g. at 5:10 AM, by typing crontab -e in your terminal and add this line:

10 5 * * * /path/to/yourscript.sh 
like image 157
Rick Avatar answered Sep 24 '22 11:09

Rick


Try the SeekWell Chrome Extension. It lets you schedule notebooks to run weekly, daily, hourly or every 5 minutes, right from Jupyter Notebooks. You can also send DataFrames directly to Sheets or Slack if you like.

Here's a demo video, and there is more info in the Chrome Web Store link above as well.

**Disclosure: I'm a SeekWell co-founder

like image 23
Thabo Avatar answered Sep 22 '22 11:09

Thabo