Crontab / Launchd: OS X User Permissions, Environment Variables, Python Virtualenvs

Question

I am trying to automate running a python script every minute on my mac, from a virtual environment. I am convinced that I don't properly understand permissions, paths, and environment variables in some crucial way that is preventing me from figuring this out.

I am an admin user with root rights enabled. I run HomeBrew, PIP and Virtualenv to manage python packages and virtual environments for different projects.

I would like to do the following every 60 seconds:

$ source /.virtualenvs/myenvironment/bin/activate
$ cd ~/desktop/python/
$ python myscript.py
$ deactivate  

I have tried:

(a) writing my own plist for Launchd - and I believe these documents were well formed.

(b) programs which manage Launchd daemons and agents for you (both Launch Control and Lingon).

(c) I have tried simply editing the crontab (only lets me if I use the sudo command).

The python script, which works on command, pulls data from an online source and stores it in a sqlite table. I can tell the cron isn't running because the sqlite db isn't being touched.

Any thoughts would be enormously appreciated.

Answer 1

You don't say exactly what you tried with launchd and cron, but I'd bet you weren't using either of them correctly. Both are oriented toward running single, isolated commands (/programs), not sequences of shell commands. While it's possible to do this with a single cron job or launchd item, it's going to be messy. The simplest thing would be to write a shell script that does the sequence you want (be sure to include a shebang at the beginning, and enable execute permission on the script with chmod +x /path/to/script), and run that from either cron or launchd:

#!/bin/bash
source /.virtualenvs/myenvironment/bin/activate
cd ~/desktop/python/
python myscript.py
deactivate

I would not recommend using Automator to wrap the command sequence; it's designed for GUI-based scripting, and may not work right in a background-only job.

Answer 2

I have had this exact same problem and have recently solved it. Look here for the steps I took. Basically it deals with the shell needing the PYTHONPATH, not just the PATH.