Python web hosting: Numpy, Matplotlib, Scientific Computing

Question

I write scientific software in Numpy/Scipy/Matplotlib. Having developed applications on my home computer, I am now interested in writing simple web applications. Example: user uploads image or audio file, my program processes it using Numpy/Scipy, and output is displayed on the browser using Matplotlib, or perhaps the user can download a processed file.

I already pay for hosting that does have Python 2.4.3 installed, but no Numpy/Scipy. I don't have shell access via command line, either. Just drag-and-drop FTP. Pretty limited, but I can get simple Python/CGI scripts working.

Surprisingly, a web search revealed few suitable options for web hosting with these capabilities already built in. (Please guide me if I am wrong.) I am learning about the Google App Engine, but I still don't have a full understanding about its tools and limitations. What the web did tell me is that others have similar concerns.

Hoping for solutions, I thought I would ask these simple questions to the awesome SO community:

1. Is there a simple way of installing numpy (or any third-party package/library) onto my already hosted space? I know the Python path on my hosted space, and I know the relevant Python/Numpy directories on my home computer. Can I simply copy files over and have it work? Both local and remote systems run Ubuntu.

2. What hosting sites exist (either free or paid) which have Numpy/Matplotlib installed or, if not installed, the possibility of installing it? Are there any documented sites that you can reference with working applications, no matter how simple?

3. Can Google App Engine help me in any way? Or is it totally for something else? Have you or others used it to write scientific applications in Python/Numpy? If so, could you reference them?

Thank you for your help.

EDIT: After the useful answers below, I bought the $20 plan at Slicehost, and I love it so far! (I first tried Amazon EC2. I must be stupid, but I just couldn't get it to work.) Setting up the Ubuntu server with Apache took mere hours (and I'm an Apache novice). It allows me to do exactly what I wanted with Python plus much more. I now have my own remote repository for version control, too. Thanks again!

EDIT 2: Nearly two years later, I tried Linode and EC2 (again). Linode is great. EC2 seemed easier this time around -- maybe it's just added experience, or maybe it's the improvements that Amazon made to the AWS management console. For those interested in Numpy/Scipy/Matplotlib/Audiolab, here is my Ubuntu cheat sheet whenever I launch an EC2 instance:

ec2:~$ sudo aptitude install build-essential python-scipy ipython         python-matplotlib python-dev python-setuptools libsndfile-dev         libasound2-dev mysql-server python-mysqldb   Upload scikits.audiolab-0.11.0  ec2:~/scikits.audiolab-0.11.0$ sudo python setup.py install  ec2:~$ sudo rm -rf scikits.audiolab-0.11.0  ec2:~$ nano .ipython/ipy_user_conf.py  ip.ex('import matplotlib; matplotlib.use("Agg"); import scipy, pylab,         scipy.signal as sig, scipy.linalg as lin, scipy.sparse as spar,         os, sys, MySQLdb, boto; from scikits import audiolab')  import ipy_greedycompleter  import ipy_autoreload 

Answer 1

1: Installing third party packages to hosted spaces

You can indeed install third party packages to your hosted space. If it's a pure python package, all that's needed is to unpack it to a directory and then add that directory to your PYTHONPATH environment variable or sys.path.

This can be tiring to do often, and won't work easily for compiled modules. If you have shell access to your python host, the excellent virtualenv package allows you to do set up a private python environment with its own libraries.

To set up your virtualenv, you'll do something like this at the shell:

$ virtualenv $HOME/my_python $ $HOME/my_python/bin/easy_install numpy 

You can keep running easy_install for anything else you want to install in your personal python environment.

Now, when you write your python scripts, you will want to use your private python interpreter, if that is possible:

#!/home/myuser/my_python/bin/python  import numpy  # script here 

If your python env cannot be specified (such as if run by mod_wsgi), you will need to add it to the import path:

import sys sys.path.insert(0, '/home/myuser/my_python/lib/python2.5/site-packages')  import numpy 

2: Hosting sites with numpy

I can't think of any hosting sites offhand which offer numpy pre-installed. However, Dreamhost/Bluehost for sharedhosts provide SSH access, and with shell access you can install numpy using the methods I described above. Any Virtual Private Server such as Linode/Slicehost will allow you to install whatever you desire, as well.

3: AppEngine

As mentioned above, AppEngine will not allow you to install C extensions (but pure python ones do work) so it's unlikely numpy will work for you on there, since I suspect some of its features use C speedups.

Answer 2

App Engine does not support any of numpy, scipy, or matplotlib, alas.

If you know exactly what OS and CPU your host is using, you could make an identical installation for yourself, download and install the same version of Python they're using, download the sources of packages you require and build them into .so (or .pyd, depending on the platform) files, and upload those -- sounds like a real tour de force.

Any of the many, many sites that offer normal virtual hosting (a virtual machine, typically Linux, with modest HW resources, but root privileges for you, ssh shell access, and a gcc you can use in particular) will be much easier to work with -- essentially, you'll download and install the software you need just about the same way you'd do on your own Linux workstation!