How do you setup Python with virtualenv locally and on a real Linode server (or similar)?

Question

Using PyCharm on Windows and would like to gain a better understanding of how to setup my local environment so that it translates cleanly as possible to my servers on Linode (or any other Linux box for that matter).

I have a physical drive set aside for development work. In my case this is drive Z:.

I will typically create one directory per project. A project being defined as an entire website.

I have currently also opted to have a directory, Z:\virtualenv, where I create my virtual environments. One per project. I suppose multiple projects could share the same virtualenv but I am not sure if this is smart for either development or production.

I've considered the idea of having the per-project virtualenv live inside it's corresponding project. This appeals to me because then each project would be monolithic. For example, if we are talking about Flask application under PyCharm:

d    z:\flask_app
d         .git
d         .idea
d         static
d         templates
d         virtualenv
          main.py

How, then, do you setup the production server given the above?

Let's assume one is using a single machine to host more than one site through virtual hosting, this being one of them:

<VirtualHost *:80>
  ServerAdmin [email protected]

  ServerName   example.com
  ServerAlias  example.com *.example.com
  DocumentRoot /var/www/example/public_html
  ErrorLog     /var/www/example/logs/access.log
  CustomLog    /var/www/example/logs/error.log combined

  <Directory   /var/www/example>
        Options Indexes FollowSymLinks
        AllowOverride All
        Order allow,deny
        Allow from all
  </Directory>
</VirtualHost>

Do I setup virtualenv at the global server level? I think that's a global "yes". It couldn't work any other way. I don't think.

OK, that does mean that the entire file structure under

z:\flask_app

can now be FTP'd into

/var/www/example/public_html

and the site is good to go?

I understand that db server, db's, tables, etc. need to be setup on the production machine to match. I am simply focusing on Python on the transition of Python with virtualenv from a desktop development environment to an external Linux production box.

I think I have to use virtualenv at the server root level to also enable that virtual environment, right? This is where I am a bit fuzzy about things. Most tutorials I've come across cover your local development environment extensively but rarely go into the transition of projects to production servers, their setup and ongoing relationship to the development setup.

I will be using a virtual machine with Ubuntu 14.04 LTS to sort this out as I move forward.

I've also considered using 14.04 Desktop for development on a VM in order to have matched environments and get off Windows.

Answer 1

1) A 14.04 Desktop VM just to muck around in and get things right before transferring to scripts and the command line for your server is a great idea.

2) You may happen to love the virtualenvwrapper tool/project. It maps almost exactly to your current workflow, but with some handy conveniences (its whole point). It essentially hosts a central folder of virtualenvs to different names (/ folders). Its most handy commands are mkproject (create a new folder and virtualenv of the same name) and workon (activate the project of that name).

3) Fortunately given 14.04 isn't too old, it has quite a recent virtualenv already present in its packages, python-virtualenv (1.11.4). I would install this and then use it to create environments on your server to run python projects under, as you suggest.

OK, that does mean that the entire file structure under ... can now be FTP'd into ... and the site is good to go?

No, because you'd be trying to transfer a virtualenv created for a Python on a Windows machine and hoping it would work under a Python on Linux/Ubuntu.

4) To keep a managed list of packages each project needs installed, list them in a requirements.txt. Then with a new virtualenv active, you can simply run pip install -r requirements.txt and all the needed packages will be installed for it.

5) For running your apps under the one server, I would suggest running a local WSGI server like Chaussette (perhaps under Circus) or uWSGI that hosts your python WSGI app under a local port / unix socket; then configure Apache or Nginx to reverse proxy all needed dynamic traffic to that server (see this SO answer as an example).

6) Some rudimentary bash scripting know-how can help a lot if you have things repeatably bootstrap-able :) If it gets even more complicated, you can use a managed configuration product like Salt.