how to make pip respect the "CC" environment variable

Question

I'm working in a very convoluted environment where different machines have access to different distributed file systems.

• Machine A has access to filesystem X and is used to install software on filesystem Y.
• Machine B has access to filesystem Y, but not X.

I'm working on machine B, and I find myself using python a lot. Sometimes, I need packages that are not preinstalled, so I use pip install PKGXYZ --user to install them locally. This generally works nicely, but there's a catch.

The python packages distutils and its monkey-patched derivative setuptools which are used by pip use the distutils.sysconfig functionality to get hold of compiler versions, paths, and somesuch. For this, they use the internal Makefile that was used to install python. While this is generally a good strategy, it fails with my concrete setup, because the paths in pythons internal Makefile point to filesystem X, which I do not have access to on my machine B. Hence, I find myself using the --no-clean option of pip and hacking the setup.py of the packages I want to install with snippets like this:

import re
import sys
import os
cc = os.getenv("CC")
if not cc:
    print("please set CC environment variable!")
    exit(0)
from distutils.sysconfig import get_config_vars
for k,v in get_config_vars().iteritems():
    try:
        if "fsX" in v:
            newv = re.sub(r'/fsX/[^ ]*/g[c+][c+]',cc,v)
            get_config_vars()[k] = newv
    except TypeError:
        pass

such that I can use the CC environment variable to overwrite the default settings of the compiler path from pythons Makefile with something that works on my machine.

However, this is an ugly hack. There must surely be a nicer way to do this and make pip use some different compiler via some environment variable, config file or command line option. Or is there?

Answer 1

It sounds like you have compiler tools on system B, so one option would be to re-build python on your system using local tools and use that instead.

If you're only doing this for your user on you system, then you can even install it in your user's home directory to keep it completely out the way, and then set up your environment to use that. Or use virtualenv.

You can get and build a new Python install easily enough. For example for python 3.5.1 on Linux:

cd
mkdir src
mkdir -p local/python351
cd src
wget https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tgz
tar -xf Python*.tgz
cd Python-3.5.1
./configure --prefix ~/local/python351
make
make install