Change python interpretor mid-script

Question

I have 3 .py files for a project I am running on windows machine. One of these files calls a library which is 32 bit only. The other 2 .py files have libraries that are both 32 and 64 bit compatible. They look like this:

fileA.py
        -- calls fileB.py (library is only 32 bit compatible)
        -- also calls function in fileC.py (libraries are both 32 and 64 bit compatible)

Now, I am running into a memory error in fileC.py which can be taken care of if I could use the 64 bit version of python. However, if I do that, I run into error while executing fileB.py since it has a 32 bit library which does not talk well with 64 bit python (The 32 bit library is ArcPy). The library with memory error is pandas, the memory error is occurring since pandas cannot read dataframes bigger than 2GB for a 32 bit system.

Is there any way I can use 32 bit interpreter for fileB.py and 64 bit for fileC.py. Keeping in mind that both are called from fileA.py.

Answer 1

There is no way to mix 32 and 64 bit code in the same process, to the best of my knowledge.

You need to write an executable wrapper around ArcPy (fileB) and execute it as a separate process using subprocess using the 32 bit interpreter. The executable ArcPy wrapper will need to return the result of its processing in a serialized format (like JSON).

Another alternative is multiprocessing. You can use set_executable() function to use the 32-bit interpreter. This method provides nicer IPC mechanism to communicate with the child process.

In both cases, you'll end up forking a child process. If ArcPy needs to be called into multiple times, you might want to write the wrapper to be long running child process and use multiprocessing.

That said, do yourself a favor and get the 64 bit version of ArcPy installed - if it is feasible. Even if you can't remove the 32 bit version, install the 64 bit one in an alternate location.