Why do no Python DLLs built with MSVC load with mod_wsgi?

Question

I recently updated from Python 2.5 to 2.7 (I tried 2.6 during my hassles) and while everything works fine from the command line or in the Django runserver, mod_wsgi cannot load any module that contains DLLs (pyd) built with MSVC.

For example, if I build my own versions of pycrypto or lxml then I will get the following error only from mod_wsgi:

ImportError at /
DLL load failed: The specified module could not be found.

Even the official PIL binaries will fail to import the _imaging C module in mod_wsgi but that may be another problem.

However, if I use a version of pycrypto built with MinGW from somewhere like http://www.voidspace.org.uk/python/modules.shtml#pycrypto then it will import fine even in mod_wsgi. I don't find this solution satisfactory though since the whole reason I updated Python was to avoid needing to hunt for prebuilt binaries and I can't build them myself because MinGW fails >50% of the time for me.

EDIT2: I noticed this in Python27/Lib/distutils/msvc9compiler.py on lines 680-705:

try:
    # Remove references to the Visual C runtime, so they will
    # fall through to the Visual C dependency of Python.exe.
    # This way, when installed for a restricted user (e.g.
    # runtimes are not in WinSxS folder, but in Python's own
    # folder), the runtimes do not need to be in every folder
    # with .pyd's.
    manifest_f = open(manifest_file)
    try:
        manifest_buf = manifest_f.read()
    finally:
        manifest_f.close()
    pattern = re.compile(
        r"""<assemblyIdentity.*?name=("|')Microsoft\."""\
        r"""VC\d{2}\.CRT("|').*?(/>|</assemblyIdentity>)""",
        re.DOTALL)
    manifest_buf = re.sub(pattern, "", manifest_buf)
    pattern = "<dependentAssembly>\s*</dependentAssembly>"
    manifest_buf = re.sub(pattern, "", manifest_buf)
    manifest_f = open(manifest_file, 'w')
    try:
        manifest_f.write(manifest_buf)
    finally:
        manifest_f.close()
except IOError:
    pass

This probably explains why everything works from the command line but not in mod_wsgi. Commenting all this out seems to fix the problem but doesn't feel like the proper fix. The question now is where to put msvcr90.dll so that Apache can use it? I notice that Apache's bin folder contains msvcr70.dll and msvcr80.dll but putting 90 in there doesn't work.

Answer 1

I've had a similar issue and eventually found a solution here: download/update your apache server with one from http://www.apachelounge.com/download/.

Answer 2

While I don't know anything about mod_wsgi, I dare to guess that the most probable reason is missing runtime dependencies. You may want to inspect your MSVC-build with Dependency Walker that ships with MSVC (e.g., in MSVC 2005, it's located at \Common7\Tools\Bin\Depends.Exe). It will show you which DLLs are required by a binary.

As another workaround, it should be possible to build your modules with statically linked runtime (see Project Properties -> C/C++ -> Code Generation -> Runtime -- choose "Multithreaded" (not "Multithreaded DLL"); or, if building from command line, make sure /MT is used instead of /MD). However there may be problems if runtime-dependent things (e.g. FILE* objects) cross module boundary.

UPD If you have correct VC redist installed, the reason may be a problem with SxS configuration (i.e. manifest of .pyd itself is wrong or missing, or conflicts with manifest of the application that loads the .pyd). You can use sxstrace utility to see what exactly is going on. See Diagnosing SideBySide failures.

Also, did you try static linking of the runtime? Or, better yet, check what are requirements of your host process.