How to include components of psutil in py2exe that py2exe can't find?

Question

I am using py2exe on windows 7 to make an app using psutil. I am using python 2.7. I am making a single file executable. When I try to compile the python python program with psutil, near the end it says:

The following modules appear to be missing
['_psutil_bsd', '_psutil_linux', '_psutil_osx', '_psutil_posix', '_psutil_sunos', '_scproxy', '_sysconfigdata', 'builtins']

However, the program compiles fine. When I run the compiled program, it crashes and generates a log file:

Traceback (most recent call last): File "TaskManager.py", line 27, in File "zipextimporter.pyc", line 82, in load_module File "psutil__init__.pyc", line 135, in File "zipextimporter.pyc", line 82, in load_module File "psutil_psmswindows.pyc", line 14, in File "zipextimporter.pyc", line 98, in load_module ImportError: MemoryLoadLibrary failed loading _psutil_mswindows.pyd

I have tried listing in the options part of the setup:

"includes": ["_psutil_mswindows.pyd"]

as well as just

"includes": ["psutil"]

I have also tried including all the modules py2exe listed as missing. With the exception of including psutil, which didn't do anything, py2exe could not find these files.

There are a surprising number of similar questions on this topic online, but none of them have good answers (if any answers at all) and many of the errors are not quite the problem I have.

Update: strangely enough, I found _psutil_mswindows.pyd in build\bdist.win32\winexe\collect-2.7 of py2exe's build directory. I guess for some reason it isn't getting packaged properly.

Answer 1

All these issues come from py2exe including a bunch of system dlls.

Adding the following list of excludes. (Mainly the last 4). Solves this issue.

dll_excludes=['msvcr71.dll', "IPHLPAPI.DLL", "NSI.dll",  "WINNSI.DLL",  "WTSAPI32.dll"]

You can take a look/cross reference the .dll with those found in your system folder for a list of which ones it is included that should not be present.

Answer 2

I met this:

ImportError: MemoryLoadLibrary failed loading psutil\_psutil_windows.pyd

It is a little bit different from the question in filename.

After reading the source code of psutil, I figured out the root cause: The target Windows platform of psutil depends on which platform it is built on.

The following code exists in _psutil_windows.c, suggesting GetIfEntry2 is referenced. You can easily confirm this by running depends.exe to load _psutil_windows.pyd on Windows XP.

#if (_WIN32_WINNT >= 0x0600) // Windows Vista and above
        SecureZeroMemory((PVOID)pIfRow, sizeof(MIB_IF_ROW2));
        pIfRow->InterfaceIndex = pCurrAddresses->IfIndex;
        dwRetVal = GetIfEntry2(pIfRow);
#else // Windows XP
        pIfRow->dwIndex = pCurrAddresses->IfIndex;
        dwRetVal = GetIfEntry(pIfRow);
#endif

And setup.py generates macros according to the current system environment like this:

# Windows
if WINDOWS:
    def get_winver():
        maj, min = sys.getwindowsversion()[0:2]
        return '0x0%s' % ((maj * 100) + min)

    macros.extend([
        # be nice to mingw, see:
        # http://www.mingw.org/wiki/Use_more_recent_defined_functions
        ('_WIN32_WINNT', get_winver()),
        ('_AVAIL_WINVER_', get_winver()),
        ('_CRT_SECURE_NO_WARNINGS', None),
        # see: https://github.com/giampaolo/psutil/issues/348
        ('PSAPI_VERSION', 1),
    ])

So the options to solve this problem are:

1. Build psutil on a Windows XP Machine, or
2. Modify setup.py to use the target Windows platform version as we want.