Unloading an Injected DLL

Question

I have a DLL I inject into other processes using SetWindowsHookEx. Inside the DLL I increment the module's reference counter by calling GetModuleHandleEx so I can control when the module is unloaded.

At this point the module reference count "should be" 2 from both of those API calls. When the calling process shuts down, it calls UnhookWindowsHookEx, decrementing the reference count to 1. The DLL has a thread that waits on a few things, one of them being the handle of the process that called SetWindowsHookEx. When the process goes away the DLL does some cleanup, terminates all threads, cleans up memory and handles and then calls FreeLibraryAndExitThread. This decrements the counter and the DLL gets unloaded.

Here's my problem. There are a few processes, especially those without a UI, where the DLL never gets unloaded. I'm pretty confident I have cleaned up everything. And I know for a fact that none of my threads are running.

First of all, if you have any troubleshooting tips to help uncover the cause, that would be helpful. Otherwise, I was thinking about using some API like NtQueryInformationProcess to get the module address and confirm the module handle count is in fact zero, then call CreateRemoteThread to inject a call to LdrUnloadDll to unload the module address from within the process. What are your thoughts to this approach? Does anyone have any example code? I'm having difficulty finding out how to get the module handle count.

Answer 1

Okay.. here goes.. There are many ways to get the module info from a process. The undocumented way and the "documented" way.

Results (documented):

Here is the "documented" way..

#include <windows.h>
#include <TlHelp32.h>
#include <iostream>
#include <sstream>


int strcompare(const char* One, const char* Two, bool CaseSensitive)
{
    #if defined _WIN32 || defined _WIN64
    return CaseSensitive ? strcmp(One, Two) : _stricmp(One, Two);
    #else
    return CaseSensitive ? strcmp(One, Two) : strcasecmp(One, Two);
    #endif
}

PROCESSENTRY32 GetProcessInfo(const char* ProcessName)
{
    void* hSnap = nullptr;
    PROCESSENTRY32 Proc32 = {0};

    if ((hSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0)) == INVALID_HANDLE_VALUE)
        return Proc32;

    Proc32.dwSize = sizeof(PROCESSENTRY32);
    while (Process32Next(hSnap, &Proc32))
    {
        if (!strcompare(ProcessName, Proc32.szExeFile, false))
        {
            CloseHandle(hSnap);
            return Proc32;
        }
    }
    CloseHandle(hSnap);
    Proc32 = { 0 };
    return Proc32;
}

MODULEENTRY32 GetModuleInfo(std::uint32_t ProcessID, const char* ModuleName)
{
    void* hSnap = nullptr;
    MODULEENTRY32 Mod32 = {0};

    if ((hSnap = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE, ProcessID)) == INVALID_HANDLE_VALUE)
        return Mod32;

    Mod32.dwSize = sizeof(MODULEENTRY32);
    while (Module32Next(hSnap, &Mod32))
    {
        if (!strcompare(ModuleName, Mod32.szModule, false))
        {
            CloseHandle(hSnap);
            return Mod32;
        }
    }

    CloseHandle(hSnap);
    Mod32 = {0};
    return Mod32;
}

std::string ModuleInfoToString(MODULEENTRY32 Mod32)
{
    auto to_hex_string = [](std::size_t val, std::ios_base &(*f)(std::ios_base&)) -> std::string
    {
        std::stringstream oss;
        oss << std::hex << std::uppercase << val;
        return oss.str();
    };

    std::string str;
    str.append("  =======================================================\r\n");
    str.append("  Module Name:             ").append(Mod32.szModule).append("\r\n");
    str.append("  =======================================================\r\n\r\n");
    str.append("  Module Path:             ").append(Mod32.szExePath).append("\r\n");
    str.append("  Process ID:              ").append(std::to_string(Mod32.th32ProcessID).c_str()).append("\r\n");
    str.append("  Load Count (Global):     ").append(std::to_string(static_cast<int>(Mod32.GlblcntUsage != 0xFFFF ? Mod32.GlblcntUsage : -1)).c_str()).append("\r\n");
    str.append("  Load Count (Process):    ").append(std::to_string(static_cast<int>(Mod32.ProccntUsage != 0xFFFF ? Mod32.ProccntUsage : -1)).c_str()).append("\r\n");
    str.append("  Base Address:            0x").append(to_hex_string(reinterpret_cast<std::size_t>(Mod32.modBaseAddr), std::hex).c_str()).append("\r\n");
    str.append("  Base Size:               0x").append(to_hex_string(Mod32.modBaseSize, std::hex).c_str()).append("\r\n\r\n");
    str.append("  =======================================================\r\n");
    return str;
}

int main()
{
    PROCESSENTRY32 ProcessInfo = GetProcessInfo("notepad.exe");
    MODULEENTRY32 ME = GetModuleInfo(ProcessInfo.th32ProcessID, "uxtheme.dll");
    std::cout<<ModuleInfoToString(ME);
}

The problem with the undocumented API is that I've never figured out why the load counts are always "6" for dynamic modules and "-1" for static modules.. For this reason, I will not post it..

It is BEST NOT to use undocumented API if you want just the load count. The undocumented API's only advantage is that you can use it to "un-link/hide" a module within a process (like viruses do).. It will "unlink/hide" it.. NOT "unload" it. This means that at any time, you can "re-link" it back into the process's module list.

Since you only need the module-reference-count, I've only included "documented" API which does exactly that.