API Hooking without Detours

Question

Intro Info: Windows 7 64-bit. C++. 64-bit Apps and DLL's. Hooking without MS Detours.

Question: I've struggled on the issue of getting a working example that demonstrates hooking in Windows. Most of the tuts out there seem to have been written during a time where 32-bit Windows XP was the only operating system... I've since overcome the 64-bit hurdles of understanding and injected a DLL successfully. My next step in this journey of knowledge is hooking. In keeping with the nostalgia of the topic, MS's Detours does not support 64-bit (for free) and I'm certainly not paying $10,000 for anything. So I pursued the conventional methods in this tutorial.

This tut is awesome, but I'm having a little trouble understanding this segment:

void BeginRedirect(LPVOID newFunction)
{
    BYTE tempJMP[SIZE] = {0xE9, 0x90, 0x90, 0x90, 0x90, 0xC3};
    memcpy(JMP, tempJMP, SIZE);
    DWORD JMPSize = ((DWORD)newFunction - (DWORD)pOrigMBAddress - 5);
    VirtualProtect((LPVOID)pOrigMBAddress, SIZE, 
                PAGE_EXECUTE_READWRITE, &oldProtect);
    memcpy(oldBytes, pOrigMBAddress, SIZE);
    memcpy(&JMP[1], &JMPSize, 4);
    memcpy(pOrigMBAddress, JMP, SIZE);
    VirtualProtect((LPVOID)pOrigMBAddress, SIZE, oldProtect, NULL);
}

Particularly, I'm struggling with the tempJMP byte and all of the memcpy going on. I have an address for the InsertDate() function of Notepad that I want to hijack, but I'm not sure where to aim it... Would this be the address of the new function? Or is it not relative? Idk, I'm just looking for some pointers.

Answer 1

Hotpatchable functions start with the following instruction mov edi,edi and are preceded by 5 NOP instructions (code cave if I remember correctly).

When hotpatching, the mov edi,edi is overwritten with a short jump to the code cave. The code cave is also re-written with a jump to your hook handler (the function where you intercept the API call then forward it to the real API function).

Answer 2

The whole idea is to "overwrite" the original code that executes Messagebox to:

JuMP <CustomMessageBoxFunction>
RETurn (back to program execution) 

So ,

First he copies his shellcode to JMP array:

 memcpy(JMP, tempJMP, SIZE);

Then he copies the original assembly code bytes from the original address to his temporary storage "oldBytes" so that he can copy it back after his custom function is executed:

memcpy(oldBytes, pOrigMBAddress, SIZE);

Then he copies the address size he previously calculated to JMP array right after the jmp command :

memcpy(&JMP[1], &JMPSize, 4);

Finally his JMP[] array contains the shellcode required to call his function, e.g.

JMP 1234
RET

so now he has to copy that over the original bytes where the program expects to find original MessageBox function :

memcpy(pOrigMBAddress, JMP, SIZE);

Now coming to your question, if you want to hook InsertDate() then instead of using pOrigMBAddress you can use the address of InsertDate.

But I am not sure this will work with 64bit windows.