Menu
Could any one explain me how to get to know the calling convention of a dll without getting and processing method names? Lets say our application is loading a third party dll and in order to handle it, is there any effective ways to get to know the calling convention of a dll? (__stdcall, __cdecl, __fastcall)
989
This calling convention is used to call methods on classes exported from an unmanaged DLL. Winapi. 1. This member is not actually a calling convention, but instead uses the default platform calling convention. For example, on Windows x86 the default is StdCall and on Linux x86 it is Cdecl.
The calling convention for the gcc compiler on Linux systems should be to pass arguments via the stack. Here I assume the arguments are passed using eax and ebx . I don't think that is right. ret probably expects to pick up a return address from somewhere.
__cdecl is the default calling convention for C and C++ programs. Because the stack is cleaned up by the caller, it can do vararg functions.
The stdcall calling convention is a variation on the Pascal calling convention in which the callee is responsible for cleaning up the stack, but the parameters are pushed onto the stack in right-to-left order, as in the _cdecl calling convention. Registers EAX, ECX, and EDX are designated for use within the function.
If the symbol begins with a _ but has no @, then it's __cdecl. If it begins with _ and has a @ it's __stdcall. If it begins with @ and has another @, it's __fastcall.
source
145
While trying to figure out why I was getting unresolved symbols when linking against a third-party dll I stumbled upon a (sort of) programmatic solution.
I wrote a small program against the Windows API using UnDecorateSymbolName from Dbghelp.hto decode the mangling scheme:
#include "Windows.h" #include "Dbghelp.h" #include "tchar.h" int _tmain(int argc, _TCHAR* argv[]) { CHAR out[512]; UnDecorateSymbolName( // Mangled symbol "?OFFReader@IO@OpenMesh@@YGAAV_OFFReader_@12@XZ", out, // Length of symbol 46, UNDNAME_32_BIT_DECODE); } There are definitely prettier ways to do it. I just run it in a debugger and look at the contents of out.
Also worth noting, contrary to Ignacio's answer, the difference between the mangled names for the cdecl methods in the dll and the stdcall methods being looked for was YAAAV vs. YGAAV.
20
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
