Finding undocumented APIs in Windows

Question

I was curious as to how does one go about finding undocumented APIs in Windows.

I know the risks involved in using them but this question is focused towards finding them and not whether to use them or not.

Answer 1

Use a tool to dump the export table from a shared library (for example, a .dll such as kernel32.dll). You'll see the named entry points and/or the ordinal entry points. Generally for windows the named entry points are unmangled (extern "C"). You will most likely need to do some peeking at the assembly code and derive the parameters (types, number, order, calling convention, etc) from the stack frame (if there is one) and register usage. If there is no stack frame it is a bit more difficult, but still doable. See the following links for references:

1. http://www.sf.org.cn/symbian/Tools/symbian_18245.html
2. http://msdn.microsoft.com/en-us/library/31d242h4.aspx

Check out tools such as dumpbin for investigating export sections.

There are also sites and books out there that try to keep an updated list of undocumented windows APIs:

1. The Undocumented Functions
2. A Primer of the Windows Architecture
3. How To Find Undocumented Constants Used by Windows API Functions
4. Undocumented Windows
5. Windows API

Edit: These same principles work on a multitude of operating systems however, you will need to replace the tool you're using to dump the export table. For example, on Linux you could use nm to dump an object file and list its exports section (among other things). You could also use gdb to set breakpoints and step through the assembly code of an entry point to determine what the arguments should be.

Answer 2

IDA Pro is your best bet here, but please please double please don't actually use them for anything ever.

They're internal because they change; they can (and do) even change as a result of a Hotfix, so you're not even guaranteed your undocumented API will work for the specific OS version and Service Pack level you wrote it for. If you ship a product like that, you're living on borrowed time.