How to alloc a executable memory buffer?

Question

I would like to alloc a buffer that I can execute on Win32 but I have an exception in visual studio cuz the malloc function returns a non executable memory zone. I read that there a NX flag to disable... My goal is convert a bytecode to asm x86 on fly with keep in mind performance.

Does somemone can help me?

JS

Answer 1

You don't use malloc for that. Why would you anyway, in a C++ program? You also don't use new for executable memory, however. There's the Windows-specific VirtualAlloc function to reserve memory which you then mark as executable with the VirtualProtect function applying, for instance, the PAGE_EXECUTE_READ flag.

When you have done that, you can cast the pointer to the allocated memory to an appropriate function pointer type and just call the function. Don't forget to call VirtualFree when you are done.

Here is some very basic example code with no error handling or other sanity checks, just to show you how this can be accomplished in modern C++ (the program prints 5):

#include <windows.h>
#include <vector>
#include <iostream>
#include <cstring>

int main()
{
    std::vector<unsigned char> const code =
    {
        0xb8,                   // move the following value to EAX:
        0x05, 0x00, 0x00, 0x00, // 5
        0xc3                    // return what's currently in EAX
    };    

    SYSTEM_INFO system_info;
    GetSystemInfo(&system_info);
    auto const page_size = system_info.dwPageSize;

    // prepare the memory in which the machine code will be put (it's not executable yet):
    auto const buffer = VirtualAlloc(nullptr, page_size, MEM_COMMIT, PAGE_READWRITE);

    // copy the machine code into that memory:
    std::memcpy(buffer, code.data(), code.size());

    // mark the memory as executable:
    DWORD dummy;
    VirtualProtect(buffer, code.size(), PAGE_EXECUTE_READ, &dummy);

    // interpret the beginning of the (now) executable memory as the entry
    // point of a function taking no arguments and returning a 4-byte int:
    auto const function_ptr = reinterpret_cast<std::int32_t(*)()>(buffer);

    // call the function and store the result in a local std::int32_t object:
    auto const result = function_ptr();

    // free the executable memory:
    VirtualFree(buffer, 0, MEM_RELEASE);

    // use your std::int32_t:
    std::cout << result << "\n";
}

It's very unusual compared to normal C++ memory management, but not really rocket science. The hard part is to get the actual machine code right. Note that my example here is just very basic x64 code.

Answer 2

Extending the above answer, a good practice is:

• Allocate memory with VirtualAlloc and read-write-access.
• Fill that region with your code
• Change that region's protection with VirtualProtectto execute-read-access
• jump to/call the entry point in this region

So it could look like this:

adr = VirtualAlloc(NULL, size, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
// write code to the region
ok  = VirtualProtect(adr, size, PAGE_EXECUTE_READ, &oldProtection);
// execute the code in the region

Answer 3

As stated in documentation for VirtualAlloc

flProtect [in]

The memory protection for the region of pages to be allocated. If the pages are being committed, you can specify any one of the memory protection constants.

one of them is:

PAGE_EXECUTE 0x10 Enables execute access to the committed region of pages. An attempt to write to the committed region results in an access violation. This flag is not supported by the CreateFileMapping function.

PAGE_EXECUTE_READ 0x20 Enables execute or read-only access to the committed region of pages. An attempt to write to the committed region results in an access violation. Windows Server 2003 and Windows XP: This attribute is not supported by the CreateFileMapping function until Windows XP with SP2 and Windows Server 2003 with SP1.

PAGE_EXECUTE_READWRITE 0x40 Enables execute, read-only, or read/write access to the committed region of pages. Windows Server 2003 and Windows XP: This attribute is not supported by the CreateFileMapping function until Windows XP with SP2 and Windows Server 2003 with SP1.

and so on from here