Error code 487 (ERROR_INVALID_ADDRESS) when using VirtualAllocEX

Question

I'm trying to use VirtualAllocEx(). When I set dwSize (the third parameter) to a number larger than about 63 MB, it cause to generate error code 487 when I look at GetLastError(). However, it works with smaller sizes such as 4MB.

Here is part of my code:

VirtualAllocEx(peProcessInformation.hProcess,
               (LPVOID)(INH.OptionalHeader.ImageBase),
               dwImageSize, 
               MEM_RESERVE | MEM_COMMIT,
               PAGE_EXECUTE_READWRITE);

In the case that I used a 4MB EXE file, the LPVOID return value is 0x00400000, but in other cases (20MB or bigger file) it returns 0x00000000.

1. Is there a maximum value for the dwSize parameter?

2. Is there any other solution for my problem, such as another function?

Answer 1

My guess from your code is that you're trying to load a DLL or EXE into memory manually using something like this technique - is that right? I'll address this at the end (pun intended) but first a quick explanation of why VirtualAllocEx is failing.

Why is VirtualAllocEx giving this error?

The problem with allocating memory at a specific address is that there needs to be enough room at that address to allocate the memory size you request. This is why, generally, when you request memory you let the OS decide where to put it. (Plus letting the OS / malloc library decide can lead to other benefits, such as decreased fragmentation etc - out of scope for this answer.)

The problem you're getting is not that VirtualAllocEx is incapable of allocating 64MB rather than 4MB. VirtualAllocEx can allocate (nearly) as much memory as you want it to. The problem is that at the address you specify, in your process, there isn't 64MB of unallocated memory.

Consider hypothetical addresses 0-15 (0x0 - 0xF), where - marks empty memory and x marks allocated memory:

0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
x  x  -  x  -  -  -  -  x  -  -  -  -  -  -  -

This is your process's memory space. Now, you want to allocate 4 bytes at address 0x4. Easy - 0x4 to 0x7 are free, so you allocate and get (new allocation marked with X):

0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
x  x  -  x  X  X  X  X  x  -  -  -  -  -  -  -

Fantastic. But now suppose that instead you wanted to allocate 6 bytes. There aren't six free bytes at address 0x4: there's some memory being used at 0x8:

0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
x  x  -  x  -  -  -  -  x  -  -  -  -  -  -  -
            1  2  3  4  bang!

You can't do it. The problem isn't that the memory allocator can't handle allocating 6 bytes, but that the memory isn't free for it to do it. Nor, most likely, can it shuffle the memory around - in a normal non-GC program you can't move memory to make space, because you might, say, leave dangling pointers which don't know the contents of the memory they were pointing at has changed address. The only thing to do is either fail and not allocate memory at all, or allocate where it has free space, say at 0x9 or 0xA.

You might wonder why VirtualAllocEx is failing with ERROR_INVALID_ADDRESS instead of NULL: most likely, it is because you specified an address it couldn't allocate at; thus, even though there is some free memory at that address (maybe) there isn't enough and the address isn't valid. This is hinted at in the documentation:

Attempting to commit a specific address range by specifying MEM_COMMIT without MEM_RESERVE and a non-NULL lpAddress fails unless the entire range has already been reserved. The resulting error code is ERROR_INVALID_ADDRESS.

This isn't quite your situation: you're specifying both flags at once, but if the method can't reserve then it effectively falls into this situation. It can't reserve the entire range at that address, so it gives error code ERROR_INVALID_ADDRESS.

Loading DLL or EXE images

So, what should you do with your problem, which I am guessing from your question and code is loading a DLL or EXE image in memory?

Here you need a bit of background on image locations in an EXE file. Generally, an EXE is loaded into memory at the process's virtual address location 0x400000. It's optional: your linker can ask it be put wherever, but this value is common. Similarly, DLLs have a common default location: 0x10000000. So, for one EXE and one DLL, you're fine: the image loader can almost certainly load them at their requested locations.

What happens when you have two DLLs, both asking to be located at 0x10000000?

The answer is image rebasing. The image location is optional, it's not necessary. Code inside the image that depends on being loaded at a specific address can be adjusted by the image loader, and so the second DLL might be loaded not at 0x10000000, but somewhere else - say, 0x1080000. That's an address difference of 0x80000, and so the loader actually patches up a bunch of addresses and code inside the DLL so all the bits that thought they should refer to 0x10000000 now refer to 0x10800000.

This is really, really common, and every time you load an EXE this will be done to several DLLs. It is so common that Microsoft have a little optimisation tool called rebase, (for "rebasing", that is, adjusting the base address) and when you distribute your EXE and your own DLLs with it, you can use this to make sure each DLL has a different base address, each of which is located so that when Windows loads your EXE and the DLLs they will already have the right addresses and it is unlikely to have to rebase - perform the above operation - on any of them. For some applications this can make a noticeable improvement in starting time. (In modern versions of Windows, sometimes DLLs are moved around anyway - this is for address space layout randomization, a security technique to deliberately make sure code is not at the same address each time it's run.)

(One other thing is that some DLL and EXE compression tools strip out the data that is used for this relocation. That's fine, because it makes the EXE smaller... right up until it needs to be relocated, and because the data is missing it can't, and so can't be loaded at all. Or you can build with a fixed base, and it will magically work right until it doesn't. Don't do this to your EXEs or DLLs.)

So, what should you do when you try to manually load a DLL into memory, and there isn't enough space for it at the address it asks to be loaded at? Easy - it's not a fatal error, just load it somewhere else, and then perform the rebasing yourself. I would suggest if you have problems to ask a new SO question, but to give you a starting point you can use the RebaseImage function, or if you can't use it or want to do it yourself, I found this code which from a quick overview seems to perform this manually. No guarantees about its correctness.

TLDR

Your process address space doesn't have 64MB of empty space at the address you specify, so you can't allocate 64MB of memory there. Instead, allocate it somewhere else and patch / rebase your loaded DLL or EXE image to match the new address.