How to resize regions allocated by VirtualAlloc?

Question

I would like to resize a region of memory allocated by MS window's VirtualAlloc. Looking at the VirtualFree documentation, it is possible to decommit a region only partly, but it's not possible to partially release it. That is, it's possible to release part of the physical memory, but not part of the virtual memory.

I'm aware it may be necessary to reallocate the region in such a case. However, copying over the entire region would be rather inefficient. Is there a way to ask windows to allocate a new region with a different size, that points to the same memory?

Answer 1

As you have mentioned, it does not appear to be possible to partially release a range of reserved pages because the VirtualFree() documentation states:

If the dwFreeType parameter is MEM_RELEASE, [lpAddress] must be the base address returned by the VirtualAlloc function when the region of pages [was] reserved.

as well as:

If the dwFreeType parameter is MEM_RELEASE, [dwSize] must be 0 (zero).

VirtualFree() is itself a thin wrapper of the kernel function NtFreeVirtualMemory(). Its documentation page (the same as for ZwFreeVirtualMemory()) also has this wording.

One possible work-around is to split up a single, large reservation with multiple smaller ones. For example, suppose that you normally reserve 8 MiB of virtual address space at a time. You could instead attempt to reserve the range in thirty-two contiguous 256 KiB reservations. The first 256 KiB reservation would contain a 32-bit unsigned bit field, where the i^th bit is set if the i^th 256 KiB reservation was obtained:

#define NOMINMAX
#include <windows.h>
#include <assert.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

#define RESERVATION_SIZE (256*1024)

typedef struct st_first_reservation {
    size_t reservation_size;
    uint32_t rfield;
    char premaining[0];
} st_first_reservation;

int main()
{
    SYSTEM_INFO sys_info = { 0 };
    GetSystemInfo(&sys_info);

    assert((RESERVATION_SIZE % sys_info.dwPageSize) == 0);

    void *vp = VirtualAlloc(NULL, 32*RESERVATION_SIZE, MEM_RESERVE, PAGE_NOACCESS);
    if (VirtualFree(vp, 0, MEM_RELEASE) == 0) {
        fprintf(stderr, "Error: VirtualFree() failed.\n");
        return EXIT_FAILURE;
    }

    st_first_reservation *pfirst_reservation = (st_first_reservation *) VirtualAlloc(vp, RESERVATION_SIZE, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
    if (pfirst_reservation == NULL) {
        pfirst_reservation = (st_first_reservation *) VirtualAlloc(NULL, RESERVATION_SIZE, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
        if (pfirst_reservation == NULL) {
            fprintf(stderr, "Error: VirtualAlloc() failed.\n");
            return EXIT_FAILURE;
        }
    }

    fprintf(stderr, "pfirst_reservation = 0x%p\n", (void *) pfirst_reservation);

    pfirst_reservation->reservation_size = RESERVATION_SIZE;
    pfirst_reservation->rfield = 1LU;

    char *p = (char *) pfirst_reservation;
    unsigned i = 1;
    for (; i < 32; ++i) {
        vp = VirtualAlloc(p += RESERVATION_SIZE, RESERVATION_SIZE, MEM_RESERVE, PAGE_NOACCESS);
        if (vp != NULL) {
            assert(((void *) vp) == p);
            pfirst_reservation->rfield |= 1LU << i;
            fprintf(stderr, "Obtained reservation #%u\n", i + 1);
        } else {
            fprintf(stderr, "Failed to obtain reservation #%u\n", i + 1);
        }
    }

    fprintf(stderr, "pfirst_reservation->rfield = 0x%08x\n", pfirst_reservation->rfield);

    return EXIT_SUCCESS;
}

Sample output:

pfirst_reservation = 0x009A0000
Obtained reservation #2
Obtained reservation #3
Obtained reservation #4
Obtained reservation #5
Obtained reservation #6
Obtained reservation #7
Obtained reservation #8
Obtained reservation #9
Obtained reservation #10
Obtained reservation #11
Obtained reservation #12
Obtained reservation #13
Obtained reservation #14
Obtained reservation #15
Obtained reservation #16
Obtained reservation #17
Obtained reservation #18
Obtained reservation #19
Obtained reservation #20
Obtained reservation #21
Obtained reservation #22
Obtained reservation #23
Obtained reservation #24
Obtained reservation #25
Obtained reservation #26
Obtained reservation #27
Obtained reservation #28
Obtained reservation #29
Obtained reservation #30
Obtained reservation #31
Obtained reservation #32
pfirst_reservation->rfield = 0xffffffff

EDIT: I have found that it is much better to "pre-reserve" the thirty-two 256 KiB ranges all at once, free, and then try to re-reserve as many as you can.

I updated the code and sample output above.

In a multithreaded environment, the code may fall back to the "place anywhere" allocation of the first reservation. Perhaps it is a good idea to attempt reserving RESERVATION_SIZE bytes at a reserved-then-freed range of 32*RESERVATION_SIZE bytes five or so times, finally falling back to the "place anywhere" allocation.

Answer 2

Not an answer, but I have to ask:

Considering the pain you're in, the performance hits of VirtualAlloc( ), and the non-portability of your code; as against any value VIrtualAlloc( ) gives, could you perhaps consider using malloc( ) and friends instead? IOW, does VirtualAlloc( ) confer any real advantage?

In my opinion (maybe only my opinion), the power and generality of malloc( ) outweigh any of the allure that VirtualAlloc( ) promises. And it would let you deal with your regions much more straightforwardly.

Sorry for the non-answer. I hate it when folks ask "who ever would even think to do that?" But of course it's all different when I'm the one asking "why" :-)

Answer 3

If you want to shrink an allocation, you can use VirtualFree with MEM_DECOMMIT on a subrange of the allocation. Note that this won't free up address space; only physical RAM. If you want to grow it, you can try VirtualAlloc passing an address immediately after your existing allocation. This may, of course, fail, at which point you need to copy memory.

You can also try using GlobalAlloc with GMEM_MOVEABLE and GlobalReAlloc (or the equivalent Heap* functions).

If you need to free address space, you might want to try using anonymous memory-mapping objects, and changing their mapped window at run-time - or simply use 64-bit to get additional address space.