I know this is probably a stupid question but i've been looking for awhile and can't find a definitive answer. If I use mmap
or malloc
(in C, on a linux machine) does either one allocate space in RAM? For example, if I have 2GB of RAM and wanted to use all available RAM could I just use a malloc/memset
combo, mmap
, or is there another option I don't know of?
I want to write a series of simple programs that can run simultaneously and keep all RAM used in the process to force swap to be used, and pages swapped in/out frequently. I tried this already with the program below, but it's not exactly what I want. It does allocate memory (RAM?), and force swap to be used (if enough instances are running), but when I call sleep
doesn't that just lock the memory from being used (so nothing is actually being swapped in or out from other processes?), or am I misunderstanding something.
For example, if I ran this 3 times would I be using 2GB (all) of RAM from the first two instances, and the third instance would then swap one of the previous two instances out (of RAM) and the current instance into RAM? Or would instance #3 just run using disk or virtual memory?
This brings up another point, would I need to allocate enough memory to use all available virtual memory as well for the swap partition to be used?
Lastly, would mmap
(or any other C function. Hell, even another language if applicable) be better for doing this?
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define MB(size) ( (size) * 1024 * 1024)
#define GB(size) ( (size) * 1024 * 1024 * 1024)
int main(){
char *p;
p = (char *)malloc(MB(512));
memset(p, 'T', MB(512));
printf(".5 GB allocated...\n");
char *q;
q = (char *)malloc(MB(512));
memset(q, 'T', MB(512));
printf("1 GB allocated...\n");
printf("Sleeping...\n");
sleep(300);
}
** Edit: I am using CentOS 6.4 (with 3.6.0 kernel) for my OS, if that helps any.
mmap() is a system call that can be used by a user process to ask the operating system kernel to map either files or devices into the memory (i.e., address space) of that process. The mmap() system call can also be used to allocate memory (an anonymous mapping).
The “malloc” or “memory allocation” method is used to dynamically allocate a single large block of memory with the specified size. It returns a pointer of type void which can be cast into a pointer of any form.
Mmap is advantageous over malloc because memory used up by mmap is immediately returned to the OS. The memory used up by malloc is never returned unless there is a data segment break. This memory is specially kept to be reused.
Almost always, memory is much faster than disk, and malloc is not what's costing time. The mmap code is faster because for your program, mmap has resulted in either less disk access, or more efficient disk access, than whatever reads and writes you compared against.
Theory and practice differ greatly here. In theory, neither mmap
nor malloc
allocate actual RAM, but in practice they do.
mmap
will allocate RAM to store a virtual memory area data structure (VMA). If mmap
is used with an actual file to be mapped, it will (unless explicitly told differently) further allocate several pages of RAM to prefetch the mapped file's contents.
Other than that, it only reserves address space, and RAM will be allocated as it is accessed for the first time.
malloc
, similarly, only logically reserves amounts of address space within the virtual address space of your process by telling the operating system either via sbrk
or mmap
that it wants to manage some (usually much larger than you request) area of address space. It then subdivides this huge area via some more or less complicated algorithm and finally reserves a portion of this address space (properly aligned and rounded) for your use and returns a pointer to it.
But: malloc
also needs to store some additional information somewhere, or it would be impossible for free
to do its job at a later time. At the very least free
needs to know the size of an allocated block in addition to the start address. Usually, malloc
therefore secretly allocates a few extra bytes which are immediately preceding the address that you get -- you don't know about that, it doesn't tell you.
Now the crux of the matter is that while in theory malloc
does not touch the memory that it manages and does not allocate physical RAM, in practice it does. And this does indeed cause page faults and memory pages to be created (i.e. RAM being used).
You can verify this under Linux by keeping to call malloc
and watch the OOP killer blast your process out of existence because the system runs out of physical RAM when in fact there should be plenty left.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With