Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Increase maximum virtual memory size above 256gb

I'm running a program which allocates 8mb stacks using mmap. While testing to see how many stacks I could allocate (aiming for 100,000), I see virtual memory size rise quickly as expected, and reserved size stay small (less than 1gb). The program then segfaults with Cannot allocate new fiber stack: Cannot allocate memory (Errno). Using gdb to rescue the segfault and then looking at htop, I have discovered this happens at around 256GB of virtual memory.

I've tried using prlimit --as=unlimited --rss=unlimited --memlock=unlimited --data=unlimited when running the program, but it doesn't seem to make a difference.

Is there a way to increase this limit? Is it advisable to increase this limit? Is there a better way for crystal to allocate stacks?

like image 911
Stephie Avatar asked Dec 21 '16 17:12

Stephie


People also ask

Is it OK to increase virtual memory?

Increasing the maximum size enables exporting large maps that would not normally export if the settings were at the recommended size. Note: Microsoft recommends that virtual memory be set at no less than 1.5 times and no more than 3 times the amount of RAM on the computer.

How do I increase VM memory?

Click the entry for the virtual machine from the Window > Virtual Machine Library. Click Settings. Click on Processors & Memory in the settings window. Drag the slider to increase the amount of memory as required.

What is maximum memory can be used as a virtual memory?

Pagefile. RAM is a limited resource, whereas for most practical purposes, virtual memory is unlimited. There can be many processes, and each process has its own 2 GB of private virtual address space.

How much virtual memory should I set for 16GB RAM?

If you are lucky enough that you have more than 16 GB of RAM in the system, we suggest that the page file minimum be set between 1 and 1.5 times the amount of RAM.

How to increase virtual memory size in Windows 10?

Increasing Virtual Memory size is very easy. After configuring this setting, your PC will surely boot and perform faster. Open the System settings and go to System, Click on About, and under the “Related Settings” section tap on the System info option. On the left pane of the Window click on the ” Advanced System Settings ”

What is the maximum amount of virtual memory you can have?

Virtual memory upper limits are set by the OS: eg. 32-bit Windows the limit is 16TB, and on 64-bit Windows the limit is 256TB. Max limitation is physical disk space. To determine how much virtual memory you need, since the user's system contains the different amount of RAM, it is based on the system.

How to add 100GB of virtual memory to a process?

There is no way to add the RAM space, disk space, OS space, or anything else to get the 100GB of virtual memory the process would be using. Virtual memory is not limited by size of memory pointers in the machine, virtual memory limits are not the same as addressing memory space.

How to increase virtual memory on MacBook Air?

How to increase virtual memory using Settings 1 Open Settings. 2 Click on System. 3 Click on About. 4 Under the "Related settings" s ... 5 Click the "Advanced system set ... 6 Click the Advanced tab. 7 Under the "Performance" sectio ... 8 Click the Advanced tab. 9 Under the "Virtual memory" sec ... 10 Clear the Automatically manage ... 更多结果...


1 Answers

Maybe you're hitting the maximum of /proc/sys/vm/max_map_count. This setting sets a maximum on the number of mmaps your process can have. The default value is 65536. So it's likely not the size of memory you want to malloc, but the number of malloc calls that causes the error Cannot allocate memory.

You can try to increase the maximum with:

sysctl -w vm.max_map_count=131070

See also NPTL caps maximum threads at 65528?

like image 199
Robert Avatar answered Nov 20 '22 01:11

Robert