Other Possible Reasons for std::bad_alloc being thrown

Question

I am working on quite a large SIP telephony application and occasionally when we use the integrated web UI (written using tntnet) under heavy call load the program will exit due to a std::bad_alloc being thrown. There are hundreds of threads in use (3 per active call), so the location of the code causing the exception is quite random but is always after using the GUI.

Now, I understand that std::bad_alloc can be thrown when out of memory, which is not the case in this situation. I also am thinking that it can be thrown when there is heap corruption, which I am still looking for where ever it may be in the code base.

But my question is, are there any other reasons that std::bad_alloc will be thrown other than out of memory or heap corruption? I am using GNU g++ on Linux.

Answer 1

Most likely, you really are out of memory. It would be an extremely rare heap corruption bug that consistently caused only bad_alloc to be thrown. That would be like scribbling with surgical precision.

It is possible there's simply a bug in the code that allocates a massive amount of memory. But you would expect the exception to be thrown, at least a large fraction of the time, in that very code. The fact that the exception is coming from a number of different places weighs against that.

Severe fragmentation can cause a problem, particularly for platforms with poor implementations of malloc. That's rare, but it does happen.

One thing I'd do immediately -- catch the exception and call a function that saves a copy of /proc/self/maps. That will give you a good idea of the peak memory usage of the process. You can judge if it's anywhere near any platform, policy, or hardware limitations.

Answer 2

On linux the current Address Space limit can be used to artificially limit the amount of memory a process can use. You can manually set this with setrlimit(RLIMIT_AS, ...). This can also be set for an entire shell in bashrc using ulimit -v. This can also be set for the entire system in /etc/security/limits.conf. There might even be a /proc/sys entry for this someplace, I'm not sure.

If the address space limit is reached, your process will throw an std::bad_alloc when trying to allocate more memory. On a 64-bit system this can be a nice "safety" to make sure a bad application or library doesn't run away with the available memory and make the system go to swap or stop working altogether. Make sure the program doesn't set this itself someplace, and make sure the rest of the environment hasn't set it either. You can just insert some code in the middle of the program someplace to call getrlimit(RLIMIT_AS, ...) to be sure it hasn't snuck in someplace.

A perhaps more common cause (other than actually running out of memory, of course) is an unsigned integer wrap around case, where a uin32_t or uint64_t is used to allocate memory but was 0 and had 1 subtracted from it, leading to a very large request allocation (in 64-bits that would be many thousands of petabytes).

In any case, the best ways to track this down are with GDB. If your application doesn't use exceptions at all (and thus has no "catch" statements at all) then you can enable core files (ulimit -c unlimited). The next time the program crashes, the OS will generate a core file and loading it up in GDB will immediately give you a backtrace showing you where the program crashed.

If you have a few (but not many) places where try is used and its catching these bad allocs, other than just commenting them out while you debug this problem, you can run the application in GDB and use the catch throw command to have GDB break every time an exception is thrown. For either of these to work, never compile with -fomit-frame-pointer and always compile (even when using -O3) with -ggdb.