My server exited with code 137

Question

I wrote a C++ server/client pair using C++11, boost::asio and HDF5. The server was running fine for a some time (2 days), and then it stopped with code 137. Since I executed the server with an infinite loop, it was restarted.

Unfortunately, my error logs don't provide sufficient information to understand the problem. So I've been trying to understand what this code means. It seems there's consensus that this means it's an error of 128+9, with 9 meaning that the program was killed with kill -9. Now I'm not sure at all why this happened. I need help to find out.

By reading further, I found out that it could have been killed by the system because it exceeded a certain allowed execution time, and thus the system killed it. Now this is not so unlikely, since my linux server is provided by my university, so they could be applying some kind of security to do this. I read about something called timeout in linux. My first question is: How can I know if this is the cause of the problem?

My second question is: what should I check also to understand this problem? What would you do? Please advise.

If you require any additional information, please ask.

Thanks.

Answer 1

Sounds like you have blown through memory limits and your linux memory manager sent SIGKILL to your process. In that case you should check /var/log/messages file to see if there is anything about it. That's the first thing I would do. Check with your sysadmin if you don't have permissions.