How to keep SSHD open when out of memory

Question

I'm on EC2 t2.small instance (2GB RAM) and keep getting locked outside the server without the ability to ssh into it.
error message is ssh_exchange_identification: read: Connection reset by peer
Knowing the application, I assume it leaks memory and that SSHD is blcoked due to that.

Is there a way to allow SSH connection to a linux box that has its RAM full?
Can I someone have SSHD reserve enough memory for new connections?

Answer 1

When server runs into out-of-memory, it usually kills several applications which is control by OOM(out of memory) Killer. In your case when your server goes into out-of-memory it kills SSH process to free ram. We can avoid this by disabling OOM killer for ssh Process :

Disabling OOM killer for any process :

  echo -17 > /proc/`pidof Process`/oom_adj

Disabling OOM Killer for ssh all process:

 pgrep -f "/usr/sbin/sshd" | while read PID; do echo -17 > /proc/$PID/oom_adj; done

To automate this we need to set crontab for 1 min

   */1 * * * * root pgrep -f "/usr/sbin/sshd" | while read PID; do echo -17 > /proc/$PID/oom_adj; done

You can read more about the Linux OOM killer here.

Answer 2

While you can't pre-allocate memory for sshd, you can set the memory limit for the application in such a way that it can't eat all the memory. See man ulimit how to do that.

If it's a Java application, try -Xmx to set the maximum amount of memory which it can allocate.