Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache/PHP7.3 running in Docker randomly drops connection with empty response

I have found several similar questions:

  • APACHE, PHP Server return randomly empty response
  • https://serverfault.com/questions/66662/apache-gives-empty-reply
  • and others

However these does not seem to help to find the cause. I can replicate the behaviour when reloading a specific page ~20 times.

  1. Running current apache2 (= 2.4.38-3+deb10u4). I tried to disable opcache, remove MaxRequestsPerChild with no effect.
  2. Apache log does not show any error. The request is not even logged.
  3. The USE_ZEND_ALLOC=0 seem to have no effect and the problem persists.
  4. I tried to install mod_forensic which shows that the request came in. No error or finished request is then logged.
  5. The container is running in Kubernetes and I cannot replicate the issue locally running directly with Docker machine, that is why I think this might be caused by some memory setting. However I couldn't find what might be causing this as there is no single error message.

Can you think of any reason why this might be happening?

Edit1:

I tried to set log level to trace:

https://gist.github.com/knyttl/861e8a0fe5651408df37cd5c3874946b

The request is handled and then you can see:

[Tue Oct 20 08:37:55.825454 2020] [core:trace4] [pid 1] mpm_common.c(536): mpm child 388 (gen 2/slot 4) exited

With no error and no response.

Edit2:

I updated to php7.4 and the issue persists.

like image 226
Vojtěch Avatar asked Oct 19 '20 19:10

Vojtěch


Video Answer


1 Answers

I finally found it:

The process is silently killed by OOM killer of the host machine:

[4019392.626796] Memory cgroup out of memory: Kill process 4178127 (apache2) score 1137 or sacrifice child
[4019392.636520] Killed process 4178127 (apache2) total-vm:143960kB, anon-rss:22856kB, file-rss:10472kB, shmem-rss:28228kB

This is never logged within the container so it was hard to find.

like image 143
Vojtěch Avatar answered Oct 10 '22 07:10

Vojtěch