Heroku PHP Immediately Consuming all of the memory

Question

I have a Heroku 1X dyno that immediately consumes all of its memory upon restart.

Here is the restart in the log:

2020-05-18T16:57:54.119229+00:00 app[web.1]: Stopping php-fpm...
2020-05-18T16:57:54.119814+00:00 app[web.1]: Stopping httpd gracefully...
2020-05-18T16:57:54.121897+00:00 app[web.1]: Stopping httpd...
2020-05-18T16:57:54.131154+00:00 app[web.1]: Shutdown complete.
2020-05-18T16:57:54.194178+00:00 heroku[web.1]: Process exited with status 143
2020-05-18T16:57:57.194751+00:00 app[web.1]: Detected 536870912 Bytes of RAM
2020-05-18T16:57:57.231186+00:00 app[web.1]: PHP memory_limit is 4M Bytes
2020-05-18T16:57:57.245920+00:00 app[web.1]: Starting php-fpm with 128 workers...
2020-05-18T16:57:57.386605+00:00 app[web.1]: Starting httpd...
2020-05-18T16:57:58.220510+00:00 heroku[web.1]: State changed from starting to up

My procfile is just:

web: vendor/bin/heroku-php-apache2

My .user.ini is:

memory_limit = 4M

None of this code is set to run in the background, it's just an API that handles requests. And yet it's immediately consuming the maximum memory and throwing R14 errors with 0 requests coming in. https://share.getcloudapp.com/GGukb5QZ

Anyone know what could be happening?

Answer 1

Here is the documentation you need to read: https://devcenter.heroku.com/articles/php-concurrency

You answered this with your comment "I'm able to reduce the memory floor (so that it doesn't take up all of the memory on start) by INCREASING the memory_limit."

TL;DR, Heroku says:

• How much memory have I got? (You have 512MB)
• How much memory max per PHP process (You set to 4MB)
• Divide the two and start that many workers (You get 128)

What this calculation ignores is the overhead of running HTTP and other processes. As clarified on that page "These defaults are intentionally chosen to not leave any “headroom” ... because applications are extremely unlikely to consume their entire memory limit ... meaning that dynos are slightly over-subscribed by default."

Usually that won't matter if you have limit set to 128MB = 4 worker processes:

• 128MB per php process, but using only 100MB on average (that's HUGE :) )
• Add 1MB per http process = 101MB
• Multiple By 4 = 404MB
• Add on overhead of logging, log rotation, SSH and all those hundred of other processes, lets say 40MB
• Total = 404 + 40 = 444MB < 512MB.
• If using only 3.5MB per process = plenty spare.
• YAY!

However, you have:

• 4MB per php process, using only 3.5MB on average (check memory-get-peak-usage in PHP)
• Add 1MB per http process = 4.5MB
• Multiple By 128 = 576MB
• Add on overhead of logging, log rotation, SSH and all those hundred of other processes, lets say 40MB
• Total = 576 + 40 = 616MB > 512MB.
• Heroku starts memory management.

You need to find your right balance, e.g. set to 8M = 64 processes as you've done.

---

Note: the statements about the overheads and HTTP processes is a bit of a simplification used for illustrative purposes only.

Configuring FPM is a bit of a nightmare all round as you can run multiple pools each with initial workers and spare workers. https://www.php.net/manual/en/install.fpm.configuration.php

Heroku's trying to simplify this for you.

Answer 2

It reads: PHP memory_limit is 4M Bytes, 4 megabytes, which I would call an obvious mis-configuration. Try to set a rather realistic value of about 128M ~ 256M. That instance overall only has 512M. KeepAlive barely affects the memory, but one may run out of ports (don't worry with 128 FPM worker processes)... maybe configure the server less restrictive once and then check with memory_get_usage(false) (used memory, not allocated memory) how much memory the script actually needs. Unless knowing that value it is difficult to know what the least possible value is.

Heroku says, that R14 means: Memory quota exceeded ...so these are probably 128 workers with 4M each, which is already the hard limit of 512M. Setting up less workers might be an option; adding more RAM might be an option. When not setting the value to 4M, this would be the default value of 128M; when setting it to -1 it would ignore that soft limit (which might help when testing).