Heroku auto restart dyno on H12 Request timeout errors

Question

We have a node dyno processing small API requests, ~10/second. All requests complete in under 0.5s

Once every few days, dyno starts giving H12 Request timeout errors on all requests. We couldn't discover the cause. Restarting fixes it.

How to make Heroku automatically restart the dyno on a H12 Request timeout threshold, e.g. more than 5/second?

Answer 1

As ryan said H12 Request timeout means that Heroku's load balancers are sending a request to your app but not getting a response in time (heroku has a max response time of 30 seconds). Sometimes a request is just intense to calculate or an inefficient DB query is delaying the response.

Yet the root of the problem does not necessary mean an application error on your side.

In our case we have multiple web dynos handling requests in parallel. Now and then one of those dynos produces H12 (timeouts) while all others are running flawless. So we can completely rule out all application problems. A restart of the affected dyno helps with a high probability, as your application lands on a different physical server whenever it is restarted (at least with a high probability).

So Heroku has "bad servers" in their rotation! And now and then your code will land on one of those bad servers. I cannot say if one has a "noisy neighbor" problem. I also asked Heroku how to prevent that and the only response that I got was to pay for dedicated performance dynos, which is quite dissatisfying...

Answer 2

H12 Request timeout means that Heroku's load balancers are sending a request to your app but not getting a response.

This can happen for lots of reasons, since the app is already working you can likely rule out configuration issues. So now you are looking at the application code and will have to inspect to the logs to understand whats happening. I'd suggest using one of their logging apps like papertrail so you can have a history of the logs when this happens.

Some things it could be, but not limited to:

• Application crashing and not restarting
• Application generating an error, but no response being sent
• Application getting stuck in event loop preventing new request

Heroku provides some documentation around the issue that might help in debugging your situation

https://devcenter.heroku.com/articles/request-timeout https://help.heroku.com/AXOSFIXN/why-am-i-getting-h12-request-timeout-errors-in-nodejs