Google Load Balancer returning 502

Question

I have a container cluster running behind a load balancer on GKE. It's working well, but I occasionally get 502 errors when I try to access pages. The logs show the following:

{
metadata:
{
severity:
"WARNING"
projectId:
"###"
serviceName:
"network.googleapis.com"
zone:
"global"
labels:
{…}
timestamp:
"2016-04-28T16:35:46.864379896Z"
projectNumber:
"###"
}
insertId:
"2016-04-28|09:35:47.696726-07|10.94.35.131|1729057675"
log:
"requests"
httpRequest:
{
requestMethod:
"GET"
requestUrl:
"https://###/user/view/111"
requestSize:
"2089"
status:
502
responseSize:
"362"
userAgent:
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.86 Safari/537.36"
remoteIp:
"###"
referer:
"###"
}
}

When I review the access logs from my containers, I do not see any matching requests hitting them at the times that the 502 errors are generated. It appears that they are not going past the load balancer.

Has anyone experienced this issue with Load Balancers? Any recommended solution? Thanks.

Answer 1

A 502 error suggests that the load balancers are sending traffic but not receiving responses. Is it possible that your endpoints are reporting healthy, but some containers are not ready to serve?