Sending a GCM message (server-side) often fails - but far from always

Question

I am using the Google Cloud Messaging (GCM) service for my Android app. I have implemented it according to all the rules, and it works. Well, almost.

Most often, I'd say in 60-70% of the cases I can successfully send a GCM message from my server, using the webservice as discussed on google webpages.

Normally, I get the following reply from the webservice, which indicates that I successfully sent the GCM message:

{
    "multicast_id":8378088572050307085,
    "success":1,
    "failure":0,
    "canonical_ids":0,
    "results":
    [
        {
            "message_id":"0:1363080282442710%7c4250c100000031"
        }
    ]
}

This is saying: all OK, message sent.

However, in many cases I get a HTTP error when calling the webservice, that says:

Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.

This is the .NET message to tell me that calling a webservice (using HttpWebRequest and POST) failed.

This is some log messages that shows the problem:

This is the code I am using for calling the WS:

public static string SendMessage(string registrationId, string command, string extra, bool retry)
{
    try
    {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://android.googleapis.com/gcm/send");
        request.Method = PostWebRequest;
        request.KeepAlive = false;

        GCMPostPacket json = new GCMPostPacket()
        {
            collapse_key = "1",
            time_to_live = 60,
            registration_ids = new List<string>(new string[] { registrationId }),
            data = new GcmData()
            {
                message = command,
                misc = extra
            }
        };
        // Converting to JSON string
        string jsonString = SICJsonProtocol.JSONHelper.Serialize<GCMPostPacket>(json);
        byte[] byteArray = Encoding.UTF8.GetBytes(jsonString);

        request.ContentType = "application/json";
        request.ContentLength = byteArray.Length;
        request.ProtocolVersion = HttpVersion.Version10;

        request.Headers.Add(HttpRequestHeader.Authorization, "key=" + "MyVerySecretKey");

        Stream dataStream = request.GetRequestStream();
        dataStream.Write(byteArray, 0, byteArray.Length);
        dataStream.Close();

        using (WebResponse response = request.GetResponse())
        {
            HttpStatusCode responseCode = ((HttpWebResponse)response).StatusCode;
            if (responseCode.Equals(HttpStatusCode.Unauthorized) || responseCode.Equals(HttpStatusCode.Forbidden))
            {
                Console.WriteLine("Unauthorized - need new token");
            }
            else if (!responseCode.Equals(HttpStatusCode.OK))
            {
                Console.WriteLine("Response from web service not OK :");
                Console.WriteLine(((HttpWebResponse)response).StatusDescription);
            }

            StreamReader reader = new StreamReader(response.GetResponseStream());
            string responseLine = reader.ReadLine();
            Console.WriteLine("************************");
            Console.WriteLine("GCM send: " + responseCode + " | " + responseLine);
            // This is the log shown in the image above
            SRef.main.gui.ServiceUpdate("GCM send: " + responseCode + " | " + responseLine);
            reader.Close();
            response.Close();
            return responseLine;
        }

    }
    catch (Exception e)
    {
        // This is the log shown in the image above
        SRef.main.gui.ServiceUpdate("Failed send GCM, " + (retry ? "retrying in 20 sec" : "not retrying") + ". Error=" + e.Message);
        if (retry)
        {
            System.Threading.ThreadPool.QueueUserWorkItem(delegate(object obj)
            {
                try
                {
                    System.Threading.Thread.Sleep(20000);
                    SendMessage(registrationId, command, extra, false);
                }
                catch (Exception ex)
                {
                }
            });
        }
        return null;
    }
}

Can anyone see if I am doing something wrong, or if I am missing something in general?

Answer 1

Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.

This error has nothing to do with GCM API in particular. It means that your client tried to contact the web service but the connection that was established has been severed in one of the network layers. Depending on where you get this error and the error message it could mean several things.

1. Your client decided to abort the connection due to a socket timeout. Increase the client read / socket timeout to improve the situation.

2. A load balancer that sits between the client and the server dropped the connection. Each party thinks that the other one dropped the connection.

Network timeouts are usually a huge pain since it is never clear where the connection was dropped and who dropped it. If increasing timeouts does not help I'd suggest sending the requests through a proxy that can sniff the HTTPS traffic (charles / TCPMON) or using Wireshark to see which packets are being dropped.

Your android app also has GCM monitoring that you can enable on the Statistics tab. Check if the GCM API reports a status message other than 200 OK on that graph. That will help narrow down the problem further. If there are no reports of status codes other than 200, it means GCM never got your API requests to begin with.