Performance of new Amazon SNS Mobile Push Service

Question

Does anyone have performance data for Amazon's new Mobile Push service?

We are looking at using it, but want to understand performance for:

1. How many requests per second it can handle
2. Latency for delivering a notifications to a device in seconds
3. How long it takes to send an identical notification to a million users (using topics)

Since Amazon doesn't publish performance numbers and because creating synthetic tests for mobile push are difficult, I was wondering if anyone had real-world data.

Answer 1

We've sent a message to around 300,000 devices and they are delivered almost instantaneously. Obviously we do not have access to each of those devices, but judging by a sampling of devices that are subscribed to various topics at different times, all receive the message less than 10 seconds from the actual send.

A single publish to a device from the AWS console is startlingly fast. It appears on your device at almost the same instant that you release the "Publish" button on the AWS console.

While the delay in the AWS delivery infrastructure is nominal, and will surely be driven to near zero as they improve and add to their infrastructure, the time between the user action that generates the message in your system and the actual message is received by AWS that says "send this notification" will likely be the larger portion of the delay in the end-to-end process. The limit per topic is 10,000 devices, so if you are sending to a million users, you'll have 100 (or more) topics to publish to. The time it takes your software to publish to all of these topics depends on how much parallelism you support in the operation. It takes somewhere around 50-100ms to publish to a topic, so if you do this serially, it may be up to 10 seconds before you even publish your message to the 100th topic.

UPDATE: As of August 19, 2014, the limit on the number of subscribers you can have per topic has been raised to 10,000,000: https://aws.amazon.com/blogs/aws/sns-large-topics-and-mpns-auth-mode/