Is Azure Throttling My WebApi?

Question

I and a few other developers are having a an AI competition. Basically, we have a simple board game and we've each written AIs that will be hosted on our own webservices to play against each other.

I've got one up and running that is a simple WebApi 2 based service that is published to Azure. My expectation is it'll be quiet most of the time, and then suddenly when a game starts, it'll receive 200-400 requests all potentially very quickly (a game can last only a few seconds). The game server and AI communicate with run-of-the-mill JSON POSTs.

My issue is, Azure seems to be throttling the requests. The first few moves happen very fast, and then everything slows down.

I'm wondering if Azure thinks it's a potential DOS attack or something. It can take over a second to get a move back once things slow down. A few points of interest:

• This never happens when hosted locally.
• There is no database at all.
• It happens even with a simple test AI that just moves pieces randomly (so no compute load)
• It happens when the game host is also deployed to Azure (so it's two Azure sites talking to each other).

So how can I fix this? Is there a setting somewhere I'm missing where I can tell Azure this kind of behavior is expected? I currently have it as a Shared service, but I tested it as a Basic service before with two instances, and I still was seeing the slowdown.

Answer 1

Yes, you might be getting throttled if you were on Free or Shared plan. With free plan you get 60 minutes CPU time per day and with Shared you get 240 minutes CPU time per day.

Also maximum average memory usage (per hour) for shared website is 1 GB. But I am assuming you would have already profiled your code for memory leaks.

There are limits on websockets but I am not sure if it is a general TCP connection limit.

Here is the link for the limits.

I think Basic plan should give you predictable performance, but it Small instance in Basic plan is just 1.6 GHz CPU, which most probably will be far less CPU and Cores than your local machine.

I think Standard A2 size cloud service web role (2 cores) might be more appropriate than a website. Depending upon your code (if they are able to utilize most of the CPU and can distribute tasks between roles) even multiple A0 instances, would be much more recommended.

For a fair competition, you might have CPU constrain as well, it is very interesting.

All the best, and May the best code win :)

Answer 2

The issue was the number of simultaneous connections allowed. The client was being very sloppy and making a new connection for each request. This caused the number of connections to exceed the limit. It could have been resolved as in this answer (as Shay pointed out in the comments, I wish I could mark a comment as the correct answer!). But as I also have access to the client code, I fixed it there.