Why are RDS queries from EC2 to RDS taking around 22ms each, which is very slow

Question

I have an EC2 instance (medium, us-east-1d), and RDS instance (us-east-1a, db.t2.medium). I have a PHP page with a few dozen queries. Every single query takes consistently about 22 to 23ms. Which is crazy slow, it should take perhaps 1 or 2 ms each (locally each query takes less than 1 ms).

Any thoughts on how to find out why these queries are so slow? The database is fairly small, there are plenty of indexes, that's not the issue. It's RDS somehow being really slow.

Partial list of queries and how slow they are: (check the first one which consistenly takes > 100ms).

UPDATE

We stopped using RDS and moved the database onto the EC2 instance itself, and now it's blazing fast. I still don't know what happened with the RDS instance, but this fixed it.

Answer 1

Probably a simple answer, relating to the speed of light...

The "slow" server (usually 23ms) is hundreds of miles/kms from your client. The "fast" server (usually 0.5ms) is in the same building as your client.

To further confirm time, run a simple SELECT 1, preferably several times. That should measure mostly the lag between client and server.

If the server were on the other side of the globe, the timing, even for SELECT 1, would be over 200ms. The ultimate limit is the speed of light (until the next Einstein figures out that wormholes really exist).

If you are stuck with a long network lag, and you need to avoid it, we can talk about writing a Stored procedure with several queries in it; and then a single cross-network CALL to execute it.