Google URL Shortener 403 Rate Limit Exceeded

Question

Using the google url shortener api, it was working fine till I started testing at load. Quickly started getting back 403 Rate Limit Exceeded errors from Google, even though I signed up to use the API and it comes with 1,000,000 hits a day. I can see the requests coming in on the google reporting tool, and they are just sending back 403's for everything. 403's started coming back at around 345/350 hits to the API, have been continuing for hours.

Thoughts?

Answer 1

The API limits requests to 1 request / per second / per user.

A user is defined as a unique IP address.

So if you were doing your load testing from a single IP this would have cause your rate limit issue.

https://developers.google.com/analytics/devguides/reporting/mcf/v3/limits-quotas#general_api

Answer 2

I don't think "1 request / per second / per user." as written in doc is 100% correct in my case, or the google url shortener case. (FYI: I am using "Public API access", not "OAuth")

I have almost the same problem but, for me, it is more likely to be "I get this error for some URLs for some period of times." What does it mean? Please continue reading.

These are what I found:

• I can use 10 threads to use google url shortener at the same time, but not always ...
• when processing, even one url is fail on one thread, the other threads still can get the other urls.
• when a url is fail, and later I tried the same url again (even there are no other processes running, it still does not work for some PERIOD OF TIME. Even, I tried to add more string like "&test=1", it does not help. But if I changed to another url, it works.

So, I guess that google's server may have cache of each url. If a url is fail, it must wait for a while to let the cache released.

So, I have to write some creepy code like this to solve my problem:

• when there is a fail, that particular thread will sleep for 1 minute (yes 1 minute)
• and keep trying for 10 times (so totally, it can be 10 minutes for a fail url)

However, this creepy code is fine for my case because I am using ExecutorService with fixed-thread-pool size of 10. So, if there is a fail, the others still can get the shorten urls. It solves the problem...at least for me.