Google Drive API request limitations

Question

Im asking for support or special knowledge.

In my app Im using Google Drive and need to insert/remove permissions to 1000+ files. Google API does not allow me to set permission to multiple files in one request, so I need to make 1000+ requests (or wrap it into batch or few batches).

My requests looks like this:

POST /drive/v2/files/0B18tlN6SgYRaUzl1MUlnNHlGSEE/permissions?sendNotificationEmails=false&quotaUser=00787465305247954313&alt=json

But when Im trying to make batch request to Google Drive API, I got batch response with alot of error responses. Few first responses in batch are OK but not all.

Most of these errored responses looks like this:

Content-Type: application/http
Content-ID: <response-280781395>

HTTP/1.1 403 Forbidden
Content-Type: application/json; charset=UTF-8
Date: Tue, 09 Sep 2014 11:45:03 GMT
Expires: Tue, 09 Sep 2014 11:45:03 GMT
Cache-Control: private, max-age=0
Content-Length: 199

{
 "error": {
  "errors": [
   {
    "domain": "usageLimits",
    "reason": "rateLimitExceeded",
    "message": "Rate Limit Exceeded"
   }
  ],
  "code": 403,
  "message": "Rate Limit Exceeded"
 }
}

Some of them looks like this:

HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=UTF-8
Date: Tue, 09 Sep 2014 11:45:03 GMT
Expires: Tue, 09 Sep 2014 11:45:03 GMT
Cache-Control: private, max-age=0
Content-Length: 180

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "internalError",
    "message": "Internal Error"
   }
  ],
  "code": 500,
  "message": "Internal Error"
 }
}

And last few responses in the batch looks like this:

Content-Type: application/http
Content-ID: <response-901482964>

HTTP/1.1 400 Bad Request
Content-Type: application/json; charset=UTF-8
Date: Tue, 09 Sep 2014 11:45:03 GMT
Expires: Tue, 09 Sep 2014 11:45:03 GMT
Cache-Control: private, max-age=0
Content-Length: 171

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "badRequest",
    "message": "Bad Request"
   }
  ],
  "code": 400,
  "message": "Bad Request"
 }
}

What is strange when I got quota limit error and decided to increase requests/user/second quota in Google Drive Console. It was set to 10000/user/second. But i sent only 1000 requests in batch but already got "403 Rate Limit Exceeded" errors. And then i began to get "500 Internal Error" errors.

Then I found that i should set 'quotaUser' param to request but nothing changed. I still get these kind of errors.

Investigation in Google Drive Web UI showed me how Google Drive sets permissions to files if decided to share 1000+ files. It tooks me about 20-30 minutes before all my files was shared. In Activity panel I saw that files shared in bunches. Sometimes these bunches was 1 file sized, othertimes 40 files sized. But all files was shared in the end, looks like Google Drive team uses something like retrying queue or just not set quotasfor own requests.

Main questions is:

• Why im still getting these quota errors (specified 10000 requests/user/second limit in Google Cloud Console) and how to avoid them?
• Why Google Drive API returns "500 Internal Error" and "400 Bad Request" errors?
• Any workarounds for this?

P.S.: Im interested into help Google Drive team, I can provide full traces and code samples.

Answer 1

While you can set a per-user QPS to keep malicious users from consuming your quota, setting it to an arbitrarily high number doesn't mean you'll be able to make that many requests. There are per-app QPS limits that kick in as well. The default is well below 1000qps.

If you need additional quota, request it via the developers console.