SocketTimeoutException: Read timed out on Insert to Google Drive - is it retryable?

Question

Similar to Java Google Drive SDK SocketTimeoutException on File.insert.execute

Uploading thousands of files, one after the other, seems every 1 out of 50 files throws

java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:152)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
        at sun.security.ssl.InputRecord.read(InputRecord.java:480)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
        at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:36)
        at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:965)
        at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithoutGZip(MediaHttpUploader.java:545)
        at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequest(MediaHttpUploader.java:562)
        at com.google.api.client.googleapis.media.MediaHttpUploader.executeUploadInitiation(MediaHttpUploader.java:519)
        at com.google.api.client.googleapis.media.MediaHttpUploader.resumableUpload(MediaHttpUploader.java:384)
        at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:336)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:418)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
...

on files().insert().execute()

I'm not sure what it "means" that I'm getting this Exception when inserting a file.

1. Does it mean the insert request was not fulfilled, or not necessarily?
2. Does it simply mean the client did not hear back from the Google servers after it sent the insert request?
3. Should I be doing exponential back-off and retry when I get this error?

Answer 1

I have the same issue. One possible solution is to increase the read timeouts: Timeouts and Errors

private HttpRequestInitializer setHttpTimeout(final HttpRequestInitializer requestInitializer) {
    return new HttpRequestInitializer() {
        @Override
        public void initialize(HttpRequest httpRequest) throws IOException {
            requestInitializer.initialize(httpRequest);
            httpRequest.setConnectTimeout(3 * 60000);  // 3 minutes connect timeout
            httpRequest.setReadTimeout(3 * 60000);  // 3 minutes read timeout
        }
    };
}
...
service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, setHttpTimeout(credential)).build();

Answer 2

The obvious thing to do is to try to prevent the timeout. I don't think there is any way to be sure if the error occurred before or after the insert post was completed. It's quite common to get 501 responses when there are internal timeouts within the Drive infrastructure, but your case is different. The only way to be sure is to set a private property on the file object, and then search Drive to see if a file with that property exists (eg. mygeneratedId = some_random_UUID).

If you're inserting thousands of files simultaneously, it's conceivable that you are choking some resource (sockets, bandwidth, etc). You might want to try throttling your inputs slightly.