Unable update file store in appdata scope - 500 Internal Server Error

Question

Previously, I have a set of Google Drive API code, which works fine in the following scenarios

1. Save new file to appdata
2. Update previous file in appdata
3. Save new file to non-appdata
4. Update previous file in non-appdata

Few days ago, I encounter scenario 2 no longer work (Update previous file in appdata), whereas other scenarios still work without problem. I will be getting the following exception.

com.google.api.client.googleapis.json.GoogleJsonResponseException: 500 Internal Server Error
{
  "code": 500,
  "errors": [
    {
      "domain": "global",
      "message": "Internal Error",
      "reason": "internalError"
    }
  ],
  "message": "Internal Error"
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:423)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
    at org.yccheok.jstock.gui.Utils.updateFile(Utils.java:1414)

I'm using both DRIVE and DRIVE_APPDATA scope - authorizeDrive()

The code is as follow

• saveToGoogleDrive() - Save or update file in appdata, doesn't work during update file. Work during saving new file.
• saveToLegacyGoogleDrive() - Save or update file in non-appdata, all works!

Exception is being thrown at Line 1414, which is

com.google.api.services.drive.model.File updatedFile = service.files().update(fileId, file, mediaContent).setNewRevision(false).execute();

Searching previous file in appdata using query title contains 'jstock-fe78440e-e0fe-4efb' and trashed = false and 'appdata' in parents is completely fine. We're able to retrieve the previous file id without problem.

However, 500 Internal Server Error is being thrown, when we perform file updating using retrieved file id.

Some users encountered problem during searching in appdata (which is not my case). Search folder inside 'appdata' folder The suggested workaround is to add drive.readonly.metadata. I had tried that once, but it makes no difference.

---

Update

An excellent workaround proposed by Jon Skeet

I've managed to reproduce the issue. Without setNewRevision(false) it works - I realize that may not be feasible in all cases, but is it a reasonable workaround for you for the moment?

However, I will on hold on such workaround at this moment. We prefer to have setNewRevision(false), to prevent from causing increased use of the user's data storage quota - http://developers.google.com/drive/v2/reference/files/update

---

Short but complete source code to demonstrate the problem

1. Create a client id & secret key. Update source code.
2. Create a document.txt
3. Run the source code first time, to upload document.txt to appdata folder. It should success. Check your uploaded file through online Google Drive. (Please refer to attachment)
4. Run the source code for second time, to perform update on previous document.txt in appdata folder. The 500 Internal Server Error exception should be thrown.

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */

package insert;

import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeFlow;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.auth.oauth2.GoogleTokenResponse;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.FileContent;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.drive.Drive;
import com.google.api.services.drive.DriveScopes;
import com.google.api.services.drive.model.FileList;
import com.google.api.services.drive.model.ParentReference;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.security.GeneralSecurityException;
import java.util.Arrays;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;


public class Insert {

    private static com.google.api.services.drive.model.File searchFromGoogleDrive(Drive drive, String qString) {
        try {
            Drive.Files.List request = drive.files().list().setQ(qString);

            do {                
                FileList fileList = request.execute();

                com.google.api.services.drive.model.File file = null;

                for (com.google.api.services.drive.model.File f : fileList.getItems()) {

                    final String title = f.getTitle();

                    if (title == null || f.getDownloadUrl() == null || f.getDownloadUrl().length() <= 0) {
                        continue;
                    }

                    file = f;

                    break;
                }

                if (file != null) {
                    return file;
                }

                request.setPageToken(fileList.getNextPageToken());
            } while (request.getPageToken() != null && request.getPageToken().length() > 0);
        } catch (IOException ex) {
            log.error(null, ex);
            return null;
        }
        return null;
    }

    public static boolean saveToGoogleDrive(Credential credential, java.io.File file) {
        final String titleName = "document.txt";
        final String qString = "title contains '" + titleName + "' and trashed = false and 'appdata' in parents";        
        return _saveToGoogleDrive(credential, file, qString, "appdata");
    }

    public static Drive getDrive(Credential credential) {
        Drive service = new Drive.Builder(httpTransport, JSON_FACTORY, credential).setApplicationName("JStock").build();
        return service;
    }

    private static boolean _saveToGoogleDrive(Credential credential, java.io.File file, String qString, String folder) {
        Drive drive = getDrive(credential);

        // Should we new or replace?

        com.google.api.services.drive.model.File googleCloudFile = searchFromGoogleDrive(drive, qString);

        final String title = "document.txt";

        if (googleCloudFile == null) {
            String id = null;
            if (folder != null) {
                com.google.api.services.drive.model.File appData;
                try {
                    appData = drive.files().get(folder).execute();
                    id = appData.getId();
                } catch (IOException ex) {
                    log.error(null, ex);
                    return false;
                }
            }
            return null != insertFile(drive, title, id, file);
        } else {
            final com.google.api.services.drive.model.File oldFile = googleCloudFile;
            return null != updateFile(drive, oldFile.getId(), title, file);
        }
    }

    /**
     * Insert new file.
     *
     * @param service Drive API service instance.
     * @param title Title of the file to insert, including the extension.
     * @param parentId Optional parent folder's ID.
     * @param mimeType MIME type of the file to insert.
     * @param filename Filename of the file to insert.
     * @return Inserted file metadata if successful, {@code null} otherwise.
     */
    private static com.google.api.services.drive.model.File insertFile(Drive service, String title, String parentId, java.io.File fileContent) {
        // File's metadata.
        com.google.api.services.drive.model.File body = new com.google.api.services.drive.model.File();
        body.setTitle(title);

        // Set the parent folder.
        if (parentId != null && parentId.length() > 0) {
            body.setParents(
                Arrays.asList(new ParentReference().setId(parentId)));
        }

        // File's content.
        FileContent mediaContent = new FileContent("", fileContent);
        try {
            com.google.api.services.drive.model.File file = service.files().insert(body, mediaContent).execute();
            return file;
        } catch (IOException e) {
            log.error(null, e);
            return null;
        }
    }

    /**
     * Update an existing file's metadata and content.
     *
     * @param service Drive API service instance.
     * @param fileId ID of the file to update.
     * @param newTitle New title for the file.
     * @param newFilename Filename of the new content to upload.
     * @return Updated file metadata if successful, {@code null} otherwise.
     */
    private static com.google.api.services.drive.model.File updateFile(Drive service, String fileId, String newTitle, java.io.File fileContent) {
        try {
            // First retrieve the file from the API.
            com.google.api.services.drive.model.File file = service.files().get(fileId).execute();

            // File's new metadata.
            file.setTitle(newTitle);

            FileContent mediaContent = new FileContent("", fileContent);

            // Send the request to the API.
            com.google.api.services.drive.model.File updatedFile = service.files().update(fileId, file, mediaContent).setNewRevision(false).execute();

            return updatedFile;
        } catch (IOException e) {
            log.error(null, e);
            return null;
        }
    }

  private static String CLIENT_ID = "CLIENT_ID";
  private static String CLIENT_SECRET = "CLIENT_SECRET";

  private static String REDIRECT_URI = "urn:ietf:wg:oauth:2.0:oob";

  public static void main(String[] args) throws IOException {   
    GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
        httpTransport, JSON_FACTORY, CLIENT_ID, CLIENT_SECRET, Arrays.asList(DriveScopes.DRIVE_APPDATA, DriveScopes.DRIVE))
        .setAccessType("online")
        .setApprovalPrompt("auto").build();

    String url = flow.newAuthorizationUrl().setRedirectUri(REDIRECT_URI).build();
    System.out.println("Please open the following URL in your browser then type the authorization code:");
    System.out.println("  " + url);
    BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
    String code = br.readLine();

    GoogleTokenResponse response = flow.newTokenRequest(code).setRedirectUri(REDIRECT_URI).execute();
    GoogleCredential credential = new GoogleCredential().setFromTokenResponse(response);

    java.io.File fileContent = new java.io.File("document.txt");
    saveToGoogleDrive(credential, fileContent);
  }

    private static final GsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();

    /** Global instance of the HTTP transport. */
    private static HttpTransport httpTransport;


    private static final Log log = LogFactory.getLog(Insert.class);

    static {
        try {
            // initialize the transport
            httpTransport = GoogleNetHttpTransport.newTrustedTransport();

        } catch (IOException ex) {
            log.error(null, ex);
        } catch (GeneralSecurityException ex) {
            log.error(null, ex);
        }
    }

}

Update on 1 January 2016

This problem seems gone. I guess Google Drive team had fixed it.

Answer 1

Note: please do not treat this as an "official answer from Google". Although I work at Google, I don't work on the Drive API.

I've reproduced the problem, and reported it to the Drive API team, who may be able to provide more details. In the meantime, one workaround I've found is to remove the

setNewRevision(false)

part of your update call on line 1414. That's not an ideal workaround as it means you'll get a new revision for each update, which will use up storage quota. However, it does seem to avoid the problem you've seen.