Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

1MB quota limit for a blobstore object in Google App Engine?

I'm using App Engine (version 1.4.3) direct write the blobstore in order to save images. when I try to store an image which is larger than 1MB I get the following Exception

com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large.

I thought that the limit for each object is 2GB

Here is the Java code that stores the image

private void putInBlobStore(final String mimeType, final byte[] data) throws IOException {
    final FileService fileService = FileServiceFactory.getFileService();
    final AppEngineFile file = fileService.createNewBlobFile(mimeType);
    final FileWriteChannel writeChannel = fileService.openWriteChannel(file, true);
    writeChannel.write(ByteBuffer.wrap(data));
    writeChannel.closeFinally();
}
like image 621
Yonatan Maman Avatar asked Apr 02 '11 11:04

Yonatan Maman


1 Answers

Here is how I read and write large files:

public byte[] readImageData(BlobKey blobKey, long blobSize) {
    BlobstoreService blobStoreService = BlobstoreServiceFactory
            .getBlobstoreService();
    byte[] allTheBytes = new byte[0];
    long amountLeftToRead = blobSize;
    long startIndex = 0;
    while (amountLeftToRead > 0) {
        long amountToReadNow = Math.min(
                BlobstoreService.MAX_BLOB_FETCH_SIZE - 1, amountLeftToRead);

        byte[] chunkOfBytes = blobStoreService.fetchData(blobKey,
                startIndex, startIndex + amountToReadNow - 1);

        allTheBytes = ArrayUtils.addAll(allTheBytes, chunkOfBytes);

        amountLeftToRead -= amountToReadNow;
        startIndex += amountToReadNow;
    }

    return allTheBytes;
}

public BlobKey writeImageData(byte[] bytes) throws IOException {
    FileService fileService = FileServiceFactory.getFileService();

    AppEngineFile file = fileService.createNewBlobFile("image/jpeg");
    boolean lock = true;
    FileWriteChannel writeChannel = fileService
            .openWriteChannel(file, lock);

    writeChannel.write(ByteBuffer.wrap(bytes));
    writeChannel.closeFinally();

    return fileService.getBlobKey(file);
}
like image 62
matt burns Avatar answered Sep 20 '22 07:09

matt burns