Google Cloud Storage throwing error for multiple large file uploads concurrently

Question

We are trying to upload multiple files i.e. 100 or more than that to Google Cloud Storage, but we get the below error randomly.
Average File Size: 30MB - 40 MB

 08:57:15.488 [ForkJoinPool-1-worker-26] INFO  
 demo.storage.GCPStorageComponent - Error writing request body to
 server com.google.cloud.storage.StorageException: Error writing
 request body to server
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:189)
     at com.google.cloud.storage.spi.v1.HttpStorageRpc.write(HttpStorageRpc.java:581)
     at com.google.cloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:51)
     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
     at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.BlobWriteChannel.flushBuffer(BlobWriteChannel.java:47)
     at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:160)
     at demo.storage.GCPStorageComponent.uploadFile(GCPStorageComponent.java:56)
     at demo.storage.StorageRepository.uploadFile(StorageRepository.java:56)
     at demo.restful.UploadRepository.uploadFileToBucket(UploadRepository.java:132)
     at demo.restful.UploadRepository.processClipResults(UploadRepository.java:56)
     at demo.restful.UploadRepository.lambda$getClipModelCallableList$0(UploadRepository.java:85)
     at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
     at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
     at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
     at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
     at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
 Caused by: java.io.IOException: Error writing request body to server
     at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3536)
     at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3519)
     at com.google.api.client.util.ByteStreams.copy(ByteStreams.java:55)
     at com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
     at com.google.api.client.http.AbstractInputStreamContent.writeTo(AbstractInputStreamContent.java:72)
     at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:80)
     at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
     at com.google.cloud.storage.spi.v1.HttpStorageRpc.write(HttpStorageRpc.java:563)
     ... 17 more

Below is our code, which we are using to upload files. We are uploading files in chunk of 1 MB using multi-part approach.

public void uploadFile(String bucketName, String bucketFileName, Path localFilePath)
        throws IOException, StorageException {
    BlobInfo blobInfo = BlobInfo.newBuilder(bucketName, bucketFileName).build();
    ByteBuffer buffer = ByteBuffer.allocate(10_000_000); //i.e. 1 MB
    try (WriteChannel writer = storage.writer(blobInfo);
            ByteChannel inChannel = Files.newByteChannel(localFilePath);) { // write to bucket
        while (inChannel.read(buffer) > 0) {
            buffer.flip();
            writer.write(buffer);
            buffer.clear();
        }
    } catch (com.google.cloud.storage.StorageException ex) {
        LOGGER.info(ex.getMessage());
        ex.printStackTrace();
    }
}

Edit: Access is being done from within Google Cloud
An Github Issue has been opened for same

Answer 1

Closing this issue as a Bug fix is done for this error and has merged into the latest version of google cloud java.

Here is the commit link https://github.com/GoogleCloudPlatform/google-cloud-java/pull/3433

Link for source code changes https://github.com/GoogleCloudPlatform/google-cloud-java/pull/3433/commits/8e20b1c41e1ae99c4761f716c1ab4dc0f9fbf5e0