Chrome FileReader returns empty string for big files (= 300MB)

Question

Goal:

In the browser, read a file from the users file system as base64 string
These files are up to 1.5GB

Issue:

The followig script works perfectly fine on Firefox. Regardless of the filesize.
On Chrome, the script works fine for smaller files (I've tested files of ~ 5MB size)
If you pick a bigger file (e.g. 400MB) the FileReader completes without an error or exception, but returns an empty string instead of the base64 string

Questions:

Is this a chrome bug?
Why is there neither an error nor an exception?
How can I fix or work around this issue?

Important:

Please note, that chunking is not an option for me, since I need to send the full base64 string via 'POST' to an API that does not support chunks.

Code:

'use strict';

var filePickerElement = document.getElementById('filepicker');

filePickerElement.onchange = (event) => {
  const selectedFile = event.target.files[0];
  console.log('selectedFile', selectedFile);

  readFile(selectedFile);
};

function readFile(selectedFile) {
  console.log('START READING FILE');
  const reader = new FileReader();

  reader.onload = (e) => {
    const fileBase64 = reader.result.toString();

    console.log('ONLOAD','base64', fileBase64);
    
    if (fileBase64 === '') {
      alert('Result string is EMPTY :(');
    } else {
        alert('It worked as expected :)');
    }
  };

  reader.onprogress = (e) => {
    console.log('Progress', ~~((e.loaded / e.total) * 100 ), '%');
  };

  reader.onerror = (err) => {
    console.error('Error reading the file.', err);
  };

  reader.readAsDataURL(selectedFile);
}

<!doctype html>
<html lang="en">

<head>
  <!-- Required meta tags -->
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">

  <!-- Bootstrap CSS -->
  <link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet"
    integrity="sha384-wEmeIV1mKuiNpC+IOBjI7aAzPcEZeedi5yW5f2yOq55WWLwNGmvvx4Um1vskeMj0" crossorigin="anonymous">

  <title>FileReader issue example</title>
</head>

<body>

  <div class="container">
    <h1>FileReader issue example</h1>
    <div class="card">
      <div class="card-header">
        Select File:
      </div>
      <div class="card-body">
        <input type="file" id="filepicker" />
      </div>
    </div>

  </div>

  <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.bundle.min.js"
    integrity="sha384-p34f1UUtsS3wqzfto5wAAmdvj+osOnFyQFpp4Ua3gs/ZVWx6oOypYoCJhGGScy+8"
    crossorigin="anonymous"></script>
  <script src="main.js"></script>
</body>

</html>

Kaiido · Accepted Answer

Is this a chrome bug?

As I said in my answer to Chrome, FileReader API, event.target.result === "", this a V8 (Chrome's but also node-js's and others' JavaScript JS engine) limitation.
It is intentional and thus can't really qualify as "a bug".
The technicalities are that what actually fails here is to build a String of more than 512MB (less the header) on 64bit systems because in V8 all heap objects must fit in a Smi (Small Integer), (cf this commit).

Why is there neither an error nor an exception?

That, might be a bug... As I also show in my linked answer, we get a RangeError when creating such a string directly:

const header = 24;
const bytes = new Uint8Array( (512 * 1024 * 1024) - header );
let txt = new TextDecoder().decode( bytes );
console.log( txt.length ); // 536870888
txt += "f"; // RangeError

And in the step 3 of FileReader::readOperation, UAs have to

If package data threw an exception error:

Set fr’s error to error.

Fire a progress event called error at fr.

But here, we don't have that error.

const bytes = Uint32Array.from( { length: 600 * 1024 * 1024 / 4 }, (_) => Math.random() * 0xFFFFFFFF );
const blob = new Blob( [ bytes ] );
const fr = new FileReader();
fr.onerror = console.error;
fr.onload = (evt) => console.log( "success", fr.result.length, fr.error );
fr.readAsDataURL( blob );

I will open an issue about this, since you should be able to handle that error from the FileReader.

How can I fix or work around this issue?

The best is definitely to make your API end-point accepts binary resources directly instead of data:// URLs, which should always be avoided anyway.

If this is not doable, a solution "for the future", will be to POST a ReadableStream to your end-point, and do the data:// URL conversion yourself, on a stream from the Blob.

class base64StreamEncoder {
  constructor( header ) {
    if( header ) {
      this.header = new TextEncoder().encode( header );
    }
    this.tail = [];
  }
  transform( chunk, controller ) {
    const encoded = this.encode( chunk );
    if( this.header ) {
      controller.enqueue( this.header );
      this.header = null;
    }
    controller.enqueue( encoded );
  }
  encode( bytes ) {
    let binary = Array.from( this.tail )
        .reduce( (bin, byte) => bin + String.fromCharCode( byte ), "" );
    const tail_length = bytes.length % 3;
    const last_index = bytes.length - tail_length;
    this.tail = bytes.subarray( last_index );
    for( let i = 0; i<last_index; i++ ) {
        binary += String.fromCharCode( bytes[ i ] );
    }
    const b64String = window.btoa( binary );
    return new TextEncoder().encode( b64String );
  }
  flush( controller ) {
    // force the encoding of the tail
    controller.enqueue( this.encode( new Uint8Array() ) );
  }
}

Live example: https://base64streamencoder.glitch.me/

For now, you'd have to store chunks of the base64 representation in a Blob as demonstrated by Endless's answer.

However beware that since this is a V8 limitation, even the server-side can face issues with strings this big, so anyway, you should contact your API's maintainer.

Chrome FileReader returns empty string for big files (>= 300MB)

Tags:

javascript

google-chrome

filereader

tmuecksch

1 Answers

Kaiido

Recent Activity

Donate For Us