Read and base64 encode a binary file

Question

I'm trying to read a binary file from the filesystem and then base64 encode it in JavaScript. I'm using the FileReader API to read the data and the base64 encoder found here.

The code I have seems close to working, the problem is that the generated base64 data is wrong. Here's what I've got so far:

function saveResource() {
    var file = $(".resourceFile")[0].files[0];

    var reader = new FileReader();
    reader.onload = function(evt) {
        var fileData = evt.target.result;
        var bytes = new Uint8Array(fileData);
        var binaryText = '';

        for (var index = 0; index < bytes.byteLength; index++) {
            binaryText += String.fromCharCode( bytes[index] );
        }

        console.log(Base64.encode(binaryText));

    };
    reader.readAsArrayBuffer(file);
};

Here's the file I'm testing with (it's a 100x100 blue square):

According to an online base64 decoder/encoder, this file should encode to:

/9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCABkAGQDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDxyiiiv3E8wKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA//Z

...but instead what I'm getting out of the JavaScript is:

w7/DmMO/w6AAEEpGSUYAAQIAAAEAAQAAw7/DmwBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDLDv8ObAEMBCQkJDAsMGA0NGDIhHCEyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMsO/w4AAEQgAZABkAwEiAAIRAQMRAcO/w4QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoLw7/DhADCtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDLCgcKRwqEII0LCscOBFVLDkcOwJDNicsKCCQoWFxgZGiUmJygpKjQ1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5esKDwoTChcKGwofCiMKJworCksKTwpTClcKWwpfCmMKZwprCosKjwqTCpcKmwqfCqMKpwqrCssKzwrTCtcK2wrfCuMK5wrrDgsODw4TDhcOGw4fDiMOJw4rDksOTw5TDlcOWw5fDmMOZw5rDocOiw6PDpMOlw6bDp8Oow6nDqsOxw7LDs8O0w7XDtsO3w7jDucO6w7/DhAAfAQADAQEBAQEBAQEBAAAAAAAAAQIDBAUGBwgJCgvDv8OEAMK1EQACAQIEBAMEBwUEBAABAncAAQIDEQQFITEGEkFRB2FxEyIywoEIFELCkcKhwrHDgQkjM1LDsBVicsORChYkNMOhJcOxFxgZGiYnKCkqNTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXrCgsKDwoTChcKGwofCiMKJworCksKTwpTClcKWwpfCmMKZwprCosKjwqTCpcKmwqfCqMKpwqrCssKzwrTCtcK2wrfCuMK5wrrDgsODw4TDhcOGw4fDiMOJw4rDksOTw5TDlcOWw5fDmMOZw5rDosOjw6TDpcOmw6fDqMOpw6rDssOzw7TDtcO2w7fDuMO5w7rDv8OaAAwDAQACEQMRAD8Aw7HDiijCosK/cTzDgMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooAMKiwoooA8O/w5k=

If i had to hazard a guess I'd say that the issue has something to do with nonprintable characters in the binary data (if I encode a plaintext document, that works fine). But what's the best approach to work around the issue?

Edit

It looks like this may be a problem with the base64 library itself (or if not that, then with how the Uint8Array is unpackaged into a string for the library call). If I use the browser's btoa() function instead, and pass it the Uint8Array binaryText directly, that works. Too bad that function doesn't exist in all browsers.

Answer 1

And Google to the rescue. I found the following code, which takes the input data as a plain array of "bytes" (numbers between 0 and 255, inclusive; also works fine if the Uint8Array is passed to it directly), and added it to the library I was using:

//note:  it is assumed that the Base64 object has already been defined
//License:  Apache 2.0
Base64.byteToCharMap_ = null;
Base64.charToByteMap_ = null;
Base64.byteToCharMapWebSafe_ = null;
Base64.charToByteMapWebSafe_ = null;
Base64.ENCODED_VALS_BASE =
    'ABCDEFGHIJKLMNOPQRSTUVWXYZ' +
    'abcdefghijklmnopqrstuvwxyz' +
    '0123456789';

/**
 * Our default alphabet. Value 64 (=) is special; it means "nothing."
 * @type {string}
 */
Base64.ENCODED_VALS = Base64.ENCODED_VALS_BASE + '+/=';
Base64.ENCODED_VALS_WEBSAFE = Base64.ENCODED_VALS_BASE + '-_.';

/**
 * Base64-encode an array of bytes.
 *
 * @param {Array.<number>|Uint8Array} input An array of bytes (numbers with
 *     value in [0, 255]) to encode.
 * @param {boolean=} opt_webSafe Boolean indicating we should use the
 *     alternative alphabet.
 * @return {string} The base64 encoded string.
 */
Base64.encodeByteArray = function(input, opt_webSafe) {
  Base64.init_();

  var byteToCharMap = opt_webSafe ?
                      Base64.byteToCharMapWebSafe_ :
                      Base64.byteToCharMap_;

  var output = [];

  for (var i = 0; i < input.length; i += 3) {
    var byte1 = input[i];
    var haveByte2 = i + 1 < input.length;
    var byte2 = haveByte2 ? input[i + 1] : 0;
    var haveByte3 = i + 2 < input.length;
    var byte3 = haveByte3 ? input[i + 2] : 0;

    var outByte1 = byte1 >> 2;
    var outByte2 = ((byte1 & 0x03) << 4) | (byte2 >> 4);
    var outByte3 = ((byte2 & 0x0F) << 2) | (byte3 >> 6);
    var outByte4 = byte3 & 0x3F;

    if (!haveByte3) {
      outByte4 = 64;

      if (!haveByte2) {
        outByte3 = 64;
      }
    }

    output.push(byteToCharMap[outByte1],
                byteToCharMap[outByte2],
                byteToCharMap[outByte3],
                byteToCharMap[outByte4]);
  }

  return output.join('');
};

/**
 * Lazy static initialization function. Called before
 * accessing any of the static map variables.
 * @private
 */
Base64.init_ = function() {
  if (!Base64.byteToCharMap_) {
    Base64.byteToCharMap_ = {};
    Base64.charToByteMap_ = {};
    Base64.byteToCharMapWebSafe_ = {};
    Base64.charToByteMapWebSafe_ = {};

    // We want quick mappings back and forth, so we precompute two maps.
    for (var i = 0; i < Base64.ENCODED_VALS.length; i++) {
      Base64.byteToCharMap_[i] =
          Base64.ENCODED_VALS.charAt(i);
      Base64.charToByteMap_[Base64.byteToCharMap_[i]] = i;
      Base64.byteToCharMapWebSafe_[i] =
          Base64.ENCODED_VALS_WEBSAFE.charAt(i);
      Base64.charToByteMapWebSafe_[
          Base64.byteToCharMapWebSafe_[i]] = i;
    }
  }
};

The full code for the library containing the above functions is available here, but in its non-modified form it appears to depend upon a number of other libraries. The slightly hacked-up version above should work for anyone who just needs a quick fix for this issue.

Answer 2

Treat the binary as arraybuffer, this is independent from any character encoding. Your blue-square (.jpg) has 361 native bytes, means octets from 0..255 (decimal) and they are no characters!

It means: Use ArrayBuffer for encoding this to Base64 with well known base64-algorithm.

With Perl back to origin, shows the blue-square as above:

my $fh = IO::File->new;
$fh->open("d:/tmp/x.jpg", O_BINARY|O_CREAT|O_RDWR|O_TRUNC) or die $!;

$fh->print(decode_base64("/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBD
AQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCABkAGQDASIAAhEBAxEB/8QAFQABAQAA
AAAAAAAAAAAAAAAAAAf/xAAUEAEAAAAAAAAAAAAAAAAAAAAA/8QAFgEBAQEAAAAAAAAAAAAAAAAAAAUH/8QAFBEBAAAAAAAAAAAAAAAAAAAAAP/aAAwDAQACEQMR
AD8AjgDcUwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB//2Q==
"));


$fh->close;