Multer-s3 Uploading empty files on mobile

Question

I'm using angular and multer-s3 to upload files from an angular app to a node server. Everything works well on the desktop but for some reason when trying to upload the photo via my iPhone 7 the uploaded file is corrupt. I'm using the same image and running through the same flow on both devices but getting different results so I'm assuming its because of mobile?

Here's the alert I get when trying to open the S3 file on the mobile

The file “1519398514215-test.png” could not be opened because it is empty.

Here's my code

var aws = require('aws-sdk');
var path = require('path');
var path3 = path.join(__dirname, "../config/config-aws.json");
var multer = require('multer');
var multerS3 = require('multer-s3');
var request = require('request');

aws.config.loadFromPath(path3);
var s3 = new aws.S3();
var fileName = '';
var uploadM = multer({
  storage: multerS3({
    s3: s3,
    bucket: 'XXXX',
    acl: 'public-read',
    metadata: function (req, file, cb) {
      cb(null, {fieldName: file.fieldname + '.png'});
    },
    key: function (req, file, cb) {
      fileName =  Date.now().toString() + "-" + file.originalname + '.png' ;
      cb(null, fileName)
    }
  })
});



router.post('/', uploadM.array('photos', 3), function(req,res) {
  if (res.error) {
    console.log(error.stack);
    return res.status(400).json({
      message: "Error",
      error: res.error
    });
  }
  const url = 'https://s3-us-west-2.amazonaws.com/XXXX/' + fileName;
  return res.status(200).json({
    fileName: url
  });
});

And here's my client-side

sendImage() {
  const formData: FormData = new FormData();

  this.removeObjectFromCanvas('polygon');


  if (!fabric.Canvas.supports('toDataURL')) {
    alert('This browser doesn\'t provide means to serialize canvas to an image');
  } else {
    // window.open(this.canvas.toDataURL('png'));
    const image = new Image();
    image.src = this.canvas.toDataURL('png');



        const blob = this.dataURItoBlob(image.src);

        const file = new File([blob], 'test.png');


        formData.append('photos', file, 'test');



        this.postFile(formData);
      }


    }

    postFile(file) {
      this.fileService.post(file)
      .subscribe(data => {

      }, error => {
        console.log(error);
      });
    }

UPDATE **********

So found out you can debug on mobile. It looks like the buffer I am sending has data in it. My first thought was the buffer was not sending.

**** Update Still can't figure this out. I've done some research and its possible it has something to do with formData and append? But as you can see by the image above both seem to be fine. Will continue to research ...

***** UPDATE Definitely uploading empty files. But its only on mobile?

Also, I checked the formData prior to sending to the node server, seems to have the correct data in it.

*** UPDATE

Ok, even weirder experience. It seems multer-s3 is uploading empty files but when I take the file on the server-side and return it to the client-side, then read that file and display it, the image is displayed perfectly. So the formData is not the issue, it's something with multer-s3 I'm assuming?

****UPDATE I forgot to mention I am using fabricjs and getting the image from the canvas. I read in some places there may be an issue there but like I said above when I send the file to the server and send it back to the client, after reading the file it displays the image perfectly.

****Update I tried adding contentType to the multer method and now I'm receiving a 503 service unavailable error when running on mobile only. For desktop it is fine.

aws.config.loadFromPath(path3);
var file1;
var s3 = new aws.S3();
var fileName = '';
var uploadM = multer({
  storage: multerS3({
    s3: s3,
    bucket: 'rent-z',
    acl: 'public-read',
    contentType: function (req, file, cb) {
      cb(null, 'image/png');
    },
    metadata: function (req, file, cb) {
      console.log(file);
      cb(null, {fieldName: file.fieldname});
    },
    key: function (req, file, cb) {
      fileName =  Date.now().toString() + "-" + file.originalname;
      file1 = file;
      cb(null, fileName)
    }
  })
}).array('photos', 1);



router.post('/', function(req,res) {
  uploadM(req, res, function (err) {
    if (err) {
      console.log(err);
      return res.status(400).json({
        message: "Error uploading to multer",
        error: err
      });
    }
  console.log('worked');

  if (res.error) {
    console.log(error.stack);
    return res.status(400).json({
      message: "Error",
      error: res.error
    });
  }
  // fs.readFile(req.body, function (err, data) { 
  const url = 'https://s3-us-west-2.amazonaws.com/rent-z/' + fileName;
  return res.status(200).json({
    fileName: url
  });
  // });
  })

});

I even tried running multer-s3's automatic find mime-type function and that did give the same result

**** Day 4

It's been 96 hours since I started debugging this problem. No progress has been made. Still trying to figure out why its working on desktop and not mobile. For anyone looking for a quick summary of behavior:

1. User uploads image on the desktop

2. User places image on canvas

3. Use scales image

4. User presses sendImage

5. This converts the image to dataUri then blob

6. This blob is added to a file which is appended to formData

7. This formData is sent to a nodejs server where multer-s3 middleware uploads the file to s3 successfully

8. User tries on mobile

9. Fails at step 7. The file is uploaded but is empty.

Let me know if anyone has any ideas on how to continue.

Answer 1

I'll make this an "official" answer since this may work for your needs. Anytime I have an intricate issue like this, my first thought is often "I wonder if there is an API/SaaS/service out there that can abstract this for me." As you've found, file uploads are tricky, particularly when you start throwing in the myriad devices we have to deal with these days.

I won't mention any particular services, but googling "file upload saas" will generally get you the top industry players. For $25 - $50/month you can abstract file uploads to a very simple api call. Not only do you get time savings now, but (assuming you choose a solid provider) you get no more headaches regarding file uploads in the future. It's the SaaS's job to make sure file uploads work on a million different devices; it's the SaaS's job to make sure S3 integration works, even when S3's api changes; it's the SaaS's job to make sure the user sees a nice friendly message if their upload fails for some reason, etc. I get to spend my time building features for our app instead of worrying about whether or not file uploads work on the iPhone 47.

"But then I'm tied to a SaaS, and live at the whim of their prices and feature set" Ah, but you can minimize that problem. For many services we use, I like to make a wrapper/interface/whatever you'd like to call it. In the case of file uploads, I made an ES6 module: fileUploads.js

In this module, I have a method upload. What does this method do? It simply implements and abstracts the API of [fileupload SaaS X]. If in the future we want, or need, to change from SaaS X to SaaS Y, I only have to change one thing in our entire app: my fileUpload.js module.