upload file from angularjs directly to amazon s3 using signed url

Question

So I am having some trouble uploading a file directly to S3. Currently my process is to make a request to nodejs/express to get a signed URL.

app.post('/s3SignedURL', function(req, res){
  var id = crypto.randomBytes(20).toString('hex');
  var ext = path.extname(req.body.fileName);
  var unambFilename = path.basename(req.body.fileName, ext) + '-' + id + ext;
  var params = {Bucket: awsBucket, Key: unambFilename, Expires: 30};
  var signedUrl = s3.getSignedUrl('putObject', params);

  res.send({signedUrl: signedUrl, s3FileName: unambFilename});
});

My angular controller then tries to upload directly to s3 using that signed URL ($scope.uploadDocument())

flqApp.controller('DocUploadModalCtrl', ['$scope', '$http', 'customProvider', 'custom',
  function($scope, $http, customProvider, custom){

  $scope.fileTypes = 
  [
    "Type 1",
    "Type 2"
  ]

  $scope.setFile = function(element){
    $scope.$apply(function($scope){
      $scope.currentDocument = element.files[0];
    });
  }

  $scope.uploadDocument = function() {
    $http.post('/s3SignedURL', {fileName: $scope.currentDocument.name} )
     .success(function(results){
      $http.put(results.signedUrl, $scope.currentDocument)
       .success(function(){
        custom.document = s3FileName;
        customProvider.save(custom, function(){
        //..do something here
        });
      });
    });
  };
}]);

My html form looks like

<form ng-submit="uploadDocument()">
  <label for="documentType">File Type</label>
  <select class="form-control" ng-model="docType" ng-options="type for type in fileTypes" required >
    <option value=""/>
  </select>
  <label for="filename">Choose file to upload</label>
  <input type="file"
     name="s3File"
     onchange="angular.element(this).scope().setFile(this)"
     ng-model="fileName"
     required />

  <input type="submit" value="Upload File">
</form>

However whenever I try to upload to S3 I get the error

Origin http://localhost:3000 is not allowed by Access-Control-Allow-Origin

I know that S3 CORS is setup correctly, on the amazon end, for that bucket, because I have developed ruby apps that use the same bucket for development storage. (granted I was using paperclip & fog for those). Secondly, since I don't have a failure catch for the amazon response, I don't suspect the error to be coming from there. However it does come from the line where I try to put the file on amazon.

So I am sure I am missing something, but I thought that with signed URL's I don't need anything more than to do a put to that url.

Answer 1

I have been struggling a lot with this issue and finally got it figured out! I will detail my steps, hopefully it can help some one out.

I used this module: https://github.com/asafdav/ng-s3upload

I followed the steps they listed, namely:

1. Create a Bucket
2. Grant "put/Delete: expand the "Permissions" sections and click on the "Add more permissions" button. Select "Everyone" and "Upload/Delete" and save.

3. Add CORS Configuration:

   <?xml version="1.0" encoding="UTF-8"?>
   <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
       <CORSRule>
           <AllowedOrigin>*</AllowedOrigin>
           <AllowedMethod>GET</AllowedMethod>
           <AllowedMethod>POST</AllowedMethod>
           <AllowedMethod>PUT</AllowedMethod>
           <AllowedHeader>*</AllowedHeader>
       </CORSRule>
   

4. Add "crossdomain.xml" to the root of your bucket making it public

   <?xml version="1.0"?>
   <!DOCTYPE cross-domain-policy SYSTEM
   "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
   <cross-domain-policy>
     <allow-access-from domain="*" secure="false" />
   </cross-domain-policy>
   

5. Create a service that will return JSON with the following:

   {
      "policy":"XXX",
      "signature":"YYY",
      "key":"ZZZ"
   }
   

• XXX - A policy json that is required by AWS, base64 encoded.
• YYY - HMAC and sha of your private key
• ZZZ - Your public key Here's a rails example, even if you're not a rails developer, read the code, it's very straight forward.

This is the most important step: make sure you are generating the correct policy document.

Here is my code in C#

            StringBuilder builder = new StringBuilder();
        builder.Append("{")
                .Append("\"expiration\": \"")
                .Append(GetFormattedTimestamp(expireInMinutes))
                .Append("\",")
                .Append("\"conditions\": [")
                .Append("{\"bucket\": \"")
                .Append(bucketName)
                .Append("\"},")
                .Append("{\"acl\": \"")
                .Append("public-read")
                .Append("\"},")
                .Append("[\"starts-with\", \"$key\", \"")
                .Append(prefix)
                .Append("\"],")
                .Append("[\"starts-with\", \"$Content-Type\", \"\"],")                    
                .Append("[ \"content-length-range\", 0, " + 10 * 1024 * 1024 + "]")
                .Append("]}");
        Encoding encoding = new UTF8Encoding();
        this.policyString = Convert.ToBase64String(encoding.GetBytes(builder.ToString().ToCharArray()));
        this.policySignature = SignPolicy(awsSecretKey, policyString);

This generates the following Json

{
   "expiration":"2014-02-13T15:17:40.998Z",
   "conditions":[
      {
         "bucket":"bucketusaa"
      },
      {
         "acl":"public-read"
      },
      [
         "starts-with",
         "$key",
         ""
      ],
      [
         "starts-with",
         "$Content-Type",
         ""
      ],
      [
         "content-length-range",
         0,
         10485760
      ]
   ]
}

This document is then base64 encoded and sent down as a string.

My issue was with my policy document. The policy document is like a set of rules you define for the session like: file names must start with something (ie. upload to a subfolder), the size must be in the range.

Use the developer tools for your browser, and take a look at the network tab, see what errors AWS are returning this really helped me, it will state things like policy errors and say what condition failed. You will generally get access denied errors and this will be based on the conditions set in the policy document or wrong keys.

One other thing some browsers have issues with localhost CORS. But using the above I was able to upload files from my local dev machine using chrome.

Origin 'localhost:3000' is not allowed by Access-Control-Allow-Origin

From your error it looks like you have not set up the CORS rules on the AWS side.

Answer 2

This example can maybe help: https://github.com/bookingbricks/file-upload-example Using: Node, aws-sdk-js, jQuery-file-upload (blueimp)

Server:

var AWS = require('aws-sdk');

AWS.config.update({accessKeyId: AWS_ACCESS_KEY, secretAccessKey:     AWS_SECRET_KEY});
AWS.config.region = 'eu-west-1';

app.post('/s', function (req, res) {
    var s3 = new AWS.S3();
    var params = {Bucket: 'BUCKETNAME', Key: req.body.name, ContentType: req.body.type};
    s3.getSignedUrl('putObject', params, function(err, url) {
        if(err) console.log(err);
        res.json({url: url});
    });
});

Client:

$.ajax({
    url: '/s',
    type: 'POST',
    data: {name: file.name, size: file.size, type:file.type},
}).success(function(res){
    $.ajax({
        url: res.url,
        type: 'PUT',
        data: file,
        processData: false,
        contentType: file.type,
    }).success(function(res){
        console.log('Done');
    });