AWS PHP SDK: Limit S3 file upload size in presigned URL

Question

I'm working on a project involving generating S3 URLs that someone else can use to upload files to my S3 bucket. Here's a minimal working example:

<?php

    require('aws.phar');
    use Aws\S3\S3Client;

    $s3client = new S3Client(...); // credentials go here

    $id = uniqid(); // generate some kind of key

    $command = $s3client->getCommand('PutObject', [
        'ACL' => 'private',
        'Body' => '',
        'Bucket' => 'mybucket',
        'Key' => 'tmp/' . $id]);

    echo (string) $s3client->createPresignedRequest($command, '+5 minutes')->getURI();

?>

Now, if I put that file at a location accessible by the internet, my web server can be used to fetch new signed upload URLs:

$ curl http://my.domain.com/some/page.php
https://s3.amazonaws.com/mybucket/tmp/someID?x-amz-acl=private&lots-of-aws-params...
$ curl -X PUT -d "@someFile" https://s3.amazonaws.com/mybucket/tmp/someID?x-amz-acl=private&lots-of-aws-params...
$

This successfully uploads a local file to my bucket, so I can play with it in S3.

Let's suppose that I'm not too worried about people generating many URLs and uploading many files to my bucket in a short period of time, but I would like to limit the size of uploaded files. Many resources suggest attaching a policy to the signed URL:

<?php

    require('aws.phar');
    use Aws\S3\S3Client;

    $s3client = new S3Client(...); // credentials go here

    $id = uniqid(); // generate some kind of key

    $policy = [
        'conditions' => [
            ['acl' => 'private'],
            ['bucket' => 'mybucket'],
            ['content-length-range', 0, 8*1024], // 8 KiB
            ['starts-with', '$key', 'tmp/']
        ], 'expiration' =>
            (new DateTime())->modify('+5 minutes')->format(DateTime::ATOM)];

    $command = $s3client->getCommand('PutObject', [
        'ACL' => 'private',
        'Body' => '',
        'Bucket' => 'mybucket',
        'Key' => 'tmp/' . $id,
        'Policy' => $policy]);

    echo (string) $s3client->createPresignedRequest($command, '+5 minutes')->getURI();

?>

This version generates URLS (without any indication of errors) that can be used in the same way. I'm not sure if I need some of those conditions in the policy (acl, bucket, starts-with), but I don't think that including them would break the policy.

In theory, attempting to use this signed URL to upload a file larger than 8 KiB should cause S3 to abort the upload. However, testing this with a larger file shows that curl still happily uploads the file:

$ ls -lh file.txt
-rw-rw-r-- 1 millinon millinon 210K Jan  2 00:41 file.txt
$ curl http://my.domain.com/some/page.php
https://s3.amazonaws.com/mybucket/tmp/someOtherID?x-amz-acl=private&lots-of-aws-params...
$ curl -X PUT -d "@file.txt" https://s3.amazonaws.com/mybucket/tmp/someOtherID?x-amz-acl=private&lots-of-aws-params...
$

Checking the bucket shows that, indeed, the large file was uploaded, and the file's size is larger than the policy supposedly indicates.

Since various pages show different ways of attaching the policy, I have also tried the following versions:

'Policy' => json_encode($policy)
'Policy' => base64_encode(json_encode($policy))

However, URLs generated with any of these versions allow files larger than the specified size to be uploaded.

Am I attaching the policy incorrectly, or is there a fundamental limitation to restricting uploads to S3 in this manner?

For my web server, I'm using HHVM 3.11.1 with version 3.14.1 of the AWS SDK for PHP.

Answer 1

An S3 upload policy cannot be used with pre-signed URLs.

A policy document can be used with browser uploads to S3 using HTML POST Forms.

Pre-signed URLs and HTML POST forms are two different methods of uploading to S3. The former is arguably simpler, but less flexible, than the latter.

UPDATE

If you must upload the files without the use of a browser, the HTML POST Form's request can be reproduced using PHP's curl functions, a library such as Guzzle, or using the command line as follows:

curl 'https://s3-bucket.s3.amazonaws.com/' \
    -F 'key=uploads/${filename}' \
    -F 'AWSAccessKeyId=YOUR_AWS_ACCESS_KEY' \
    -F 'acl=private' \
    -F 'success_action_redirect=http://localhost/' \
    -F 'policy=YOUR_POLICY_DOCUMENT_BASE64_ENCODED' \
    -F 'signature=YOUR_CALCULATED_SIGNATURE' \
    -F 'Content-Type=image/jpeg' \
    -F '[email protected]'

Answer 2

can you try to add the specific policy to your bucket

$s3client->putBucketPolicy(array(
    'Bucket' => $bucket,
    'Policy' => json_encode(array(
        'conditions' => array(
            array(
                'content-length-range', 0, 8*1024,
                'starts-with', '$key', 'tmp/'
                ...

and after this, just use the normal putObject command

$command = $s3client->getCommand('PutObject', [
    'ACL' => 'private',
    'Body' => '',
    'Bucket' => 'mybucket',
    'Key' => 'tmp/' . $id]);