uploading, processing, storing and delivering user-provided files and images

Question

Later or earlier in a web developers' life you'll have to deal with files and images uploaded by users.

Common questions:

How do I upload?

modern web browsers and techniques allow several ways to upload user-provided files to my server. What are best practises and what do I have to consider?

How do I process?

Once the upload is finished what do I need to know about security, and further processing of the files

How do I store and deliver?

Are there best practises on how to store the uploaded files?

disclaimer: I've put almost 30 minutes into the following answer, the initial question was removed so I decided to ask general questions a user may face when dealing with user-provided files. See my answer, you are free to contribute and add your ideas and experience

Answer 1

There is no "best" way for each of those steps, but a whole bunch of established ways and best practises among a variety of techniques and libraries which can help you to achieve user friendly uploads, security and speed.

For another question which was unfortunately deleted, I've written this small guide.

A user-contributed file or image walks through several stages in it's lifetime:

1. the upload itself
2. validaton
3. processing for storage (e.g. type conversion,resizing,thumbnail generation,...)
4. storing
5. delivery on the frontend side

This will adress all of the steps and provide examples as far as possible

1. The Upload

Modern web browsers offer several methods to upload a file.

The simplest way is to create a form with simple <input type="file" name="userFile" /> tag. Once the form is submitted, the file will be sent using POST and can be accessed with the $_FILES superglobal in PHP

1.1. Simple form in HTML

<form action="upload.php" method="post" enctype="multipart/form-data">
    <input type="file" name="userFile1" />
    <input type="file" name="userFile2" />
    <input type="submit" value="upload files" />
</form>

When form is submitted, a POST request is sent to upload.php This kind of upload isn't very userfriendly. Your user will not see any progress on the upload, and uploading several files will be a pain

1.2. Javascript ajax upload

In this solution, the POST is done asynchronously with javascript

http://www.saaraan.com/2012/05/ajax-image-upload-with-progressbar-with-jquery-and-php

1.3. HTML 5 Upload

modern browsers support HTML5, which allows very nice and userfriendly upload masks with progress bar and preview - border is your own creativity

For example, I really like this demo: http://html5demos.com/dnd-upload

On the server side

No matter which of those upload techniques you decide to use, the upload will end up in a HTTP POST sent to your script. All further processing will be done on the server.

PHP copies the file into a temporary path (upload_tmp_dir)

There are a few other upload-related settings you can tweak in your php.ini or on-the-fly with ini_set: http://php.net/manual/en/ini.core.php#ini.sect.file-uploads

upload.php

<pre>
<?php
print_r( $_FILES );

This is how the $_FILES superglobal looks like after a successful upload.

• name: the original name
• type: the browser-provided mime-type
• size: the size in bytes
• tmp_name: temporary filename on your server
• error: error codes as described here, 0 when everything went file

Each file has it's own index

Array
( 
    [userFile1] => Array
    (
         [name] => i_love_ponies.png
         [type] => image/png
         [size] => 42233
         [tmp_name] => /tmp/_x123tfsdayx134
         [error] => 0
    )
    [userFile2] => Array
    (
         [name] => ponies_run_my_server.png
         [type] => image/png
         [size] => 12325
         [tmp_name] => /tmp/_x3123asdad3ssy
         [error] => 0
    )
)

A few additional hints

If you are expecting large files, consider webserver and php execution timeouts. If an upload takes 10minutes you don't want your users to end up in an error.

In PHP you can increase the max_execution_time

2. Validation

Questions you might want to ask

Do I want to allow only specific files - what kind of file do I receive?

the type you find in the $_FILES array is set by the browser. If you want to be sure which type was uploaded you can use the fileinfo extension of PHP.

This example would check the success of the upload itself and write files of the allowed types into a fresh array for furter processing

$allowedTypes = array(
   'image/png'
   'image/gif'
   'image/jpeg'
);

$useFiles = array();

$finfo = new finfo(FILEINFO_MIME_TYPE);
foreach ( $_FILES as $index => $file )
{
    if ( $file['error'] === UPLOAD_ERR_OK ) {
        $type = $finfo->file( $file['tmp_name'] );
        if ( in_array( $type, $allowedTypes ) ) {
             $useFiles[$index] = $file;
        }
    }
}

Do I want to set a maximum filesize - what is the size?

In this case you can safely rely on the $_FILES['key']['size'] value, which is the size in bytes. You should not just rely on the client-side set file size limitations

if talking about images, do I want to scale or create thumbnails - what are the image dimensions?

Have a look at getimagesize to determine image dimensions

3. Processing

Before saving the file to it's final location you might want to do some processing with it. That's the time where you can

• change image size for later displaying
• create thumbnail
• add watermark or branding
• convert into another type. no link here because this can be any type, I am sure you find a solution to convert your .xxx to .yyy :)

Common libraries used for image processing are the gd lib and imagick. personally I prefer the gd lib. It is a bit more manual work, but it is faster and comes with most installations by default or is simple to install additionally. Imagemagick natively has a very huge pool of image processing functions

4. Storing your data

Now it is time to talk about storage. First a save way to copy your temporary file to a temporary or final location.

You should use move_uploaded_file

move_uploaded_file( $useFiles['tmp_name'], $destination );

Again, there's no "best way" for storing, this depends on your environment, files and use. I will talk about a few

4.1. server file system

This is probably the most used and simplest way to store your files. You can simply serve files statically with your already running webserver. This is usually a location within your document root.

To make your life easier, you can save the filename into a database - preferrably referencing or referenced by connected data like the user or a location.

You should not save all files into the same folder for various reasons.

You'd have to generate a unique name for each file, otherwise you overwrite an existing file with a new one. Also usual filesystems are getting slower to seek with increasing amounts of files in a node. This will result in slow response times when a file is requested for delivery.

You could save it into a folder for each user

/uploaded/user-id/file.jpg

If you expect a lot of files per user you could split a checksum of the filname to get a deeper folder structure

e.g.

$path =  $_SERVER['DOCUMENT_ROOT'] . '/uploaded/' . $userId . '/' . chunk_split( md5( $useFiles['index']['name'] ), 12, '/' );

would result in

/var/www/htdocs/uploaded/1/97e262286853/d52aa8c58eb7/f0431f03/

4.2. Database

You could store your files into a MySQL database using the blob type. I don't recommend this generally

4.3. Content delivery network

If you expect a lot of traffic all over the globe, you might want to consider storing your files on a CDN like Amazon S3 with Cloudfront.

S3 is a cheap, reliable storage for files up to 5TB (as far as I know)

http://www.9lessons.info/2012/08/upload-files-to-amazon-s3-php.html

You will not have to worry about file backups, available harddisk size, delivery speed etc. etc. Cloudfront adds a CDN layer on top of S3 with edge locations all over the world