Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to upload multiple file under 1 http request

Using HTML5 chunking, I could do file upload with smaller piece. But the problem starts when it started using multiple http POST request which will cause the computer slowing down, or probably crash. Is there anyway to have the splitted file under one http request.. so if I have 5 files it would be only 5 http request eventhough I use html5 split chunk

e.g: if I upload 5 files, each file will be split to 1mb chunk, so if first file is 10mb, then it will become 10 pieces of 1mb chunk. And the problem is, each chunk will be under 1 http request so just the first file it will be 10 HTTP request. Imagine if I have 1gb files, it will become 1000 HTTP request and slow down the computer.

This is example code:

        //Prepare element progress after the page load completely
        var uploaders = [];
        var totalChunks = 0;
        var progress;
        var bars;
        $(document).ready(function() {
            //progress = document.querySelector('progress');
            //bars = document.querySelector('#bars'); 
        });        

        //function for after the button is clicked, slice the file 
        //and call upload function
        function sendRequest() {       
            //clean the screen
            //bars.innerHTML = '';


            var file = document.getElementById('fileToUpload');   

            for(var i = 0; i < file.files.length; i++) {      
                var blob = file.files[i];         
                var originalFileName = blob.name;
                var filePart = 0

                const BYTES_PER_CHUNK = 10 * 1024 * 1024; // 10MB chunk sizes.
                const SIZE = blob.size;

                var start = 0;
                var end = BYTES_PER_CHUNK;

                totalChunks = Math.ceil(SIZE / BYTES_PER_CHUNK);

                while( start < SIZE ) {                    
                    if (blob.webkitSlice) {
                        //for Google Chrome
                        var chunk = blob.webkitSlice(start, end); 
                    } else if (blob.mozSlice) {
                        //for Mozilla Firefox
                        var chunk = blob.mozSlice(start, end);
                    }       

                    uploadFile(chunk, originalFileName, filePart, totalChunks, i);
                    filePart++;
                    start = end;
                    end = start + BYTES_PER_CHUNK;
                }
            }                
        }

        function uploadFile(blobFile, fileName) {
            var fd = new FormData();
            fd.append("fileToUpload", blobFile);

            var xm = $.ajax({
                url: "upload.php"+"?"+"file1="+fileName,
                type: "POST",
                data: fd,
                processData: false,
                contentType: false,
            });               
        }

        function uploadFile(blobFile, fileName, filePart, totalChunks, divBarsSelector) {
            if(filePart == 0) {
                bars = document.querySelector('#bars' + divBarsSelector);  
            }

            var progress = document.createElement('progress');
            progress.min = 0;
            progress.max = 100;
            progress.value = 0;
            bars.appendChild(progress);   

            var fd = new FormData();
            fd.append("fileToUpload", blobFile);

            var xhr = new XMLHttpRequest();                
            xhr.open("POST", "upload.php"+"?"+"file="+fileName + filePart, true);

            xhr.onload = function(e) {
                //make sure if finish progress bar at 100%
                progress.value = 100;

                //counter if everything is done using stack
                uploaders.pop();

                if (!uploaders.length) {
                    bars.appendChild(document.createElement('br'));
                    bars.appendChild(document.createTextNode('DONE :)'));
                    //mergeFile(fileName, totalChunks);
                }                  
            };

            // Listen to the upload progress for each upload.   
            xhr.upload.onprogress = function(e) {;
                if (e.lengthComputable) {
                    progress.value = (e.loaded / e.total) * 100;
                }
            };                 

            uploaders.push(xhr);
            xhr.send(fd);
        }

and the server part for receiving will be upload.php

$target_path = "uploads/";
$tmp_name = $_FILES['fileToUpload']['tmp_name'];
$size = $_FILES['fileToUpload']['size'];
$name = $_FILES['fileToUpload']['name'];

$originalName = $_GET['file'];

print_r("*******************************************\n");
print_r($originalName);
print_r("\n");
print_r($_FILES);
print_r("\n");
print_r("*******************************************\n");
$target_file = $target_path . basename($name);

//Result File
$complete = $originalName;
$com = fopen("uploads/".$complete, "ab");
error_log($target_path);

if ( $com ) {
    // Read binary input stream and append it to temp file
    $in = fopen($tmp_name, "rb");
    if ( $in ) {
        while ( $buff = fread( $in, 1048576 ) ) {
            fwrite($com, $buff);
        }   
    }
    fclose($in);
    fclose($com);
}
like image 818
Harts Avatar asked May 24 '12 19:05

Harts


People also ask

How do I use HTTP to upload a file?

Configure 'HTTP Request' actionMethod (Mandatory): Select the HTTP method that you wish to use. Since we wish to upload the file, select POST method. URL (Mandatory): Enter the URL of the service to which you wish to upload the file. For this example, we will enter the URL for Built.io Backend.


2 Answers

After reading your motivation in your comment I would like to point out a few 'misconceptions'. First of all, it's not advisable to split a file up and next upload all the splitted parts at once. The entire point of splitting a file up is not to bypass the PHP upload limit (which, if applicable, should be changed and that may be a real solution*), but rather by doing the different part sequentially this allows the load on the client computer to be minimal, especially if you are considering uploading 1GB of content. Either way, there is seriously no reason to split a file up and next combine it in a single request (although this would be theoretically possible with XMLHttpRequest2, but if you can use XMLHttpRequest2 then you shouldn't worry about splitting the file up either way, as it provides the necessary controls to upload multiple files cleanly).

*Please note that in case you do that you will have to make sure your php memory settings are correctly set up (to prevent php trying to load it entirely into memory before writing it to a temp file, but this shouldn't happen on recent versions of PHP with the default settings I believe). (I feel obliged to add that I haven't worked with PHP and PHP uploads for a few years, so I might be very well mistaken with this last comment)

Either way, chunking the files to about 5-25MB (depending on how good you expect the connection to be :P ) + sequential uploads (plus a nice progressbar if XMLHttpRequest2 is available, otherwise a progressbar per chunk) seem a sensible way to go whilst preventing the browser from getting overloaded. (Oh and, if you need to support older browser I would really advise you to look into flash uploaders, because despite Apple preaching flash to be evil, on the majority of (outdated) computers it will give the best experience by far)

like image 88
David Mulder Avatar answered Sep 20 '22 13:09

David Mulder


Java uploaders [namely, JumpLoader] - I am not saying "use them", but learn how they work. So far, the best upload practice I have seen is: 1) split files to chunks of certain size, 2) upload chunks sequentially (additionally by providing hashes of chunks, if data is sensitive), 3) unite chunks at server-side (but verify data-integrity through hashes, if you are using them).

Thus you will bypass PHP's max_upload_size restriction. Otherwise, I personally don't see any merit why someone should split the data into chunks at first place.

like image 35
Shahriyar Imanov Avatar answered Sep 17 '22 13:09

Shahriyar Imanov