Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Processing large amounts of data in PHP without a browser timeout

I have array of mobile numbers, around 50,000. I'm trying to process and send bulk SMS to these numbers using third-party API, but the browser will freeze for some minutes. I'm looking for a better option.

Processing of the data involves checking mobile number type (e.g CDMA), assigning unique ids to all the numbers for further referencing, check for network/country unique charges, etc.

I thought of queuing the data in the database and using cron to send about 5k by batch every minute, but that will take time if there are many messages. What are my other options?

I'm using Codeigniter 2 on XAMPP server.

like image 466
elf1984 Avatar asked Apr 03 '11 22:04

elf1984


People also ask

How does PHP handle large amounts of data?

The fastest way to accomplish this is to grab a large chunk (many thousands of rows), blast it out to a CSV, and perform a LOAD DATA INFILE. Some environments lack the ability to use the LOAD DATA LOCAL INFILE .... LOCAL part due to server settings.

How does PHP handle connection timeout?

The default timeout is 30 seconds. It can be changed using the max_execution_time php. ini directive or the corresponding php_value max_execution_time Apache httpd. conf directive as well as with the set_time_limit() function.


1 Answers

I would write two scripts:

File index.php:

<iframe src="job.php" frameborder="0" scrolling="no" width="1" height="1"></iframe>
<script type="text/javascript">
    function progress(percent){
        document.getElementById('done').innerHTML=percent+'%';
    }
</script><div id="done">0%</div>

File job.php:

set_time_limit(0);                   // ignore php timeout
ignore_user_abort(true);             // keep on going even if user pulls the plug*
while(ob_get_level())ob_end_clean(); // remove output buffers
ob_implicit_flush(true);             // output stuff directly
// * This absolutely depends on whether you want the user to stop the process
//   or not. For example: You might create a stop button in index.php like so:
//     <a href="javascript:window.frames[0].location='';">Stop!</a>
//     <a href="javascript:window.frames[0].location='job.php';">Start</a>
// But of course, you will need that line of code commented out for this feature to work.

function progress($percent){
    echo '<script type="text/javascript">parent.progress('.$percent.');</script>';
}

$total=count($mobiles);
echo '<!DOCTYPE html><html><head></head><body>'; // webkit hotfix
foreach($mobiles as $i=>$mobile){
    // send sms
    progress($i/$total*100);
}
progress(100);
echo '</body></html>'; // webkit hotfix
like image 134
Christian Avatar answered Sep 21 '22 13:09

Christian