I've tried everything to change the max_execution_time
of a php crawler script so that it can run an infinite amount of time.
I have changed the php.ini file setting max_execution_time
to 0
or 100000000
but with no change
I've also tried setting it from the php script itself by using ini_set('max_execution_time', 0);
All php scripts throw the same error Fatal error: Maximum execution time of 3000 seconds exceeded
, what could I be missing and how can I make sure there is no max execution time limit?
php script
<?php
ini_set('MAX_EXECUTION_TIME', -1);
error_reporting(E_ALL); // turn on all errors, warnings and notices for easier debugging
//ini_set('max_execution_time', 123456);
ini_set('max_input_time', -1);
ini_set('memory_limit', '512M');
set_time_limit(0);
date_default_timezone_set('Europe/London');
/*code which scrapes websites*/
?>
phpinfo()
max_execution_time 0 0
max_input_time -1 -1
ini_set('max_execution_time', 108000); which equals to 30 hours.
The default limit is 30 seconds or, if it exists, the max_execution_time value defined in the php. ini . When called, set_time_limit() restarts the timeout counter from zero.
Try turning off safe mode in php and then try the below code
if( !ini_get('safe_mode') ){
set_time_limit(0); //this won't work if safe_mode is enabled.
}
This should allow you to run the script for infinite time.
In Apache you can change maximum execution time by .htaccess with this line
php_value max_execution_time 200
set_time_limit() php manual ref.
You shouldn't let your crawler run under apache, it's better to run it stand-alone via cli as part of a Gearman setup.
That way it won't hog your web server and it can run as long as you want. You can find many bindings for Gearman that you can use, including PHP of course.
use
set_time_limit(0);
at the top of the script
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With