debugging long running PHP script

Question

I have php script running as a cron job, extensively using third party code. Script itself has a few thousands LOC. Basically it's the data import / treatment script. (JSON to MySQL, but it also makes a lot of HTTP calls and some SOAP).

Now, performance is downgrading with the time. When testing with a few records (around 100), performance is ok, it is done in a 10-20 minutes. When running whole import (about 1600 records), mean time of import of one record grows steadily, and whole thing takes more than 24 hours, so at least 5 times longer than expected.

Memory seems not to be a problem, usage growing as it should, without unexpected peaks.

So, I need to debug it to find the bottleneck. It can be some problem with the script, underlying code base, php itself, database, os or network part. I am suspecting for now some kind of caching somewhere which is not behaving well with a near 100 % miss ratio.

I cannot use XDebug, profile file grows too fast to be treatable.

So question is: how can I debug this kind of script?

PHP version: 5.4.41 OS: Debian 7.8 I can have root privileges if necessary, and install the tools. But it's the production server and ideally debugging should not be too disrupting.

Answer 1

Yes its possible and You can use Kint (PHP Debugging Script)

What is it? Kint for PHP is a tool designed to present your debugging data in the absolutely best way possible.

In other words, it's var_dump() and debug_backtrace() on steroids. Easy to use, but powerful and customizable. An essential addition to your development toolbox.

Still lost? You use it to see what's inside variables.

Act as a debug_backtrace replacer, too

you can download here or Here

Total Documentations and Help is here

Plus, it also supports almost all php framework

• CodeIgniter
• Drupal
• Symfony
• Symfony 2
• WordPress
• Yii
• framework
• Zend Framework

All the Best.... :)

Answer 2

There are three things that come to mind:

1. Set up an IDE so you can debug the PHP script line by line
2. Add some logging to the script
3. Look for long running queries in MySQL

Debug option #2 is the easiest. Since this is running as a cron job, you add a bunch of echo's in your script:

<?php

function log_message($type, $message) {
    echo "[{strtoupper($type)}, {date('d-m-Y H:i:s')}] $message";
}

log_message('info', 'Import script started');

// ... the rest of your script

log_message('info', 'Import script finished');

Then pipe stdout to a log file in the cron job command.

01 04 * * * php /path/to/script.php >> /path/to/script.log

Now you can add log_message('info|warn|debug|error', 'Message here') all over the script and at least get an idea of where the performance issue lies.

Debug option #3 is just straight investigation work in MySQL. One of your queries might be taking a long time, and it might show up in a long running query utility for MySQL.