PHP closures give strange performance behaviour

Question

Earlier today I was working on a PHP 5.3+ application, which meant I was free to use PHP closures. Great I thought! Then I ran into a piece of code where the use of functional PHP code would make things a lot easier, but, while I had a logical answer in mind, it made me wonder what the performance impact between directly calling a closure within array_map() and passing it down as a variable. I.e. the following two:

$test_array = array('test', 'test', 'test', 'test', 'test' );
array_map( function ( $item ) { return $item; }, $test_array );

and

$test_array = array('test', 'test', 'test', 'test', 'test' );
$fn = function ( $item ) { return $item; };
array_map( $fn, $test_array );

As I had thought, the latter was indeed faster, but the difference wasn't that big. In fact, the difference was 0.05 secs for repeating these same tests 10000 times and taking the average. May have even been a fluke.

This made me even more curious. What about create_function() and closures? Again, experience tells me create_function() should be slower when it comes to something like array_map() as it creates a function, evaluates it, and then stores this as well. And again, as I had thought, create_function() was indeed slower. This was all with array_map().

Then, and I'm not sure why I did this, but I did, I checked what the difference was between create_function() and closures while saving it and just calling it once. No processing, no nothing, just simply passing a string, and returning that string.

The tests became:

$fn = function($item) { return $item; };
$fn('test');

and

$fn = create_function( '$item', 'return $item;' );
$fn('test');

I ran both these tests 10000 times each, and looked at the results and got the average. I was quite surprised by the results.

Turns out, the closure was about 4x slower this time. This couldn't be I thought. I mean, running a closure through array_map() was much faster, and running the same function through a variable through array_map() was even faster, which was practically the same as this test.

The results were

array
  0 => 
    array
      'test' => string 'Closure test' (length=12)
      'iterations' => int 10000
      'time' => float 5.1327705383301E-6
  1 => 
    array
      'test' => string 'Anonymous test' (length=14)
      'iterations' => int 10000
      'time' => float 1.6745710372925E-5

So curious as to why it was doing this, I checked the CPU usage and other system resources and made sure nothing unnecessary was running, everything was fine now so I ran the tests again, but I got similar results.

So I tried the same tests only once, and ran it multiple times (timing it every time of course). It turns out the closure was indeed 4 times slower, except every now and then it would be about two or three times faster than create_function(), which I'm guess were just flukes, but it seemed to be enough to cut the time by half for when I did the tests 1000 times.

Below is the code I used to make these tests. Can anyone tell me what the hell is going on here? Is it my code or is it just PHP acting up?

<?php

/**
 * Simple class to benchmark code
 */
class Benchmark
{
    /**
     * This will contain the results of the benchmarks.
     * There is no distinction between averages and just one runs
     */
    private $_results = array();

    /**
     * Disable PHP's time limit and PHP's memory limit!
     * These benchmarks may take some resources
     */
    public function __construct() {
        set_time_limit( 0 );
        ini_set('memory_limit', '1024M');
    }

    /**
     * The function that times a piece of code
     * @param string $name Name of the test. Must not have been used before
     * @param callable|closure $callback A callback for the code to run.
     * @param boolean|integer $multiple optional How many times should the code be run,
     * if false, only once, else run it $multiple times, and store the average as the benchmark
     * @return Benchmark $this
     */
    public function time( $name, $callback, $multiple = false )
    {
        if($multiple === false) {
            // run and time the test
            $start = microtime( true );
            $callback();
            $end = microtime( true );

            // add the results to the results array
            $this->_results[] = array(
                'test' => $name,
                'iterations' => 1,
                'time' => $end - $start
            );
        } else {
            // set a default if $multiple is set to true
            if($multiple === true) {
                $multiple = 10000;
            }

            // run the test $multiple times and time it every time
            $total_time = 0;
            for($i=1;$i<=$multiple;$i++) {
                $start = microtime( true );
                $callback();
                $end = microtime( true );
                $total_time += $end - $start;
            }
            // calculate the average and add it to the results
            $this->_results[] = array(
                'test' => $name,
                'iterations' => $multiple,
                'time' => $total_time/$multiple
            );
        }
        return $this; //chainability
    }

    /**
     * Returns all the results
     * @return array $results
     */
    public function get_results()
    {
        return $this->_results;
    }
}

$benchmark = new Benchmark();

$benchmark->time( 'Closure test', function () {
    $fn = function($item) { return $item; };
    $fn('test');
}, true);

$benchmark->time( 'Anonymous test', function () {
    $fn = create_function( '$item', 'return $item;' );
    $fn('test');
}, true);

$benchmark->time( 'Closure direct', function () {
    $test_array = array('test', 'test', 'test', 'test', 'test' );
    $test_array = array_map( function ( $item ) { return $item; }, $test_array );
}, true);

$benchmark->time( 'Closure stored', function () {
    $test_array = array('test', 'test', 'test', 'test', 'test' );
    $fn = function ( $item ) { return $item; };
    $test_array = array_map( $fn, $test_array );
}, true);

$benchmark->time( 'Anonymous direct', function () {
    $test_array = array('test', 'test', 'test', 'test', 'test' );
    $test_array = array_map( create_function( '$item', 'return $item;' ), $test_array );
}, true);

$benchmark->time( 'Anonymous stored', function () {
    $test_array = array('test', 'test', 'test', 'test', 'test' );
    $fn = create_function( '$item', 'return $item;' );
    $test_array = array_map( $fn, $test_array );
}, true);

var_dump($benchmark->get_results());

And the results for this code:

array
  0 => 
    array
      'test' => string 'Closure test' (length=12)
      'iterations' => int 10000
      'time' => float 5.4110765457153E-6
  1 => 
    array
      'test' => string 'Anonymous test' (length=14)
      'iterations' => int 10000
      'time' => float 1.6784238815308E-5
  2 => 
    array
      'test' => string 'Closure direct' (length=14)
      'iterations' => int 10000
      'time' => float 1.5178990364075E-5
  3 => 
    array
      'test' => string 'Closure stored' (length=14)
      'iterations' => int 10000
      'time' => float 1.5463256835938E-5
  4 => 
    array
      'test' => string 'Anonymous direct' (length=16)
      'iterations' => int 10000
      'time' => float 2.7537250518799E-5
  5 => 
    array
      'test' => string 'Anonymous stored' (length=16)
      'iterations' => int 10000
      'time' => float 2.8293371200562E-5

Answer 1

5.1327705383301E-6 is not 4 times slower than 1.6745710372925E-5; it is about 3 times faster. You are reading the numbers wrong. It seems that in all of your results, the closure is consistently faster than the create_function.

Answer 2

See this benchmark:

<?php
$iter = 100000;


$start = microtime(true);
for ($i = 0; $i < $iter; $i++) {}
$end = microtime(true) - $start;
echo "Loop overhead: ".PHP_EOL;
echo "$end seconds".PHP_EOL;

$start = microtime(true);
for ($i = 0; $i < $iter; $i++) {
    $fn = function($item) { return $item; };
    $fn('test');
}
$end = microtime(true) - $start;
echo "Lambda function: ".PHP_EOL;
echo "$end seconds".PHP_EOL;


$start = microtime(true);
for ($i = 0; $i < $iter; $i++) {
    $fn = create_function( '$item', 'return $item;' );
    $fn('test');
}
$end = microtime(true) - $start;
echo "Eval create function: ".PHP_EOL;
echo "$end seconds".PHP_EOL;

Results:

Loop overhead: 
0.011878967285156 seconds
Lambda function: 
0.067019939422607 seconds
Eval create function: 
1.5625419616699 seconds

Now, interestingly enough, if you place the function declarations outside the for loops:

Loop overhead: 
0.0057950019836426 seconds
Lambda function: 
0.030204057693481 seconds
Eval create function: 
0.040947198867798 seconds

And to answer your original question, there is no difference between assigning the lambda function to a variable and simply using it. Unless you use it more than once, in which case using a variable would be better for code clarity