php merge two or more ArrayIterators by one of their values

Question

I have two iterators which i have to merge into one result.

Here are data samples:

ArrayIterator Object
(
[storage:ArrayIterator:private] => Array
    (
        [0] => Array
            (
                [period] => 04/04/2012 16:00:00
                [bl_subs] => 1
                [bl_unsubs] => 1
                [bl_block_total] => 1
            )

        [1] => Array
            (
                [period] => 04/04/2012 17:00:00
                [bl_subs] => 1
                [bl_unsubs] => 2
                [bl_block_total] => 0
            )

        [2] => Array
            (
                [period] => 04/04/2012 18:00:00
                [bl_subs] => 0
                [bl_unsubs] => 0
                [bl_block_total] => -1
            )

        [3] => Array
            (
                [period] => 04/04/2012 19:00:00
                [bl_subs] => 2
                [bl_unsubs] => 0
                [bl_block_total] => -2
            )

        [4] => Array
            (
                [period] => 04/04/2012 20:00:00
                [bl_subs] => 2
                [bl_unsubs] => 0
                [bl_block_total] => 1
            )

    )

)


ArrayIterator Object
(
[storage:ArrayIterator:private] => Array
    (
        [0] => Array
            (
                [period] => 04/04/2012 15:00:00
                [bl_avg] => 5
                [bl_full] => 0
            )

        [1] => Array
            (
                [period] => 04/04/2012 17:00:00
                [bl_avg] => 0
                [bl_full] => 7
            )

        [2] => Array
            (
                [period] => 04/04/2012 18:00:00
                [bl_avg] => 1
                [bl_full] => 0
            )

    )

)

I would like to merge them by the key 'period' into a summary iterator.

The final result should look like:

ArrayIterator Object
(
[storage:ArrayIterator:private] => Array
    (
        [0] => Array
            (
                [period] => 04/04/2012 15:00:00
                [bl_subs] => 0
                [bl_unsubs] => 0
                [bl_avg] => 5
                [bl_full] => 0
                [bl_block_total] => 0
            )

        [1] => Array
            (
                [period] => 04/04/2012 16:00:00
                [bl_subs] => 1
                [bl_unsubs] => 1
                [bl_avg] => 0
                [bl_full] => 0
                [bl_block_total] => 1
            )

        [2] => Array
            (
                [period] => 04/04/2012 17:00:00
                [bl_subs] => 1
                [bl_unsubs] => 2
                [bl_avg] => 0
                [bl_full] => 7
                [bl_block_total] => 0
            )

        [3] => Array
            (
                [period] => 04/04/2012 18:00:00
                [bl_subs] => 0
                [bl_unsubs] => 0
                [bl_avg] => 1
                [bl_full] => 0
                [bl_block_total] => -1
            )

        [4] => Array
            (
                [period] => 04/04/2012 19:00:00
                [bl_subs] => 2
                [bl_unsubs] => 0
                [bl_avg] => 0
                [bl_full] => 0
                [bl_block_total] => -2
            )

        [5] => Array
            (
                [period] => 04/04/2012 20:00:00
                [bl_subs] => 2
                [bl_unsubs] => 0
                [bl_avg] => 0
                [bl_full] => 0
                [bl_block_total] => 1
            )

    )

)

It will be best if we are not using foreach,for,while or any other loop. That's because the data will be big and we don't want to have memory issues. I was trying using current() and next() to use the inner array pointers.

If anyone knows a way out of this, please advise.

Answer 1

It both iterators are always sorted, you could cache them both, compare per each iteration which one comes first (if not equal) and process that one. If equal, process both equally.

Not equal:

$it1[[period] => 04/04/2012 16:00:00] > $it2[[period] => 04/04/2012 15:00:00]

=> process $it2 data:

    [period] => 04/04/2012 15:00:00
    [bl_avg] => 5
    [bl_full] => 0

  as current():

    [period] => 04/04/2012 15:00:00
    [bl_subs] => 1
    [bl_unsubs] => 1
    [bl_avg] => 5
    [bl_full] => 0
    [bl_block_total] => 1

+ $it2->next();

Note: I have no clue how the elements that don't exist in the source data ($it2[0] (15:00)) [bl_subs => 1], [bl_unsubs] => 1 and [bl_block_total] => 1 come from. Are that default values?

Equal: (one iteration skipped)

$it1[[period] => 04/04/2012 17:00:00] == $it2[[period] => 04/04/2012 17:00:00]

=> process $it1 and $it2 data:

    $it1:
        [period] => 04/04/2012 17:00:00
        [bl_subs] => 1
        [bl_unsubs] => 2
        [bl_block_total] => 0

    $it2:
        [period] => 04/04/2012 17:00:00
        [bl_avg] => 0
        [bl_full] => 7

  as current():

        [period] => 04/04/2012 17:00:00
        [bl_subs] => 1
        [bl_unsubs] => 2
        [bl_avg] => 0
        [bl_full] => 7
        [bl_block_total] => 0

 + $it1->next(); $it2->next();

You can wrap this processing into an Iterator of it's own so it's nicely encapsulated. Because the information given was limited, I created a simplified example that reduces the date to the problem's domain: Iterate over two iterators at once. If both iterators are equal, return both. If not equal, return the one that would be first when both are compared.

The simplified data used:

$ar1 = array('04/04/2012 16:00:00', '04/04/2012 17:00:00', '04/04/2012 18:00:00', '04/04/2012 19:00:00', '04/04/2012 20:00:00');
$ar2 = array('04/04/2012 15:00:00', '04/04/2012 17:00:00', '04/04/2012 18:00:00');

Just two arrays that contain the compare-value. Those are turned into the two iterators:

$it1 = new ArrayIterator($ar1);
$it2 = new ArrayIterator($ar2);

The problem written out is limited to two iterators. To solve the problem more generically it should work with 0 or more iterators. So what happens is that per each iteration the iterators are compared with each other based on their current value. For that a comparison function is used. You could compare that with how usort^Docs works: A function compares A and B and based on the two it returns an integer value:

• A < B: -1 (A is less than B, return value is less than zero)
• A = B: 0 (A equals B, return value equals zero)
• A > B: 1 (A is greater than B, return value is greater than zero)

This allows to compare an unlimited number of pairs with each other. It only needs two functions: One that obtains the current value from an iterator we use, and the other one that does the actual comparison between A and B (actually you can merge both into one function, but as this is exemplary and your arrays/iterators differ a bit, I thought it's worth to separate so you can modify it more easily later on). So first the function to get a value out of an iterator, I compare with ISO date-time values because I can do this with a simple strcmp:

/**
 * Get Comparison-Value of an Iterator
 *
 * @param Iterator $iterator
 * @return string
 */
$compareValue = function(Iterator $iterator) {
    $value = $iterator->current();
    sscanf($value, '%d/%d/%d %s', $month, $day, $year, $timeISO);
    $dateISO = sprintf('%04d-%02d-%02d %s', $year, $month, $day, $timeISO);
    return $dateISO;
};

Note: I don't know which date-format you use, maybe I mixed month with day, just exchange the variables, it's pretty much self-descriptive.

All this function does is to get a value that can be easily compared from an iterator. This does not yet do the comparison described above, so another function is needed which will use this comparison-value function as a dependency:

/**
 * Compare two Iterators by it's value
 *
 * @param Iterator $a
 * @param Iterator $b
 * @return int comparison result (as of strcmp())
 */
$compareFunction = function(Iterator $a, Iterator $b) use ($compareValue) {
    return strcmp($compareValue($a), $compareValue($b));
};

And that's the compare function now, based on the strcmp string comparison function and using the $compareValue function to obtain the strings for comparison.

So let's say you have an array with two iterators, it's now possible to sort it. It's also possible to compare the first element with the next element to say whether those are equal.

That done, it's now possible to create an iterator that consists of multiple iterators and at each iteration, the attached iterators are sorted and only the first iterator (and those equal to it) will be returned as current and also forwarded. Something like this flow:

^Src

As the sorting is done with the comparison function already, only this iteration logic needs to be encapsulated. As the sorting works for an array of any size (0 or more elements), it's already generalized. The usage example:

/**
 * Usage
 */
$it = new MergeCompareIterator($compareFunction, array($it1, $it2));

foreach ($it as $index => $values) {
    printf("Iteration #%d:\n", $index);
    foreach ($values as $iteratorIndex => $value) {
        printf("  * [%d] => %s\n", $iteratorIndex, $value);
    }
}

This usage example will output the iteration it's in and the associated value(s) to that iteration. In this case only the time information as the example arrays only consist of these. It also puts into the square brackets of which iterators it is from (0 for the first, 1 for the second). This generates the following output:

Iteration #0:
  * [1] => 04/04/2012 15:00:00
Iteration #1:
  * [0] => 04/04/2012 16:00:00
Iteration #2:
  * [0] => 04/04/2012 17:00:00
  * [1] => 04/04/2012 17:00:00
Iteration #3:
  * [0] => 04/04/2012 18:00:00
  * [1] => 04/04/2012 18:00:00
Iteration #4:
  * [0] => 04/04/2012 19:00:00
Iteration #5:
  * [0] => 04/04/2012 20:00:00

As you can see, for those comparison values that are equal in both (pre-sorted) iterators, are returned as a pair. In your case you would need to further process these values, e.g. merge them while providing default values:

$defaults = array('bl_subs' => 0, ...);
foreach ($it as $values) {
    array_unshift($values, $default);
    $value = call_user_func_array('array_merge', $values);
}

So that's actually the usage of that MergeCompareIterator. The implementation is rather straight forward, this one does not cache the sorting/current iterators so far, I leave this as an exercise if you want to improve it.

The full code:

<?php
/**
 * @link http://stackoverflow.com/q/10024953/367456
 * @author hakre <http://hakre.wordpress.com/>
 */

$ar1 = array('04/04/2012 16:00:00', '04/04/2012 17:00:00', '04/04/2012 18:00:00', '04/04/2012 19:00:00', '04/04/2012 20:00:00');
$ar2 = array('04/04/2012 15:00:00', '04/04/2012 17:00:00', '04/04/2012 18:00:00');

$it1 = new ArrayIterator($ar1);
$it2 = new ArrayIterator($ar2);

/**
 * Get Comparison-Value of an Iterator
 *
 * @param Iterator $iterator
 * @return string
 */
$compareValue = function(Iterator $iterator)
{
    $value = $iterator->current();
    sscanf($value, '%d/%d/%d %s', $month, $day, $year, $timeISO);
    $dateISO = sprintf('%04d-%02d-%02d %s', $year, $month, $day, $timeISO);
    return $dateISO;
};

/**
 * Compare two Iterators by it's value
 *
 * @param Iterator $a
 * @param Iterator $b
 * @return int comparison result (as of strcmp())
 */
$compareFunction = function(Iterator $a, Iterator $b) use ($compareValue)
{
    return strcmp($compareValue($a), $compareValue($b));
};

/**
 * Iterator with a comparison based merge-append strategy over 0 or more iterators.
 *
 * Compares 0 or more iterators with each other. Returns the one that comes first
 * and any additional one that is equal to the first as an array of their current()
 * values in this current().
 * next() forwards all iterators that are part of current().
 */
class MergeCompareIterator implements Iterator
{
    /**
     * @var Iterator[]
     */
    private $iterators;

    /**
     * @var callback
     */
    private $compareFunction;

    /**
     * @var int
     */
    private $index;

    /**
     * @param callback $compareFunction (same sort of usort()/uasort() callback)
     * @param Iterator[] $iterators
     */
    public function __construct($compareFunction, array $iterators = array())
    {
        $this->setCompareFunction($compareFunction);
        foreach ($iterators as $iterator) {
            $this->appendIterator($iterator);
        }
    }

    /**
     * @param callback $compareFunction
     */
    public function setCompareFunction($compareFunction)
    {
        if (!is_callable($compareFunction)) {
            throw new InvalidArgumentException('Compare function is not callable.');
        }
        $this->compareFunction = $compareFunction;
    }

    public function appendIterator(Iterator $it)
    {
        $this->iterators[] = $it;
    }

    public function rewind()
    {
        foreach ($this->iterators as $it) {
            $it->rewind();
        }
        $this->index = 0;
    }

    /**
     * @return Array one or more current values
     * @throws RuntimeException
     */
    public function current()
    {
        $current = array();
        foreach ($this->getCurrentIterators() as $key => $value) {
            $current[$key] = $value->current();
        }
        return $current;
    }

    /**
     * @return Iterator[]
     */
    private function getCurrentIterators()
    {
        /* @var $compareFunction Callable */
        $compareFunction = $this->compareFunction;

        $iterators = $this->getValidIterators();
        $r = uasort($iterators, $compareFunction);
        if (FALSE === $r) {
            throw new RuntimeException('Sorting failed.');
        }

        $compareAgainst = reset($iterators);
        $sameIterators = array();
        foreach ($iterators as $key => $iterator) {
            $comparison = $compareFunction($iterator, $compareAgainst);
            if (0 !== $comparison) {
                break;
            }
            $sameIterators[$key] = $iterator;
        }
        ksort($sameIterators);
        return $sameIterators;
    }

    /**
     * @return Iterator[]
     */
    private function getValidIterators()
    {
        $validIterators = array();
        foreach ($this->iterators as $key => $iterator) {
            $iterator->valid() && $validIterators[$key] = $iterator;
        }
        return $validIterators;
    }

    /**
     * @return int zero based iteration count
     */
    public function key()
    {
        return $this->index;
    }

    public function next()
    {
        foreach ($this->getCurrentIterators() as $iterator) {
            $iterator->next();
        }
        $this->index++;
    }

    public function valid()
    {
        return (bool)count($this->getValidIterators());
    }
}

/**
 * Usage
 */
$it = new MergeCompareIterator($compareFunction, array($it1, $it2));

foreach ($it as $index => $values) {
    printf("Iteration #%d:\n", $index);
    foreach ($values as $iteratorIndex => $value) {
        printf("  * [%d] => %s\n", $iteratorIndex, $value);
    }
}

Hope this is helpful. It does only work with presorted data within your "inner" iterators, otherwise the merge/append strategy with the comparison of current elements does not make sense.