How to add uncovered files to PHPUnit code coverage report of the Yii application

Question

I am currently trying to get the code coverage report for a PHP application based on Yii framework.

Code coverage gets generated by PHPUnit 3.6, and I am using whitelist approach to source files filtering.

Problem is, when I set the option addUncoveredFilesFromWhitelist="true", code coverage breaks with the following error:

Generating code coverage report, this may take a moment.PHP Warning:  include(CButtonColumn.php): failed to open stream: No such file or directory in /home/hijarian/systems/yii/framework/YiiBase.php on line 418
PHP Stack trace:
PHP   1. {main}() /usr/bin/phpunit:0
PHP   2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46
PHP   3. PHPUnit_TextUI_Command->run() /usr/share/php/PHPUnit/TextUI/Command.php:125
PHP   4. PHPUnit_TextUI_TestRunner->doRun() /usr/share/php/PHPUnit/TextUI/Command.php:187
PHP   5. PHP_CodeCoverage_Report_HTML->process() /usr/share/php/PHPUnit/TextUI/TestRunner.php:373
PHP   6. PHP_CodeCoverage->getReport() /usr/share/php/PHP/CodeCoverage/Report/HTML.php:133
PHP   7. PHP_CodeCoverage_Report_Factory->create() /usr/share/php/PHP/CodeCoverage.php:141
PHP   8. PHP_CodeCoverage->getData() /usr/share/php/PHP/CodeCoverage/Report/Factory.php:65
PHP   9. PHP_CodeCoverage->processUncoveredFilesFromWhitelist() /usr/share/php/PHP/CodeCoverage.php:173
PHP  10. include_once() /usr/share/php/PHP/CodeCoverage.php:516
PHP  11. YiiBase::autoload() /home/hijarian/systems/yii/framework/YiiBase.php:0

PHP Warning:  include(): Failed opening 'CButtonColumn.php' for inclusion (include_path='.:/home/hijarian/projects/lexgarant/webapp/protected/components:/home/hijarian/projects/lexgarant/webapp/protected/models:/usr/share/php:/usr/share/pear') in /home/hijarian/systems/yii/framework/YiiBase.php on line 418
PHP Stack trace:
PHP   1. {main}() /usr/bin/phpunit:0
PHP   2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46
PHP   3. PHPUnit_TextUI_Command->run() /usr/share/php/PHPUnit/TextUI/Command.php:125
PHP   4. PHPUnit_TextUI_TestRunner->doRun() /usr/share/php/PHPUnit/TextUI/Command.php:187
PHP   5. PHP_CodeCoverage_Report_HTML->process() /usr/share/php/PHPUnit/TextUI/TestRunner.php:373
PHP   6. PHP_CodeCoverage->getReport() /usr/share/php/PHP/CodeCoverage/Report/HTML.php:133
PHP   7. PHP_CodeCoverage_Report_Factory->create() /usr/share/php/PHP/CodeCoverage.php:141
PHP   8. PHP_CodeCoverage->getData() /usr/share/php/PHP/CodeCoverage/Report/Factory.php:65
PHP   9. PHP_CodeCoverage->processUncoveredFilesFromWhitelist() /usr/share/php/PHP/CodeCoverage.php:173
PHP  10. include_once() /usr/share/php/PHP/CodeCoverage.php:516
PHP  11. YiiBase::autoload() /home/hijarian/systems/yii/framework/YiiBase.php:0

PHP Fatal error:  Class 'CButtonColumn' not found in /home/hijarian/projects/lexgarant/webapp/protected/components/CLexgarantButtonColumn.php on line 4
PHP Stack trace:
PHP   1. {main}() /usr/bin/phpunit:0
PHP   2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46
PHP   3. PHPUnit_TextUI_Command->run() /usr/share/php/PHPUnit/TextUI/Command.php:125
PHP   4. PHPUnit_TextUI_TestRunner->doRun() /usr/share/php/PHPUnit/TextUI/Command.php:187
PHP   5. PHP_CodeCoverage_Report_HTML->process() /usr/share/php/PHPUnit/TextUI/TestRunner.php:373
PHP   6. PHP_CodeCoverage->getReport() /usr/share/php/PHP/CodeCoverage/Report/HTML.php:133
PHP   7. PHP_CodeCoverage_Report_Factory->create() /usr/share/php/PHP/CodeCoverage.php:141
PHP   8. PHP_CodeCoverage->getData() /usr/share/php/PHP/CodeCoverage/Report/Factory.php:65
PHP   9. PHP_CodeCoverage->processUncoveredFilesFromWhitelist()     /usr/share/php/PHP/CodeCoverage.php:173
PHP  10. include_once() /usr/share/php/PHP/CodeCoverage.php:516

(It's interesting by itself, why there is three almost identical errors instead of just one.) When addUncoveredFilesFromWhitelist="false" code coverage generates successfully, but, of course, without the files not covered by tests.

Files of the Yii framework placed in the $HOME/systems/yii/ directory.

Application I'm trying to cover placed in the $HOME/projects/$APPNAME/.

Here's my phpunit.xml:

<phpunit bootstrap="bootstrap.php"
    colors="true"
    convertErrorsToExceptions="true"
    convertNoticesToExceptions="true"
    convertWarningsToExceptions="true"
    stopOnFailure="false">

<filter>
<!-- DO NOT SET TO "true" CODE COVERAGE WILL BREAK -->
  <whitelist addUncoveredFilesFromWhitelist="false">
    <directory suffix=".php">..</directory>
    <exclude>
      <directory suffix=".php">../config/</directory>
      <directory suffix=".php">../messages/</directory>
      <directory suffix=".php">../tests/</directory>
      <directory suffix=".php">../runtime/</directory>
      <directory suffix=".php">/home/hijarian/systems/yii/framework/</directory>
    </exclude>
  </whitelist>
</filter>

<logging>
    <log type="coverage-html" 
        target="report/html" 
        title="Lexgarant Online"
        charset="UTF-8" 
        yui="true" 
        highlight="true"
        lowUpperBound="35" highLowerBound="70" />
</logging>

<testsuites>
    <testsuite name="All_Unit_Tests">
        <directory>./unit/models/</directory>
        <directory>./unit/logic/</directory>
        <directory>./unit/components/</directory>
    </testsuite>
</testsuites>

<selenium>
    <browser name="Firefox" browser="*firefox" />
</selenium>
</phpunit>

I have all the tests placed under the protected/tests directory in the application's catalogue, as described in the Test Environment Setup in The Definitive Guide to Yii.

PHPUnit is invoked as follows:

cd $APPDIR/protected/tests/
phpunit

The unit testing itself goes smoothly, only code coverage breaks.

Maybe I should set up the exlude list to exclude directories with Yii's classes, but I am somewhat afraid of this experience because there's dozens of folders, Yii is a highly modular system. Of course, if it's an only solution...

---

EDIT: So, for curious about it, full solution looks like this:

phpunit.xml

<php>
    <!--  HACK -->
    <includePath>YIIPATH/framework/zii/widgets/grid</includePath>
</php>

<filter>
  <whitelist addUncoveredFilesFromWhitelist="true">
    <directory suffix=".php">../models/</directory>
    <directory suffix=".php">../components/</directory>
    <!--  any other directories you want to check -->
  </whitelist>
</filter>

Just replace YIIPATH placeholder with real path to Yii framework.

It definitely has something to do with CButtonColumn, but for now I'm just using this hack with includePath and happy about it. Removing the hack brings this whole bug alive.

Answer 1

Why this is happening:

PHPUnit will require / include all files in the white-listed if you tell it do addUncoveredFilesFromWhitelist="true".

If those files contain executable code it will be executed and you are seeing the results of the arising issues.

---

What to do

• Exclude all files with executable code from the whitelist

It's a little trouble but it works

• Fix your include paths so the include/require calls don't break anything

I have no real clue as to WHY those errors pop up but it seems to be an include path issue? The framework should have a way of resolving that I'd say.

• PHPUnit 3.7 will offer an "generate coverage but not via include" mode that looks at the files and "guesses" the executable lines for you.

Answer 2

The whitest and code coverage report should include only the files you are testing. Assuming you aren't writing tests for the Yii framework itself, remove it from the whitest. Not only will this hopefully solve the problem, but the report will build much faster.