PHPUnit - Why would PHPUnit appear to be running in strict mode?

Question

The question: Why would PHPUnit appear to be running in strict mode?

The issue:

PHPUnit 4.3.1 by Sebastian Bergmann.

Configuration read from /full/path/to/configuration.xml

R

Time: 2.65 seconds, Memory: 11.50Mb

OK, but incomplete, skipped, or risky tests! Tests: 1, Assertions: 1, Risky: 1. Done.

Also:

Risky Test: Test code or tested code did not (only) close its own output buffers

My PHP Version is 5.4.

As stated in the documentation (https://phpunit.de/manual/current/en/strict-mode.html) this only appears to apply to PHPUnits' strict mode setting.

PHPUnit can perform additional checks while it executes the tests. In addition to the fine-grained control over the various strict mode checks (see below) you may use the --strict commandline option or set strict="true" in PHPUnit's XML configuration file to enable all of them.

-

Output During Test Execution

PHPUnit can be strict about output during tests. This check can be enabled by using the --disallow-test-output option on the commandline or by setting beStrictAboutOutputDuringTests="true" in PHPUnit's XML configuration file.

A test that emits output, for instance by invoking print in either the test code or the tested code, will be marked as risky when this check is enabled.

I believe though, that I did not activate strict mode. My command line is "/usr/bin/php /usr/bin/phpunit --colors --bootstrap /full/path/to/bootstrap.php --configuration /full/path/to/configuration.xml /full/path/to/Test.php". I also used the configuration as provided at "https://phpunit.de/manual/current/en/appendixes.configuration.html".

<phpunit
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.3/phpunit.xsd"
   backupGlobals="true"
   backupStaticAttributes="false"
   cacheTokens="false"
   colors="false"
   convertErrorsToExceptions="true"
   convertNoticesToExceptions="true"
   convertWarningsToExceptions="true"
   forceCoversAnnotation="false"
   mapTestClassNameToCoveredClassName="false"
   printerClass="PHPUnit_TextUI_ResultPrinter"
   processIsolation="false"
   stopOnError="false"
   stopOnFailure="false"
   stopOnIncomplete="false"
   stopOnSkipped="false"
   testSuiteLoaderClass="PHPUnit_Runner_StandardTestSuiteLoader"
   timeoutForSmallTests="1"
   timeoutForMediumTests="10"
   timeoutForLargeTests="60"
   strict="false"
   verbose="false">
</phpunit>

I had used a shorter version of this configuration previously which provided the same result.

<phpunit
   beStrictAboutOutputDuringTests="false"
   strict="false"
   colors="false">
</phpunit>

Answer 1

Looking at the code available on GitHub it seems that regardless of what the documentation may say, the output buffering problem is checked and reported always.

So the symptoms you observe don't mean that the tests run in strict mode.

github.com/sebastianbergmann/phpunit/blob/4.3/src/Framework/TestCase.php#L818

// ...

try {
    $this->stopOutputBuffering();
} catch (PHPUnit_Framework_RiskyTestError $_e) {
    if (!isset($e)) {
        $e = $_e;
    }
}

github.com/sebastianbergmann/phpunit/blob/4.3/src/Framework/TestCase.php#L1938-L1946

private function stopOutputBuffering()
{
    if (ob_get_level() != $this->outputBufferingLevel) {
        while (ob_get_level() > 0) {
            ob_end_clean();
        }
        throw new PHPUnit_Framework_RiskyTestError(
            'Test code or tested code did not (only) close its own output buffers'
        );
    }

    // ...

    $this->outputBufferingActive = false;
    $this->outputBufferingLevel  = ob_get_level();
}

Placing breakpoint at the above lines in your favorite PHPUnit test debugger may reveal some other dependencies (like disallowTestOutput flag...?)