Split PHPunit tests to stop out of memory error

Question

I have a business app that uses phpunit for unit testing. The business app is growing in size and growing in unit tests. When I run phpunit to get a code coverage report, I am running out of memory. What is a good way to generate a code coverage report with lots of unit test?

Running the below phpunit.xml, I get the following error before finishing:

PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1068290 bytes)

I am using PHP5.6 and Jenkins to run the job.

<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
  backupStaticAttributes="false"
  bootstrap="./src/bootstrap/autoload.php"
  colors="true"
  convertErrorsToExceptions="true"
  convertNoticesToExceptions="true"
  convertWarningsToExceptions="true"
  processIsolation="false"
  stopOnFailure="false"
  syntaxCheck="false"
>
 <testsuites>
  <testsuite name="module1">
   <directory>./tests/module1</directory>
   <directory>./tests/module2</directory>
   <directory>./tests/module3</directory>
   <directory>./tests/module4</directory>
   <directory>./tests/module5/directory>
   <directory>./tests/module6</directory>
   <directory>./tests/module7</directory>
   <directory>./tests/module8</directory>
  </testsuite>
  <testsuite name="module2">
   <directory>./tests/module9</directory>
   <directory>./tests/module10</directory>
   <directory>./tests/module11</directory>
   <directory>./tests/module12</directory>
   <directory>./tests/module13</directory>
   <directory>./tests/module14</directory>
   <directory>./tests/module15</directory>
   <directory>./tests/module16</directory>
  </testsuite>
 </testsuites>

 <filter>
  <blacklist>
   <directory>./src</directory>
  </blacklist>
  <whitelist>
   <directory>./src/app/module</directory>
  </whitelist>
 </filter>

 <logging>
  <log type="coverage-html" target="build/module" title="module"
   charset="UTF-8" yui="true" highlight="true"
   lowUpperBound="70" highLowerBound="90"/>
  <log type="junit" target="build/logs/module.xml" logIncompleteSkipped="false"/>
 </logging>

</phpunit>

Answer 1

well, you'll rarely need to blacklist something that isn't contained inside the whitelisted files - though that may just save you time starting the tests. Whitelist the code you are testing and then <exclude> the directories that don't have useful source code.

As for reducing the amount of memory used, there are a few techniques that will help - first, PHP7.1+ will use a significant amount less memory. Second, find out what tests or code are using so much memory. Test listeners like atrapalo/phpunit-memory-and-time-usage-listener can report when tests use a lot of memory that is not being cleaned up. You may also find code improvements with this.

Other PHPUnit plugins like mybuilder/phpunit-accelerator try to automatically 'unset' variables. Doing so with your own tearDown() functions in tests will also assist.

In the short term, if the machines you are running the tests on do have a lot of RAM still available, you can simply increase the limit:

<!-- phpunit.xml -->
<php>
    <ini name="memory_limit" value="-1" />
    <!-- other PHP.ini or environment variables -->
</php>

Finally, if your codebase is truly huge, you can split the tests and coverage reports up into separate parts, writing the coverage reports to different files that are merged using the phpcov tool.