Code Coverage with PHPUnitSeleniumTestcase

Question

I have written Selenium Test case for PHP. I would like to get the code coverage for while I execute these test cases. My testcase:

<?php
class Example extends PHPUnit_Extensions_SeleniumTestCase
{
  protected $coverageScriptUrl = 'http://applicationname/phpunit_coverage.php';

  protected function setUp()
  {
    $this->setBrowser("*firefox");
    $this->setBrowserUrl("http://applicationname");
    $this->setCollectCodeCoverageInformation(true);
    $this->setTestId("10001");
    $this->setHost("applicationname");
  }

  public function testMyTestCase()
  {
    $this->open("http://applicationame");
    $this->assertEquals("title", $this->getTitle());
    $this->type("id=ext-comp-1002", "testuser");
    $this->fireEvent("id=ext-comp-1002", "blur");
    $this->type("id=ext-comp-1003", "testpassword");
    $this->fireEvent("id=ext-comp-1003", "blur");
    $this->click("ext-gen45");
    $this->waitForPageToLoad("200000");
}
}
?>

I have followed the steps mentioned in the link "http://www.phpunit.de/manual/current/en/selenium.html"

After running the test I am not able to find the code coverage. In phpunit_coverage.php, it is looking cookie with name PHPUNIT_SELENIUM_TEST_ID. This cookie is being created in Driver.php and I see cookie is available, but it has hostname to set to "localhost" rather than my application name.

Cookie life time is set session i.e. means immediately after test case execution this cookie will no longer available and when I try to launch phpunit_coverage.php, it is not able to find the cookie and information so no code coverage is appearing.

Things I don't understand:

1. protected $coverageScriptUrl = 'http://applicationname/phpunit_coverage.php';
2. If the cookie has has different host other than application can this cookie be accessable

I have seen this problem being discussed in many forums, but one gave concrete answer

Many forums suggested to use localhost instead of 127.0.0.1 as server name. In my case it is already localhost.

Any suggestion in this regard will be helpful.

Thanks, Ravuri

Answer 1

I think frosty's answer could be made even stronger, but I'm such a total noob with both PHPUnit and Selenium that I'm not completely sure of what I'm saying. So I'll say it and see if I get corrected.

Unit tests exercise your application code under the direct control of PHPUnit. You give PHPUnit the method in your code to invoke, and it invokes that method under Xdebug to gather the coverage information. I think of it as having your code running in the same address space as PHPUnit, even though that might not be strictly true - does anybody know if it is?

With tests run under Selenium, your code is not directly under the control of PHPUnit at all. Instead of a method in your code, you give PHPUnit a URL, and it arranges to feed that URL to a real web browser. The web browser itself need not be running on the same host machine as PHPUnit; and even if it is, your application code being tested runs on the webserver designated by the URL. Ain't no way PHPUnit can tell Firefox to tell the server handling a request that if handling the request invokes PHP, then run that PHP code under Xdebug and send trace output back along with the response! PHPUnit only gets to see the URL you specified and the output from the web browser that serviced the request. It has no way to find out what code the webserver handling the request actually ran.

So where the previous answer said that code coverage reports for these tests wouldn't provide useful information, and that unit tests would generate more meaningful reports, I'd go all the way to say that it's not possible for these tests to measure code coverage at all, so you should not ask for code coverage reports when you run them! Or rather that if you do generate code coverage reports for selenium tests, and the reports say that even one line of your code ran, then something is seriously wrong with your setup.

Answer 2

Integration or Functional tests with Selenium aren't covering code inasmuch as they're covering behavior. Code coverage reports for tests like this aren't going to generate any sort of useful information. Unit tests will generate much more meaningful code coverage reports. The tests are being run based on information provided to and from Selenium, it's not really testing your "code" so to speak.