Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 9 down vote favorite
4

I have written Selenium Test case for PHP. I would like to get the code coverage for while I execute these test cases. My testcase:

<?php
class Example extends PHPUnit_Extensions_SeleniumTestCase
{
  protected $coverageScriptUrl = 'http://applicationname/phpunit_coverage.php';

  protected function setUp()
  {
    $this->setBrowser("*firefox");
    $this->setBrowserUrl("http://applicationname");
    $this->setCollectCodeCoverageInformation(true);
    $this->setTestId("10001");
    $this->setHost("applicationname");
  }

  public function testMyTestCase()
  {
    $this->open("http://applicationame");
    $this->assertEquals("title", $this->getTitle());
    $this->type("id=ext-comp-1002", "testuser");
    $this->fireEvent("id=ext-comp-1002", "blur");
    $this->type("id=ext-comp-1003", "testpassword");
    $this->fireEvent("id=ext-comp-1003", "blur");
    $this->click("ext-gen45");
    $this->waitForPageToLoad("200000");
}
}
?>

I have followed the steps mentioned in the link "http://www.phpunit.de/manual/current/en/selenium.html"

After running the test I am not able to find the code coverage. In phpunit_coverage.php, it is looking cookie with name PHPUNIT_SELENIUM_TEST_ID. This cookie is being created in Driver.php and I see cookie is available, but it has hostname to set to "localhost" rather than my application name.

Cookie life time is set session i.e. means immediately after test case execution this cookie will no longer available and when I try to launch phpunit_coverage.php, it is not able to find the cookie and information so no code coverage is appearing.

Things I don't understand:

  1. protected $coverageScriptUrl = 'http://applicationname/phpunit_coverage.php';
  2. If the cookie has has different host other than application can this cookie be accessable

I have seen this problem being discussed in many forums, but one gave concrete answer

Many forums suggested to use localhost instead of 127.0.0.1 as server name. In my case it is already localhost.

Any suggestion in this regard will be helpful.

Thanks, Ravuri

share|improve this question
1  
Maybe answers to this questions might offer some help: stackoverflow.com/questions/11270527/… – Potherca Sep 7 '12 at 20:57

2 Answers

up vote 2 down vote

Integration or Functional tests with Selenium aren't covering code inasmuch as they're covering behavior. Code coverage reports for tests like this aren't going to generate any sort of useful information. Unit tests will generate much more meaningful code coverage reports. The tests are being run based on information provided to and from Selenium, it's not really testing your "code" so to speak.

share|improve this answer
up vote 0 down vote

I think frosty's answer could be made even stronger, but I'm such a total noob with both PHPUnit and Selenium that I'm not completely sure of what I'm saying. So I'll say it and see if I get corrected.

Unit tests exercise your application code under the direct control of PHPUnit. You give PHPUnit the method in your code to invoke, and it invokes that method under Xdebug to gather the coverage information. I think of it as having your code running in the same address space as PHPUnit, even though that might not be strictly true - does anybody know if it is?

With tests run under Selenium, your code is not directly under the control of PHPUnit at all. Instead of a method in your code, you give PHPUnit a URL, and it arranges to feed that URL to a real web browser. The web browser itself need not be running on the same host machine as PHPUnit; and even if it is, your application code being tested runs on the webserver designated by the URL. Ain't no way PHPUnit can tell Firefox to tell the server handling a request that if handling the request invokes PHP, then run that PHP code under Xdebug and send trace output back along with the response! PHPUnit only gets to see the URL you specified and the output from the web browser that serviced the request. It has no way to find out what code the webserver handling the request actually ran.

So where the previous answer said that code coverage reports for these tests wouldn't provide useful information, and that unit tests would generate more meaningful reports, I'd go all the way to say that it's not possible for these tests to measure code coverage at all, so you should not ask for code coverage reports when you run them! Or rather that if you do generate code coverage reports for selenium tests, and the reports say that even one line of your code ran, then something is seriously wrong with your setup.

share|improve this answer
I don't know if I'd use the term "under the control" of PHPUnit, because the code in both cases is being executed by PHPUnit. The difference, at least in my mind is that because unit tests exist to test each individual unit of code in seclusion; while Selenium type tests deal with the integration of multiple units of code. In those cases code coverage doesn't really tell you anything... – frosty Feb 25 at 1:30
Second try: To me, "executed by" is consistent with unit tests, where the php interpreter that is interpreting the PHPUnit source is making a call into a method in your source code. E.g. the unit testing mechanism in PHPUnit can catch an unhandled exception thrown by the method being tested. But in Selenium tests, your code is being run by some other means, and need not even be written in PHP. Tests run via selenium cannot catch exceptions thrown by your application, they can only see what effect the exception might have had on the browser output. To me, that is very different. – sootsnoot Feb 25 at 5:46
I hadn't read the link by @Potherca in a note to the original question. It doesn't change how I think of "under control of" or "executed by". But it does point out that if you have control over the webserver running the code being tested, it is possible to make it record coverage for tests run on that webserver through selenium, such that they are included in the coverage report. I think that's awesome! While I agree with frosty that getting good coverage through unit tests is more valuable/meaningful, coverage through selenium tests is not worthless - I just thought it wasn't possible! – sootsnoot Feb 25 at 6:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.