Manual:PHP unit testing/Writing unit tests

« Unit testing
PHPUnit testing

Contents

• Getting started
• Installing PHPUnit
• Running the unit tests
  • PostgreSQL
  • Oracle
• Writing testable code
• Writing unit tests
  • ...for extensions
• Continuous integration
• Fixing broken tests
• Appendix
  (how to help, resources ..)

---

Tools

• PHPUnit

The PHPUnit Manual provides good instructions for understanding and developing unit tests. Pay particularly close attention to the sections on writing and organizing tests.

Developers new to unit testing in MediaWiki should use SampleTest.php as a starting point – it contains helpful comments that will ease the process of learning to write unit tests.

Another resource is the slides from the PHPUnit Best Practices talk that Sebastian Bergmann gave at OSCON 2010.

Write testable code

Please try to write testable code.

MediaWiki was not written with the objective of being testable. It uses global variables all over the place and static methods in many places. This is a legacy that we have to accept, but try not to introduce these things into new code, and try to change going forward.

A good resource might be Miško Hevery's Guide to Testability. (Miško Hevery is [one of?] Google's Agile Coaches.)

Test conventions

The filename must end with Test.php. Use SampleTest.php as a starting point.

setUp() and tearDown()

• Must be protected functions.
• tearDown() should be in the opposite order of setUp().
• setUp() starts with calling its parent.
• tearDown() ends with calling its parent.

Assertion group functions

• Must be public functions.
• The name of the function should be in lowerCamelCase and begin with the word test; e.g., function testFooBar.
• Whenever possible, refer to the most important method being tested; e.g., Html::expandAttributes is tested in HtmlTest::testExpandAttributes.

Data providers

• Must be public functions.
• The name of the data provider should be in lowerCamelCase and begin with the word provide; e.g., provideHtml5InputTypes.

Telling a story

Test output should tell a story (output options). The testdox format enlighten this.

@see http://phpunit.de/manual/3.8/en/other-uses-for-tests.html.

# e.g., run from /core/
tests/phpunit/phpunit.php --testdox

Number of assertions

Only one assertion per test unless there is a good reason (expensive tests may need to be grouped).

Grouping tests

PHPUnit allows tests to be put into arbitrary groups. Groups of tests can be selected for execution or excluded from execution when the test suite is run (see the @group annotation, The Command-Line Test Runner and XML Configuration File documentation in the PHPUnit manual for additional details.)

To add a test (or class) to a group, use the @group annotation in the docblock preceding the code. For example:

/**
 * @group Broken
 * @group Internationalization
 */
class HttpTest extends MediaWikiTestCase {
    ...

Several functional groups are currently used in MediaWiki unit tests:

• API: Tests that exercise the MediaWiki API.
• Broken: Put broken tests into group Broken. Tests in this group will not be run (as is configured in tests/phpunit/suite.xml).
• Database: Tests that require database connectivity should be put into group Database. NOTE: this causes temporary tables to be overlayed over the real wiki database, so test cases can perform database operations without changing the actual wiki.
• Destructive: Tests that alter or destroy data should be put into group Destructive.
• Search: Tests that use MediaWiki's built-in search put into group Search.
• SeleniumFramework: Tests that require SeleniumFramework to be installed should be put in group SeleniumFramework.
• Stub: Put test stubs into group Stub. Tests in this group will not be run (as is configured in tests/phpunit/suite.xml).
• sqlite: Tests that use SQLite should be put into group sqlite.
• Upload: Tests that upload files should be put into group Upload.
• Utility: Currently unused by any test. Tests in this group will be not be run (as is configured in tests/phpunit/suite.xml).

In addition, tests may also be grouped based on development team:

• Fundraising
• EditorEngagement
• Internationalization
• etc.

To test only a particular group, use the --group flag from the command line:

php phpunit.php --group Search

or if you use the Makefile in core/tests/phpunit:

make FLAGS="--group Search" target

where target can be phpunit, safe, etc.

Coverage

The PHPUnit documentation has a chapter about coverage. There is a coverage report for mediawiki core generated twice a day. As the forceCoversAnnotation option should be in effect the test should be marked with @covers annotations to express which parts of the code the test actually checks (as opposed to code that is just run but whose results are never tested for with assertions).

Developing best practices

Developers should avoid inventing new conventions or borrowing conventions from other frameworks; using the already well-established PHPUnit conventions will serve to make MediaWiki's tests both useful and usable. Pay particularly close attention to the sections in the PHPUnit manual on writing and organizing tests.

Databases

When testing database-dependent code, you should put your test case in the Database group (see above). That tells MediaWikiTestCase to setup a DB_MASTER database connection for you to use in $this->db. Normally, this uses a separate temporary database, with certain limited data prefilled by addCoreDBData, including a 'UTSysop' user and a 'UTPage' title. A test case can add additional data to the database by overriding addDBData (which by default does nothing).

You can directly test the current contents of the database with $this->assertSelect.

Failing

Usually tests code shouldn't do die("Error"), but use the phpunit fail method:

$this->fail( 'A useful error message' )

This would show up as a failure in the testing summary rather than bringing the whole test suite down.