Extending PHPUnit : adding a decorator

Question

Context

I recently inherited the development and maintenance of a well-programmed PHP application (sarcasm). The application is based on a commercial software (which I will not name), and there is a layer of customization (ours) that is built on top of it.

Unfortunately, this application uses a ton of globals and singletons (pun-intended). I have built test cases for all the things we've overridden. However, a lot of things relies on some global state of something, which can cause race conditions and all sorts of weird stuff.

Randomizing the tests

In order to catch most of these weird-o-tons (I like to call them that), I have built a PHPUnit TestDecorator, [as documented in the manual][1]. This one:

class PHPUnit_Extensions_Randomizer extends PHPUnit_Extensions_TestDecorator
{
    public function __construct(PHPUnit_Framework_Test $test)
    {
        $tests = $test->tests();

        $shuffle = array();
        foreach ($tests as $t) {
            if ($t instanceof PHPUnit_Framework_TestSuite) {
                $shuffle = array_merge($shuffle, $t->tests());
            } else {
                $shuffle[] = $t;
            }
        }
        shuffle($shuffle);

        $suite = new PHPUnit_Framework_TestSuite();
        foreach ($shuffle as $t) $suite->addTest($t);

        parent::__construct($suite);
    }
}

It basically randomizes the tests order to make sure a test doesn't depend on a global state that may or may not be correct.

The question

The problem arose when came the time to test my custom decorator. I have not found anywhere in the manual, Google, or Stack Overflow how to load it.

When analyzing the code, I saw that PHPUnit itself was instantiating the RepeatedTest decorator in the TextUI_TestRunner::doRun() method. I know I can subclass the TestRunner, override doRun(), arrange for my decorator to be created and then just call the parent::doRun() with my decorator instance as the argument, override TextUI_Command and create a new CLI script to use my stuff instead of the built-in stuff.

Before I (re-)invent the wheel, I was just wondering, is it possible do load a custom decorator without subclassing the TestRunner?

Thanks.

Answer 1

With current PHPUnit Versions there is no easy way to plug in in your own code for the most part. The only thing that offers interchangeability is the TestRunner and what you described makes sense to me.

I'm not aware of any other way to provide test decorators or change out most of the other classes phpunit uses.

The way you want to change the test execution order seems to work out even so I'm not sure how well it would shuffle suits.

---

Another way to achieve this thats maybe less hassle would be to create a subclass of PHPUnit_Framework_TestSuite randomly addTest your code.

---

If that doesn't work out another option would be to use the xml configuration file and build the test suite from <file> tags and before every execution have some code shuffle those tags. Afaik phpunit doesn't sort them in any way so execution will be random

---

Are you looking to see if every tests on its on really works and want to find interdependent tests?

Or do you really want to see if something breaks horribly when you do a lot of stuff that should not change anything in the wrong order?

I'm asking just in case you haven't considered --process-isolation yet. (Which i assume you have but asking isn't going to hurt and might save you some time and effort :) )

When you run every test with a fresh set of globals you will at least find all test interdependencies and thats just one cli switch away to make sure every test in your suite works fine on its own.