TDD: How to test a search?

Question

My website will have an advanced search. Pleople can go there and search about an entitiy (cars, for example). I've created some tests that check the number of results based on the search parameters. I think about what tests should I write, then i write it, then I add data to the test database. But here comes the issue. When I insert new values to the database my old tests break. That's because I'm checking the number of records...

<?php defined('SYSPATH') or die('No direct access allowed!');

class Search_Test extends PHPUnit_Extensions_Database_TestCase
{
    /**
     * @return PHPUnit_Extensions_Database_DB_IDatabaseConnection
     */
    public function getConnection()
    {
        $pdo = new PDO('mysql:dbname=db_test;host=127.0.0.1', 'root', null);
        return $this->createDefaultDBConnection($pdo, 'db_test');
    }

    /**
     * @return PHPUnit_Extensions_Database_DataSet_IDataSet
     */
    public function getDataSet()
    {
        $fixture = realpath(dirname(__FILE__).'/../data/fixture.xml');
        return $this->createXMLDataSet($fixture);
    }

    public function numberOfResultsDataProvider()
    {
        return array(
            array(1, null, null, 1),
            array(2, null, null, 3),
            array(3, null, null, 0),
            array('abc', null, null, 5),
            array(null, 1996, 2003, 3),
            array(null, 1996, 1999, 2),
            array(null, 2002, 2003, 1),
            array(null, 1500, 1800, 0),
            array(null, 2003, 2003, 1),
            array(null, null, 2005, 4),
            array(null, 1996, null, 4),
            array(null, null, null, 4),
            array(null, 2003, 1996, 0),
            array(null, 'abc', 2003, 4),
            array(null, '1996', '1999', 2),
            array(2, 2003, 2005, 2),
            array(null, null, null, 4),
        );
    }

    /**
     * @dataProvider numberOfResultsDataProvider
     */
    public function testNumberOfResults($brandId, $startYear, 
        $endYear, $numberOfResults
    ) {
        $search = ORM::factory('search');
        $search->setBrand($brandId)
            ->setYearRange($startYear, $endYear);
        $results = $search->results();
        $this->assertEquals($results->count(), $numberOfResults);
    }
}
?>

Is it normal? Should my old tests break when I create new tests?

Should my tests be bounded to data?

My search has too many parameters and they will be used in the same form (view). Should I create tests searching for each parameter or should I test they together? Should I split it up in more test classes?

Thanks.

Answer 1

when doing a unit test that involves a database, the test should provide the actual database that it will test against. It might just be a simple database that contains only values that are relevant to the test, like one row that would match and one row that doesn't. It's also reasonable to build a bunch of tests against a snapshot of the live database as it existed at a particular point in time.

One particularly convenient way of achieving this is to us a Sqlite3 Database specifically for the test, and set your application to use that for the test.

Answer 2

If at all possible, you should sever the link between how your code is fetching your data and the database itself. Your unit tests should not depend on the database, and you've stumbled upon a few reasons why: You have to maintain your data across tests, creating new tests causes others to break, and so on. If you could somehow fake the data access point and return data strictly in-memory, that would be the ideal case. I don't know how easy it would be in PHP, but building an interface around your data access code and using a fake/mocked implementation of that interface could go a long way in helping to achieve this goal.

Answer 3

In short:
No way of doing this on a live database

You need a separate database for tests. This may be a mock of database actually.

Then, before the tests, you need to prepare the database to be in the fixed state you expect it to be (e.g. load data from fixtures or import sql file).

Then, the best option is start database transaction when you plan to do any modification to the original database state.

Once the transaction is started, you may work on the database.

After you complete the tests, you just rollback the transaction, and have database in the clean state.

Sqlite (already mentioned) will do fine in most cases, but it may differ from the live database. Anyway, having different database adapters will be very helpful.