How do you know what to test when writing unit tests? [closed]

Question

Many great responses to this are also on my question: "Beginning TDD - Challenges? Solutions? Recommendations?"

May I also recommend taking a look at my blog post (which was partly inspired by my question), I have got some good feedback on that. Namely:

I Don’t Know Where to Start?

• Start afresh. Only think about writing tests when you are writing new code. This can be re-working of old code, or a completely new feature.
• Start simple. Don’t go running off and trying to get your head round a testing framework as well as being TDD-esque. Debug.Assert works fine. Use it as a starting point. It doesn’t mess with your project or create dependencies.
• Start positive. You are trying to improve your craft, feel good about it. I have seen plenty of developers out there that are happy to stagnate and not try new things to better themselves. You are doing the right thing, remember this and it will help stop you from giving up.
• Start ready for a challenge. It is quite hard to start getting into testing. Expect a challenge, but remember – challenges can be overcome.

Only Test For What You Expect

I had real problems when I first started because I was constantly sat there trying to figure out every possible problem that could occur and then trying to test for it and fix. This is a quick way to a headache. Testing should be a real YAGNI process. If you know there is a problem, then write a test for it. Otherwise, don’t bother.

Only Test One Thing

Each test case should only ever test one thing. If you ever find yourself putting “and” in the test case name, you’re doing something wrong.

I hope this means we can move on from "getters and setters" :)

Test your code, not the language.

A unit test like:

Integer i = new Integer(7);
assert (i.instanceOf(integer));

is only useful if you are writing a compiler and there is a non-zero chance that your instanceof method is not working.

Don't test stuff that you can rely on the language to enforce. In your case, I'd focus on your authenticate and save methods - and I'd write tests that made sure they could handle null values in any or all of those fields gracefully.

This got me into unit testing and it made me very happy

We just started to do unit testing. For a long time I knew it would be good to start doing it but I had no idea how to start and more importantly what to test.

Then we had to rewrite an important piece of code in our accounting program. This part was very complex as it involved a lot of different scenarios. The part I'm talking about is a method to pay sales and/or purchase invoices already entered into the accounting system.

I just didn't know how to start coding it, as there were so many different payment options. An invoice could be $100 but the customer only transferred $99. Maybe you have sent sales invoices to a customer but you have also purchased from that customer. So you sold him for $300 but you bought for $100. You can expect your customer to pay you $200 to settle the balance. And what if you sold for $500 but the customer pays you only $250?

So I had a very complex problem to solve with many possibilities that one scenario would work perfectly but would be wrong on an other type of invocie/payment combination.

This is where unit testing came to the rescue.

I started to write (inside the test code) a method to create a list of invoices, both for sales and purchases. Then I wrote a second method to create the actual payment. Normally a user would enter that information through a user interface.

Then I created the first TestMethod, testing a very simple payment of a single invoice without any payment discounts. All the action in the system would happen when a bankpayment would be saved to the database. As you can see I created an invoice, created a payment (a bank transaction) and saved the transaction to disk. In my asserts I put what should be the correct numbers ending up in the Bank transaction and in the linked Invoice. I check for the number of payments, the payment amounts, the discount amount and the balance of the invoice after the transaction.

After the test ran I would go to the database and double check if what I expected was there.

After I wrote the test, I started coding the payment method (part of the BankHeader class). In the coding I only bothered with code to make the first test pass. I did not yet think about the other, more complex, scenarios.

I ran the first test, fixed a small bug until my test would pass.

Then I started to write the second test, this time working with a payment discount. After I wrote the test I modified the payment method to support discounts.

While testing for correctness with a payment discount, I also tested the simple payment. Both tests should pass of course.

Then I worked my way down to the more complex scenarios.

1) Think of a new scenario

2) Write a test for that scenario

3) Run that single test to see if it would pass

4) If it didn't I'd debug and modify the code until it would pass.

5) While modifying code I kept on running all tests

This is how I managed to create my very complex payment method. Without unit testing I did not know how to start coding, the problem seemed overwhelming. With testing I could start with a simple method and extend it step by step with the assurance that the simpler scenarios would still work.

I'm sure that using unit testing saved me a few days (or weeks) of coding and is more or less guaranteeing the correctness of my method.

If I later think of a new scenario, I can just add it to the tests to see if it is working or not. If not I can modify the code but still be sure the other scenarios are still working correctly. This will save days and days in the maintenance and bug fixing phase.

Yes, even tested code can still have bugs if a user does things you did not think of or prevented him from doing

Below are just some of tests I created to test my payment method.

public class TestPayments
{
    InvoiceDiaryHeader invoiceHeader = null;
    InvoiceDiaryDetail invoiceDetail = null;
    BankCashDiaryHeader bankHeader = null;
    BankCashDiaryDetail bankDetail = null;



    public InvoiceDiaryHeader CreateSales(string amountIncVat, bool sales, int invoiceNumber, string date)
    {
        ......
        ......
    }

    public BankCashDiaryHeader CreateMultiplePayments(IList<InvoiceDiaryHeader> invoices, int headerNumber, decimal amount, decimal discount)
    {
       ......
       ......
       ......
    }


    [TestMethod]
    public void TestSingleSalesPaymentNoDiscount()
    {
        IList<InvoiceDiaryHeader> list = new List<InvoiceDiaryHeader>();
        list.Add(CreateSales("119", true, 1, "01-09-2008"));
        bankHeader = CreateMultiplePayments(list, 1, 119.00M, 0);
        bankHeader.Save();

        Assert.AreEqual(1, bankHeader.BankCashDetails.Count);
        Assert.AreEqual(1, bankHeader.BankCashDetails[0].Payments.Count);
        Assert.AreEqual(119M, bankHeader.BankCashDetails[0].Payments[0].PaymentAmount);
        Assert.AreEqual(0M, bankHeader.BankCashDetails[0].Payments[0].PaymentDiscount);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[0].InvoiceHeader.Balance);
    }

    [TestMethod]
    public void TestSingleSalesPaymentDiscount()
    {
        IList<InvoiceDiaryHeader> list = new List<InvoiceDiaryHeader>();
        list.Add(CreateSales("119", true, 2, "01-09-2008"));
        bankHeader = CreateMultiplePayments(list, 2, 118.00M, 1M);
        bankHeader.Save();

        Assert.AreEqual(1, bankHeader.BankCashDetails.Count);
        Assert.AreEqual(1, bankHeader.BankCashDetails[0].Payments.Count);
        Assert.AreEqual(118M, bankHeader.BankCashDetails[0].Payments[0].PaymentAmount);
        Assert.AreEqual(1M, bankHeader.BankCashDetails[0].Payments[0].PaymentDiscount);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[0].InvoiceHeader.Balance);
    }

    [TestMethod]
    [ExpectedException(typeof(ApplicationException))]
    public void TestDuplicateInvoiceNumber()
    {
        IList<InvoiceDiaryHeader> list = new List<InvoiceDiaryHeader>();
        list.Add(CreateSales("100", true, 2, "01-09-2008"));
        list.Add(CreateSales("200", true, 2, "01-09-2008"));

        bankHeader = CreateMultiplePayments(list, 3, 300, 0);
        bankHeader.Save();
        Assert.Fail("expected an ApplicationException");
    }

    [TestMethod]
    public void TestMultipleSalesPaymentWithPaymentDiscount()
    {
        IList<InvoiceDiaryHeader> list = new List<InvoiceDiaryHeader>();
        list.Add(CreateSales("119", true, 11, "01-09-2008"));
        list.Add(CreateSales("400", true, 12, "02-09-2008"));
        list.Add(CreateSales("600", true, 13, "03-09-2008"));
        list.Add(CreateSales("25,40", true, 14, "04-09-2008"));

        bankHeader = CreateMultiplePayments(list, 5, 1144.00M, 0.40M);
        bankHeader.Save();

        Assert.AreEqual(1, bankHeader.BankCashDetails.Count);
        Assert.AreEqual(4, bankHeader.BankCashDetails[0].Payments.Count);
        Assert.AreEqual(118.60M, bankHeader.BankCashDetails[0].Payments[0].PaymentAmount);
        Assert.AreEqual(400, bankHeader.BankCashDetails[0].Payments[1].PaymentAmount);
        Assert.AreEqual(600, bankHeader.BankCashDetails[0].Payments[2].PaymentAmount);
        Assert.AreEqual(25.40M, bankHeader.BankCashDetails[0].Payments[3].PaymentAmount);

        Assert.AreEqual(0.40M, bankHeader.BankCashDetails[0].Payments[0].PaymentDiscount);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[1].PaymentDiscount);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[2].PaymentDiscount);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[3].PaymentDiscount);

        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[0].InvoiceHeader.Balance);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[1].InvoiceHeader.Balance);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[2].InvoiceHeader.Balance);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[3].InvoiceHeader.Balance);
    }

    [TestMethod]
    public void TestSettlement()
    {
        IList<InvoiceDiaryHeader> list = new List<InvoiceDiaryHeader>();
        list.Add(CreateSales("300", true, 43, "01-09-2008")); //Sales
        list.Add(CreateSales("100", false, 6453, "02-09-2008")); //Purchase

        bankHeader = CreateMultiplePayments(list, 22, 200, 0);
        bankHeader.Save();

        Assert.AreEqual(1, bankHeader.BankCashDetails.Count);
        Assert.AreEqual(2, bankHeader.BankCashDetails[0].Payments.Count);
        Assert.AreEqual(300, bankHeader.BankCashDetails[0].Payments[0].PaymentAmount);
        Assert.AreEqual(-100, bankHeader.BankCashDetails[0].Payments[1].PaymentAmount);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[0].InvoiceHeader.Balance);
        Assert.AreEqual(0, bankHeader.BankCashDetails[0].Payments[1].InvoiceHeader.Balance);
    }

If they really are trivial, then don't bother testing. Eg, if they are implemented like this;

public class User
{
    public string Username { get; set; }
    public string Password { get; set; }
}

If, on the other hand, you are doing something clever, (like encrypting and decrypting the password in the getter/setter) then give it a test.

The rule is that you have to test every piece of logic you write. If you implemented some specific functionality in the getters and setters I think they are worth testing. If they only assign values to some private fields, don't bother.

This question seems to be a question of where does one draw the line on what methods get tested and which don't.

The setters and getters for value assignment have been created with consistency and future growth in mind, and foreseeing that some time down the road the setter/getter may evolve into more complex operations. It would make sense to put unit tests of those methods in place, also for the sake of consistency and future growth.

Code reliability, especially while undergoing change to add additional functionality, is the primary goal. I am not aware of anyone ever getting fired for including setters/getters in the testing methodology, but I am certain there exists people who wished they had tested methods which last they were aware or can recall were simple set/get wrappers but that was no longer the case.

Maybe another member of the team expanded the set/get methods to include logic that now needs tested but didn't then create the tests. But now your code is calling these methods and you aren't aware they changed and need in-depth testing, and the testing you do in development and QA don't trigger the defect, but real business data on the first day of release does trigger it.

The two teammates will now debate over who dropped the ball and failed to put in unit tests when the set/gets morphed to include logic that can fail but isn't covered by a unit test. The teammate that originally wrote the set/gets will have an easier time coming out of this clean if the tests were implemented from day one on the simple set/gets.

My opinion is that a few minutes of "wasted" time covering ALL methods with unit tests, even trivial ones, might save days of headache down the road and loss of money/reputation of the business and loss of someone's job.

And the fact that you did wrap trivial methods with unit tests might be seen by that junior team mate when they change the trivial methods into non-trivial ones and prompt them to update the test, and now nobody is in trouble because the defect was contained from reaching production.

The way we code, and the discipline that can be seen from our code, can help others.