Understanding how software testing works and what to test [closed]

Question

Intro:
I've seen lots of topics here on SO about software testing and other terms I don't understand.

Problem:
As a beginner developer I, unfortunately, have no idea how software testing works, not even how to test a simple function. This is a shame, but thats the truth. I also hope this question can help others beginners developers too.

Question:
Can you help me to understand this subject a little bit more?

Maybe some questions to start would help:

• When I develop a function, how should I test it? For example: when working with a sum function, should I test every input value possible or just some limits? How about testing functions with strings as parameters?
• In a big program, do I have to test every single piece of code of it? When you guys program do you test every code written?
• How automated test works and how can I try one? How tools for automated testing works and what they do?
• I`ve heard about unit testing. Can I have a brief explanation on this?
• What is a testing framework?

If possible please post some code with examples to clarify the ideas.

Any help on this topic is very welcome! Thanks.

Answer 1

Let's start with the obvious:

How does testing work ? In test-driven development you first think about the functionality you want to implement and then write a test for it. In the example you gave of a sum function it is quite obvious what it should do. You then write a test that makes sure that the summation worked.

Unit tests should be as lightweight as possible so you can run them each time you hit the build button. When you do this now in this example your test will fail, because you have not yet implemented a sum function.

Now you write the actual function and continue debugging and implementing until the test passes. Then you are sure you have implemented the feature you wanted.

Now how should you design your test ? You cannot test all, that is impossible. As an example lets say you take user input you have to validate. Then it would be a natural thing to write at least two test cases for that validation code: one that makes sure that valid input is parsed as such. The second test case gets invalid input and you make sure that it fails, raises an exception or whatever behavior you desired. So in this case it is good to have a positive test which is expected to pass and a negative test which checks if invalid input is not validated.

When should one test? As I mentioned before the test should be lightweight enough so that they can be run on each build. And yes, run all of them. This makes sure that you do not miss a dependency in your code that breaks things far away from the point you edited.

Can anything be tested ? Well, usually methods that rely on external ressources are hard to test. What I mean by that are databases, network connections or specific hardware and drivers. It can be done but then you have to set up a larger test setup.

Bugfixing and tests A typical scenario where test get really useful is if you are squeezing bugs. Not literally of course. If you have a bug you have to fix, first try to write a test for it. And then fix your code until your test passes. From this point on this test "watches over your code" that this bug will never come back again.

What do you gain by testing ? In my opinion there are many things

1. More modular, easier to maintain code because it has to be testable
2. Confidence. Having a code base that is largely tested gives you the confidence that it works as expected and stays like this.
3. You find bugs early. This means you can fix them more easily.

It takes some effort to get used to using tests but I think it is worth it. Especially if you are writing libraries of some kind.

Answer 2

Well, Usually, there're three kinds of tests. Unit Tests, System tests and QA tests. Unit tests, as the name suggests, test small units - separate functions and classes.

For all modern development environments there're unit test frameworks. There's Nunit for .net, as well as MS unit test framework in Visual Studio, CPPUnit for C++, JUnit, etc. All meant for one thing: connect to parts of your program, run your pre-defined scripts and report error.

CPPUnit, for example, is based on macros like CPPUNIT_ASSERT_EQUAL, meant to be used as something like this: CPPUNIT_ASSERT_EQUAL(sum(arr), 17). And it would say if it's not equal, in which case the test will be considered failed.

You are supposed to develop the tests for every function, and after that - you are not afraid to change and optimize the code. It is generally referred to a "repeatability" - ability to do a complex action, such as full testing of all codebase, with a single click.

Unit tests are required for every moden software development, because the experience shows that they improve development speed. It is also suggested that unit test code may serve as a kind of "documentation" for the library.

System tests are automated tests of larger, high-level functionality. The idea of system tests is to feed clean input (such as databases, user input, etc) to test the whole thing, validating the output against pre-defined resutls. It is essential that the system ouput is deterministic, and depends only on the input. Every time the system changes, the system tests change also.

TDD is a cool-sounding bad idea, suggesting that you should not develop anything before implementing the proper automated tests, and then writing code to satisfy the tests. It is regardedd as failure, because changes in the design are inevitable during development, and a small design change usually causes drastic changes in the unit tests.

Manual QA is the final, and most important type of software testing. The idea is to prepare a test plan, whcih is done during design and coding phases, collecting all ideas developers had during coding of every if statement, how to actually make this particular if statement run along the less expected code path. The QA personnel, meant to be capable of anything that can be done with the program without development environment, can follow the resulting test procedure and their own ideas, to find more bugs.