Unit testing a script that opens a file

Question

I've written a script that opens up a file, reads the content and does some operations and calculations and stores them in sets and dictionaries.

How would I write a unit test for such a thing? My questions specifically are:

1. Would I test that the file opened?
2. The file is huge (it's the unix dictionary file). How would I unit test the calculations? Do I literally have to manually calculate everything and test that the result is right? I have a feeling that this defeats the whole purpose of unit testing. I'm not taking any input through stdin.

Answer 1

That's not what unit-testing is about!

1. Your file doesn't represent an UNIT, so no you don't test the file or WITH the file!
2. your unit-test should test every single method of your functions/methods which deals with the a)file-processing b) calculations
3. it's not seldom that your unit-tests exceeds the line of code of your units under test.

Unit-test means (not complete and not the by-the-book definition):

• minimalistic/atomic - you split your units down to the most basic/simple unit possible; an unit is normally a callable (method, function, callable object)
• separation of concern - you test ONE and only ONE thing in every single test; if you want to test different conditions of a single unit, you write different tests
• determinism - you give the unit something to process, with the beforehand knowledge of what it's result SHOULD be
• if your unit-under-test needs a specific enviroment you create a fixture/test-setup/mock-up
• unit-tests are (as a rule of thumb) blazingly fast! if it's slow check if you violated another point from above
• if you need to test somethin which violates somethin from above you may have made the next step in testing towards integration-tests
• you may use unit-test frameworks for not unit-testings, but don't call it unit-test just because of the use of the unittest-framework

This guy (Gary Bernhardt) has some interesting practical examples of what testing and unit-testing means.

Update for some clarifications:

"1. Would I test that the file opened?"

Well you could do that, but what would be the "UNIT" for that? Keep in mind, that a test has just two solutions: pass and fail. If your test fails, it should (ideally must) have only one reason for that: Your unit(=function) sucks! But in this case your test can fail, because: * the file doesn't exist * is locked * is corrupted * no file-handles left * out of memeory (big file) * moon- phase and so on.

so what would a failing (or passing) "unit" test say about your unit? You don't test your unit alone, but the whole surrounding enviroment with it. That's more a system-test! If you would like to test nontheless for successful file-opening you should at least mock a file.

"2 ... How would I unit test the calculations? Do I literally have to manually calculate everything and test that the result is right?"

No. You would write test for the corner- and regular-cases and check the expected outcome against the processed one. The amount of tests needed depends on the complexity of your calculations and the exceptions to the rule.

e.g.:

def test_negative_factor(self):
   assert result 

def test_discontinuity(self):
   assert raise exception if x == undefined_value

I hope i made myself clearer!

Answer 2

You should refactor your code to be unit-testable. That, on the top of my head, would say:

1. Take the functionality of the file opening into a separate unit. Make that new unit receive the file name, and return the stream of the contents.
2. Make your unit receive a stream and read it, instead of opening a file and reading it.
3. Write a unit test for your main (calculation) unit. You would need to mock a stream, e.g. from a dictionary. Write several test cases, each time provide your unit with a different stream, and make sure your unit calculates the correct data for each input.
4. Get your code coverage as close to 100% as you can. Use nosetests for coverage.
5. Finally, write a test for your 'stream provider'. Feed it with several files (store them in your test folder), and make sure your stream provider reads them correctly.
6. Get the second unit test coverage as close to 100% as you can.
7. Now, and only now, commit your code.