who wrote 250k tests for webkit?

Question

assuming a yield of 3 per hour, that's 83000 hours. 8 hours a day makes 10,500 days, divide by thirty to get 342 mythical man months. I call them mythical because writing 125 tests per person per week is unreal.

can any wise soul out there on SO shed some light on what sort of mythical men write unreal quantities of tests for large software projects? thank you.

update chrisw thinks there are only 20k tests (check out his explanation below).
PS I'd really like to hear from folks who have worked on projects with large test bases

Answer 1

The original directory contained 240K files:

 Total Files Listed:        243541 File(s)  1,062,470,729 bytes        64718 Dir(s) 

Many of these are svn files. If I remove all the subdirectories named ".svn" then the number of files drops to 90K:

 Total Files Listed:        90615 File(s)    537,457,618 bytes         7190 Dir(s)  

Some directories have a subdirectory named "resources" and/or "script-tests". I think that these subdirectories contain helper files which are used by test cases in the superdirectories. If I remove these subdirectories (because they don't add to the total number of tests) then the number of files drops to 87K :

 Total Files Listed:        87672 File(s)    534,598,610 bytes         6305 Dir(s) 

Condensing 'similar' filenames (e.g. "arrow-keys-on-body.html" and "arrow-keys-on-body-expected.txt" are two files which define a single test) reduces the total number from 87K to 43K.

The only subdirectories which contain more than 1500 of these test cases (counted as described above) are:

2761   LayoutTests\dom 10330  LayoutTests\fast (of which 5934 are in LayoutTests\fast\js) 22575  LayoutTests\platform (with various O/S-specific subdirectories). 

Within the platform subdirectories there seems to have been some copy-and-pasting between platforms. For example, the css3-modsel-37-expected.txt file exists:

• In the LayoutTests\platform\mac\css3 subdirectory
• In the LayoutTests\platform\chromium-win\css3 subdirectory
• In the LayoutTests\platform\qt\css3 subdirectory.

If I discard filenames which are duplicated into several platform subdirectories, then there are only 5716 (instead of 22575) unique platform tests.

In summary I think there are about 18K unique tests: which is still an impressive number of tests, but fewer than the 250K that you had estimated in your OP.

---

By way of comparison, I recently found CSS2.1 Test Suite: that looks like about 9000 test cases for CSS.

Answer 2

I just wrote 4 unit tests in 5 minutes. Not all unit tests takes long to make. Some are very simple ;)