Should LOC counting include tests and comments?

Question

While LOC (# lines of code) is a problematic measurement of a code's complexity, it is the most popular one, and when used very carefully, can provide a rough estimate of at least relative complexities of code bases (i.e. if one program is 10KLOC and another is 100KLOC, written in the same language, by teams of roughly the same competence, the second program is almost certainly much more complex).

When counting lines of code, do you prefer to count comments in ? What about tests?

I've seen various approaches to this. Tools like cloc and sloccount allow to either include or exclude comments. Other people consider comments part of the code and its complexity.

The same dilemma exists for unit tests, that can sometimes reach the size of the tested code itself, and even exceed it.

I've seen approaches all over the spectrum, from counting only "operational" non-comment non-blank lines, to "XXX lines of tested, commented code", which is more like running "wc -l on all code files in the project".

What is your personal preference, and why?

Answer 1

A wise man once told me 'you get what you measure' when it comes to managing programmers.

If you rate them in their LOC output amazingly you tend to get a lot of lines of code.

If you rate them on the number of bugs they close out, amazingly you get a lot of bugs fixed.

If you rate them on features added, you get a lot of features.

If you rate them on cyclomatic complexity you get ridiculously simple functions.

Since one of the major problems with code bases these days is how quickly they grow and how hard they are to change once they've grown, I tend to shy away from using LOC as a metric at all, because it drives the wrong fundamental behavior.

That said, if you have to use it, count sans comments and tests and require a consistent coding style.

But if you really want a measure of 'code size' just tar.gz the code base. It tends to serve as a better rough estimate of 'content' than counting lines which is susceptible to different programming styles.