Basis for claim that the number of bugs per line of code is constant regardless of the language used

Question

I've heard people say (although I can't recall who in particular) that the number of bugs per line of code is roughly constant regardless of what language is used. What is the research that backs this up?

Edited to add: I don't have access to it, but apparently the authors of this paper "asked the question whether the number of bugs per lines of code (LOC) is the same for programs written in different programming languages or not."

Answer 1

In his book Code Complete (quoting from the 2nd Edition), in the chapter "Developer Testing," Steve McConnell cites a handful of studies across a variety of languages:

• Industry average experience is about 1-25 errors per 1000 lines of code for delivered software. The software has usually been developed using a hodgepodge of techniques (Boehm 1981, Gremillion 1984, Yourdon 1989a, Jones 1998, Jones 2000, Weber 2003). Cases that have one-tenth as many errors as this are rare; cases that have 10 times more tend not to be reported. (They probably aren't ever completed!)

• The Applications Division at Microsoft experiences about 10–20 defects per 1000 lines of code during in-house testing and 0.5 defects per 1000 lines of code in released product (Moore 1992). The technique used to achieve this level is a combination of the code-reading techniques described in Other Kinds of Collaborative Development Practices, and independent testing.

• Harlan Mills pioneered "cleanroom development," a technique that has been able to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defects per 1000 lines of code in released product (Cobb and Mills 1990).

These studies ranged from high-level languages like Java, down to C++ and C, all the way down to assembly. Considering the massive impact of Code Complete on software engineering as a discipline, I suspect it is responsible for popularizing this idea.

Answer 2

One possible source would be Les Hatton's 1995 paper "Computer programming languages and safety-related systems", in which he concludes that language choice is at least close to irrelevant and other factors (chiefly fluency in the chosen language) are the controlling factors.

About all I could add to that would be to summarize various other papers, in which defect rates for individual projects (and such) are given. I've done a bit of looking, and never found a correlation between language and defect rate, but that's not really the same as saying the defect rate is constant across languages (i.e., they may be different, but they vary so widely within each language that I've never been able to prove a difference).