Motivation for Scala underscore in terms of formal language theory and good style?

Question

Why is it that many people say that using underscore is good practice in Scala and makes your code more readable? They say the motivation comes from formal language theory. Nevertheless many programmers, particularly from other languages, especially those that have anonymous functions, prefer not to use underscores particularly for placeholders.

So what is the point in the underscore? Why does Scala (and some other functional languages as pointed by om-nom-nom) have the underscore? And what is the formal underpinning, in terms of complexity and language theory, as to why it often good style to use it?

Answer 1

Linguistics

The origin and motivation for most of the underscore uses in Scala is to allow one to construct expressions and declarations without the need to always give every variable (I mean "variable" as in Predicate Calculus, not in programming) of the language a name. We use this all the time in Natural Language, for example I referred to a concept in the previous sentence in this sentence using "this" and I referred to this sentence using "this" without there being any confusion over what I mean. In Natural Language these words are usually called "pronouns", "anaphors", "cataphors", the referents "antecedent" or "postcedent", and the process of understanding/dereferencing them is called "anaphora".

Algorithmic Information Theory

If we had to name every 'thing' in Natural Language before we can refer to it, similarly every type of thing in order to quantify over it, as in Predicate Calculus and in most programming languages, then speaking would become extremely long winded. It is thanks to context that we can infer what is meant by words like "this", "it", "that", etc, we do it easily.

Therefore why restrict this simple, elegant and efficient means to communicate to Natural Language? So it was added to Scala.

If we did attempt to name every single 'thing' or 'type of thing', sentences become so long and complicated that it becomes very difficult to understand due to it's verbosity and the introduction of redundant symbols. The more symbols you add to a sentence the more difficult it becomes to understand, ergo this is why it's good practice, not only in Natural Language, but in Scala too. In fact one could formalize this assertion in terms of Kolmogorov Complexity and prove that a sequence of sentences adopting placeholders have lower complexity than those that unnecessarily name everything (unless the name is exactly the same in every instance, but that usually doesn't make sense). Therefore we can conclusively say contrary to some programmers belief, that the placeholder syntax is simpler and easier to read.

The reason why it has some resistance in it's use, is that if one is already a programmer, one must make an effort to retrain the brain not to name everything, just as (if they can remember) they may have found learning to code in the first place required quite an effort.

Examples

Now let's look at some specific uses more formally:

Placeholder Syntax

Means "it", "them", "that", "their" etc (i.e. pronouns), e.g. 1

lines.map(_.length)

can be read as "map lines to their length", similarly we can read lineOption.map(_.length) as "map the line to it's length". In terms of complexity theory, this is simpler than "for each 'line' in lines, take the length of 'line'" - which would be lines.map(line => line.length).

Can also be read as "the" (definite article) when used with type annotation, e.g.

(_: Int) + 1

"Add 1 to the integer"

Existential Types

Means "of some type" ("some" the pronoun), e.g

foo: Option[_]

means "foo is an Option of some type".

Higher Kinded type parameters

Again, basically means "of some type" ("some" the pronoun), e.g.

class A[K[_],T](a: K[T])

Can be read "class A takes some K of some type ..."

Pattern Match Wildcards

Means "anything" or "whatever" (pronouns), e.g.

case Foo(_) => "hello"

can be read as "for a Foo containing anything, return 'hello'", or "for a Foo containing whatever, return 'hello'"

Import Wildcards

Means "everything" (pronoun), e.g.

import foo._

can be read as "import everything from foo".

Default Values

Now I read this like "a" (indefinite article), e.g.

val wine: RedWine = _

"Give me a red wine", the waiter should give you the house red.

Other uses of underscore

The other uses of underscores are not really related to the point of this Q&A, nevertheless we breifly discuss them

Ignored Values/Params/Extractions

Allow us to ignore things in an explicit 'pattern safe' way. E.g.

val (x, _) = getMyPoint

Says, we are not going to use the second coordinate, so no need to get freaky when you cant find a use in the code.

Import Hidding

Just a way to say "except" (preposition).

Function Application

E.g.

val f: String => Unit = println _

This is an interesting one as it has an exact analogue in linguistics, namely nominalization, "the use of a verb, an adjective, or an adverb as the head of a noun phrase, with or without morphological transformation" - wikipedia. More simply it is the process of turning verbs or adjectives into nouns.

Use in special method names

Purely a syntax thing and doesn't really relate to linguistics.