Why isn't the Prelude's words function written more simply?

Question

Consider the words Prelude function; it is really easy and one could write it in the following manner:

words' :: String -> [String]
words' [] = []
words' str = before : words' (dropWhile isSpace after) where
    (before, after) = break isSpace str

However, I noticed that its original Prelude code seems much less... natural:

words                   :: String -> [String]
words s                 =  case dropWhile {-partain:Char.-}isSpace s of
                                "" -> []
                                s' -> w : words s''
                                      where (w, s'') =
                                             break {-partain:Char.-}isSpace s'

I assume that there are optimization-related reasons for it. The question is: am I wrong to expect that the compiler should optimize the words' function just as well as its Prelude version? I did use the same functions (break, dropWhile, isSpace).

I was once very surprised that GHC did not perform some of the simplest low-level optimizations:

C vs Haskell Collatz conjecture speed comparison

but aside for the {-partain:Char.-} bits (this hint for the compiler does not seem very helpful in this situation IMO) the words code seems unnecesarily bloated for a high-level language. What is the reason behind it in this case?

Answer 1

This is nearly exactly the same code. The only difference is if we're doing the dropWhile isSpace before every call or only the recursive call. Neither is more complex than the other, but rather the latter (Prelude) version seems more verbose because the pattern matching isn't directly in the function.

You can observe the difference (and why the Prelude version has better behavior) like so:

*Main> words "    "
[]
*Main> words' "     "
[""]

Note that you can quickly verify if your "improved" versions are the same as the originals using QuickCheck.