Why doesn't Data.Text.Lazy.replace and Data.Text.Lazy.append work?

Question

In the following code are three different ways (str1, str2, and str3) of replacing a string using Data.Text.Lazy.replace. They should give the same output.

import Data.Text.Lazy as DTL

str1 :: String
str1 = DTL.unpack $ DTL.replace key value text
  where key = DTL.pack "<<name>>"
        value = DTL.pack "Joyce"
        text = DTL.pack "Hello, <<name>>."

str2 :: String
str2 = DTL.unpack $ DTL.replace key value text
  where key = DTL.pack "<<" `DTL.append` DTL.pack "name" 
                            `DTL.append` DTL.pack ">>"
        value = DTL.pack "Joyce"
        text = DTL.pack "Hello, <<name>>."

str3 :: String
str3 = DTL.unpack $ DTL.replace key value text
  where key = DTL.pack $ "<<" ++ "name" ++ ">>"
        value = DTL.pack "Joyce"
        text = DTL.pack "Hello, <<name>>."

main :: IO ()
main = do putStrLn str1
          putStrLn str2
          putStrLn str3

However the outcome of running the program is:

Hello, Joyce.
Hello, <<name>>.
Hello, Joyce.

Why does str2 not work correctly? Is there anything wrong with the code?

Answer 1

Thanks for the bug report, guys. I'll look into it, and follow up here with what I find.

Answer 2

Looks like a bug in the text library to me.

(I've added an issue to the bug-tracker in case the author doesn't happen to visit Stack Overflow.)

Answer 3

It was a bug in the search code: on chunk boundaries of the pattern, one character (well, code unit) was omitted in the creation of the skip-table, so on certain inputs, the search window was moved too far. I've sent Bryan a pull request for the fix.

Since your patterns are composed of short literals, had you compiled with optimisations, you wouldn't have located the bug. Good find.