remove duplicates in a text file in haskell

Question

I am trying to remove the duplicates from a text file and i tried a code like following;

import Data.List
main = do  
    let singlewords = []
    handle <- readFile "/tmp/foo.txt" 
    singlewords = words handle
    nub singlewords

It gives an error of course since i am very new to haskell and i've been making some exercises but still i think i have some more time to get used to that. I would really be very appreciated for your help.

Answer 1

Your code fixed:

import Data.List

main = do
    -- let singlewords = [] -- this line removed
    content <- readFile "/tmp/foo.txt"
    let singlewords = words content -- added the let keyword
    return (nub singlewords) -- added call to return

On the first line you write let singlewords = [] and then later try to assign a new value to singlewords. This is not how we do it in Haskell, there is no need to "declare" or "define" names before we use them.

In Haskell we separate effectful computations (IO is one type of effectful computations) from pure computations. We bind results of effectful computations using

name <- computation

and we bind results of pure computations using

let name = computation

when in a do-block.

The last line in a do-block is what the whole block will compute, and must therefore be an effectful computation. In you example you want to return the result of a pure computation and must therefore lift the result to an effectful one, we do that using return.

---

To see your single words you want to output them to the console, there are a couple of functions to do so: https://hackage.haskell.org/package/base-4.8.1.0/docs/Prelude.html#g:27.

The simplest would be to output the list singlewords using print:

main = do
    content <- readFile "/tmp/foo.txt"
    let singlewords = nub (words content)
    print singlewords