How do I get distinct items from a list?

Question

It's not clear to me how to retrieve distinct items from a list.

I have the following code:

topicsFromLinks : List Link -> List Topic
topicsFromLinks links =
    links
        |> List.map (\l -> l.topics)
        |> List.concat
        |> Set.fromList
        |> Set.toList

Error:

The definition of topicsFromLinks does not match its type annotation. - The type annotation for topicsFromLinks says it always returns:

List Topic

But the returned value (shown above) is a:

List comparable

I expect the following lines to just work in regards to structural equality:

|> Set.fromList
|> Set.toList

Why am I receiving a List of Comparables?

How do I resolve this compilation error?

Appendix:

type alias Topic =
    { name : String, isFeatured : Bool }

type alias Link =
    {
    ...
    , topics : List Topic
    ...
    }

Answer 1

According to the documentation for Set:

The values can be any comparable type. This includes Int, Float, Time, Char, String, and tuples or lists of comparable types.

You are attempting to put a Topic value in where only a comparable type will work.

Thankfully there is the elm-community/list-extra package which exposes a uniqueBy function that lets you specify your own function to turn something into a comparable.

If you want to get the distinct list of topics, matching both on the name and isFeatured fields, then you can use toString to serialize the value into something comparable:

import List.Extra exposing (uniqueBy)

topicsFromLinks : List Link -> List Topic
topicsFromLinks links =
    links
        |> List.map .topics
        |> List.concat
        |> uniqueBy toString

Answer 2

In 2021, rather than using an external library and writing a clumsy toString method (since Debug.toString isn't available for production use anymore), consider using a fold:

unique : List a -> List a
unique list =
    List.foldl
        (\a uniques ->
            if List.member a uniques then
                uniques

            else
                uniques ++ [ a ]
        )
        []
        list