Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Idiomatic Clojure way to find most frequent items in a seq

Tags:

idioms

clojure

Given a sequence of items I want to find the n most frequent items, in descending order of frequency. So for example I would like this unit test to pass:

(fact "can find 2 most common items in a sequence"
      (most-frequent-n 2 ["a" "bb" "a" "x" "bb" "ccc" "dddd" "dddd" "bb" "dddd" "bb"]) 
      =>
      '("bb" "dddd"))

I am fairly new to Clojure and still trying to get to grip with the standard library. Here is what I came up with:

(defn- sort-by-val [s]        (sort-by val s))
(defn- first-elements [pairs] (map #(get % 0) pairs))

(defn most-frequent-n [n items]
  "return the most common n items, e.g. 
     (most-frequent-n 2 [:a :b :a :d :x :b :c :d :d :b :d :b])  => 
         => (:d :b)"
  (take n (->
           items               ; [:a :b :a :d :x :b :c :d :d :b :d :b]
           frequencies         ; {:a 2, :b 4, :d 4, :x 1, :c 1}
           seq                 ; ([:a 2] [:b 4] [:d 4] [:x 1] [:c 1])
           sort-by-val         ; ([:x 1] [:c 1] [:a 2] [:b 4] [:d 4])
           reverse             ; ([:d 4] [:b 4] [:a 2] [:c 1] [:x 1])
           first-elements)))   ; (:d :b :a :c :x)

However this seems like a complicated chain of functions to do a fairly common operation. Is there a more elegant or more idiomatic (or more efficient) way to do this?

like image 382
Eamonn O'Brien-Strain Avatar asked Sep 30 '12 00:09

Eamonn O'Brien-Strain


1 Answers

As you have discovered, typically you would use a combination of sort-by and frequencies to get a frequency-sorted list.

(sort-by val (frequencies ["a" "bb" "a" "x" "bb" "ccc" "dddd" "dddd" "bb" "dddd" "bb"]))
=> (["x" 1] ["ccc" 1] ["a" 2] ["dddd" 3] ["bb" 4])

Then you can manipulate this fairly easily to get the lowest / highest frequency items. Perhaps something like:

(defn most-frequent-n [n items]
  (->> items
    frequencies
    (sort-by val)
    reverse
    (take n)
    (map first)))

Which again is pretty similar to your solution (apart from that you don't need the helper functions with clever use of the ->> macro).

So overall I think your solution is pretty good. Don't worry about the chain of functions - it's actually a very short solution for what is logically quite a complicated concept. Try coding the same thing in C# / Java and you will see what I mean......

like image 150
mikera Avatar answered Sep 18 '22 14:09

mikera