Stripping duplicate elements in a list of strings in elisp

Question

Given a list such as

(list "foo" "bar" nil "moo" "bar" "moo" nil "affe")

how would I build a new list with the duplicate strings removed, as well as the nils stripped, i.e.

(list "foo" "bar" "moo" "affe")

The order of the elements needs to be preserved - the first occurence of a string may not be removed.

The lists I'm dealing with here are short, so there's no need to use anything like a hash table for the uniqueness check, although doing so certainly wouldn't hurt either. However, using cl functionality is not a viable option.

Answer 1

Try "Sets and Lists" in the "Lists" section of the Emacs Lisp Reference Manual:

(delq nil (delete-dups (list "foo" "bar" nil "moo" "bar" "moo" nil "affe")))

Answer 2

The Common Lisp package contains many list manipulation functions, in particular remove-duplicates.

(require 'cl)
(remove-duplicates (list "foo" "bar" nil "moo" "bar" "moo" nil "affe")
                   :test (lambda (x y) (or (null y) (equal x y)))
                   :from-end t)

---

Yes, I realize you said you didn't want to use cl. But I'm still mentioning this as the right way to do it for other people who might read this thread.

(Why is cl not viable for you anyway? It's been shipped with Emacs for about 20 years now, not counting less featured past incarnations.)

Answer 3

If you use dash.el library, that's all you need:

(-distinct (-non-nil '(1 1 nil 2 2 nil 3)) ; => (1 2 3)

dash.el is written by Magnar Sveen and it's a great list manipulation library with many functions for all kinds of tasks. I recommend to install it if you write lots of Elisp code. Function -distinct removes duplicate elements in a list, -non-nil removes nil elements. While the above code is sufficient, below I describe an alternative approache, so feel free to ignore the rest of the post.

-non-nil was added in version 2.9, so if for some reason you have to use earlier versions, another way to achieve the same is to use -keep with built-in identity function, which just returns whatever it is given: (identity 1) ; => 1. The idea is that -keep keeps only elements, for which the predicate returns true (“non-nil” in Lisp jargon). identity obviously returns non-nil only for whatever values that are not nil:

(-distinct (-keep 'identity '(1 1 nil 2 2 nil 3)) ; => (1 2 3)