Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove duplicates from a list (in vim)

This is my list:

['02', '03', '03', '16', '17', '17', '28', '29', '29']

I would like to know how I can remove the duplicates from this list.

Would it be also possible when I add an item to the list to check if the item is already in the list (to avoid duplicates?)

like image 557
Reman Avatar asked Jul 08 '11 21:07

Reman


People also ask

How do I remove all duplicates from a list?

To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of the set() method is that it returns distinct elements.

How do I remove duplicates from a list in Linux?

Remove duplicate lines with uniq If you don't need to preserve the order of the lines in the file, using the sort and uniq commands will do what you need in a very straightforward way. The sort command sorts the lines in alphanumeric order. The uniq command ensures that sequential identical lines are reduced to one.

How do I use uniquify lines in Vim?

:%! sort | uniq -u will do just that: sort, remove all lines that are not unique, and leave the result in the file.

How do I remove duplicates in Unix?

The uniq command in UNIX is a command line utility for reporting or filtering repeated lines in a file. It can remove duplicates, show a count of occurrences, show only repeated lines, ignore certain characters and compare on specific fields.


1 Answers

Try

let list=['02', '03', '03', '16', '17', '17', '28', '29', '29']
let unduplist=filter(copy(list), 'index(list, v:val, v:key+1)==-1')

. For the second question, see :h index().

By the way, if

  1. all list items are strings;
  2. there is no empty strings possible;
  3. you don't care about order of list items

then you should probably use a Dictionary instead: for large number of strings searching for duplicates is faster (and really not required).

like image 119
ZyX Avatar answered Sep 18 '22 03:09

ZyX