Menu
Is there a possibility to remove duplicates in a list of words without sorting in a makefile?
$(sort foo bar lose) does remove duplicates (which is for me the main functionality in this case), but also sorts (for me an unfortunate side effect in this case). I want to avoid that.
[update]
bobbogo's answer works very nicely. Just remember to use define uniq for v3.81 and (did not check this) define uniq = for later versions.
larsmans' answer works very nicely too if your record separator is not a space, e.g. if you want to remove duplicates from _foo_bar_lose_lose_bar_baz_ or the like. Just remember to use the RS and ORS awk options instead of tr, and wrap it all with $(firstword $(shell ... ))
429
We've learned we can use the compact one-liner awk '!a[$0]++' input to remove duplicate lines from an input file. Here, the $0 means the whole line.
Remove duplicate lines with uniq If you don't need to preserve the order of the lines in the file, using the sort and uniq commands will do what you need in a very straightforward way. The sort command sorts the lines in alphanumeric order. The uniq command ensures that sequential identical lines are reduced to one.
Removing Duplicate Lines With Sort, Uniq and Shell Pipes -u : check for strict ordering, remove all duplicate lines.
The uniq command in UNIX is a command line utility for reporting or filtering repeated lines in a file. It can remove duplicates, show a count of occurrences, show only repeated lines, ignore certain characters and compare on specific fields.
Boring $eval based method:
define uniq = $(eval seen :=) $(foreach _,$1,$(if $(filter $_,${seen}),,$(eval seen += $_))) ${seen} endef w := z z x x y c x $(info $(sort $w)) $(info $(call uniq,$w)) Extremely fiendish make standard library recursive call (recursive make considered extremely fiendish?):
uniq = $(if $1,$(firstword $1) $(call uniq,$(filter-out $(firstword $1),$1))) It's worth noting that no variables are damaged in this second formulation (see seen in the first). It is preferable just for that (given the lack of locals in make)!
My obscure comment about recursive make above seems to have muddied the waters somewhat. "Recursive" in the context of this post means recursive function. It really has nothing to do with the execrable recursive make.
The latter (recursive) definition of uniq is extremely nice, performant, small, and is definitely the one to use.
113
Depends on where you need it and whether you use GNU make. If you just want to uniq the list of target prerequisites, it's as easy as (http://www.gnu.org/software/make/manual/make.html#Quick-Reference) :
The value of $^ omits duplicate prerequisites, while $+ retains them and preserves their order.
So, a rule like
exe: $(OBJS) $(LD) -o $@ $^ will filter duplicates from $(OBJS) automagically, while still leaving order of other items the same.
43
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
